You can not select more than 25 topics Topics must start with a letter or number, can include dashes ('-') and can be up to 35 characters long.

328 lines
13 KiB

9 years ago
MDEV-12266: Change dict_table_t::space to fil_space_t* InnoDB always keeps all tablespaces in the fil_system cache. The fil_system.LRU is only for closing file handles; the fil_space_t and fil_node_t for all data files will remain in main memory. Between startup to shutdown, they can only be created and removed by DDL statements. Therefore, we can let dict_table_t::space point directly to the fil_space_t. dict_table_t::space_id: A numeric tablespace ID for the corner cases where we do not have a tablespace. The most prominent examples are ALTER TABLE...DISCARD TABLESPACE or a missing or corrupted file. There are a few functional differences; most notably: (1) DROP TABLE will delete matching .ibd and .cfg files, even if they were not attached to the data dictionary. (2) Some error messages will report file names instead of numeric IDs. There still are many functions that use numeric tablespace IDs instead of fil_space_t*, and many functions could be converted to fil_space_t member functions. Also, Tablespace and Datafile should be merged with fil_space_t and fil_node_t. page_id_t and buf_page_get_gen() could use fil_space_t& instead of a numeric ID, and after moving to a single buffer pool (MDEV-15058), buf_pool_t::page_hash could be moved to fil_space_t::page_hash. FilSpace: Remove. Only few calls to fil_space_acquire() will remain, and gradually they should be removed. mtr_t::set_named_space_id(ulint): Renamed from set_named_space(), to prevent accidental calls to this slower function. Very few callers remain. fseg_create(), fsp_reserve_free_extents(): Take fil_space_t* as a parameter instead of a space_id. fil_space_t::rename(): Wrapper for fil_rename_tablespace_check(), fil_name_write_rename(), fil_rename_tablespace(). Mariabackup passes the parameter log=false; InnoDB passes log=true. dict_mem_table_create(): Take fil_space_t* instead of space_id as parameter. dict_process_sys_tables_rec_and_mtr_commit(): Replace the parameter 'status' with 'bool cached'. dict_get_and_save_data_dir_path(): Avoid copying the fil_node_t::name. fil_ibd_open(): Return the tablespace. fil_space_t::set_imported(): Replaces fil_space_set_imported(). truncate_t: Change many member function parameters to fil_space_t*, and remove page_size parameters. row_truncate_prepare(): Merge to its only caller. row_drop_table_from_cache(): Assert that the table is persistent. dict_create_sys_indexes_tuple(): Write SYS_INDEXES.SPACE=FIL_NULL if the tablespace has been discarded. row_import_update_discarded_flag(): Remove a constant parameter.
8 years ago
Merge 10.1 into 10.2 This only merges MDEV-12253, adapting it to MDEV-12602 which is already present in 10.2 but not yet in the 10.1 revision that is being merged. TODO: Error handling in crash recovery needs to be improved. If a page cannot be decrypted (or read), we should cleanly abort the startup. If innodb_force_recovery is specified, we should ignore the problematic page and apply redo log to other pages. Currently, the test encryption.innodb-redo-badkey randomly fails like this (the last messages are from cmake -DWITH_ASAN): 2017-05-05 10:19:40 140037071685504 [Note] InnoDB: Starting crash recovery from checkpoint LSN=1635994 2017-05-05 10:19:40 140037071685504 [ERROR] InnoDB: Missing MLOG_FILE_NAME or MLOG_FILE_DELETE before MLOG_CHECKPOINT for tablespace 1 2017-05-05 10:19:40 140037071685504 [ERROR] InnoDB: Plugin initialization aborted at srv0start.cc[2201] with error Data structure corruption 2017-05-05 10:19:41 140037071685504 [Note] InnoDB: Starting shutdown... i================================================================= ==5226==ERROR: AddressSanitizer: attempting free on address which was not malloc()-ed: 0x612000018588 in thread T0 #0 0x736750 in operator delete(void*) (/mariadb/server/build/sql/mysqld+0x736750) #1 0x1e4833f in LatchCounter::~LatchCounter() /mariadb/server/storage/innobase/include/sync0types.h:599:4 #2 0x1e480b8 in LatchMeta<LatchCounter>::~LatchMeta() /mariadb/server/storage/innobase/include/sync0types.h:786:17 #3 0x1e35509 in sync_latch_meta_destroy() /mariadb/server/storage/innobase/sync/sync0debug.cc:1622:3 #4 0x1e35314 in sync_check_close() /mariadb/server/storage/innobase/sync/sync0debug.cc:1839:2 #5 0x1dfdc18 in innodb_shutdown() /mariadb/server/storage/innobase/srv/srv0start.cc:2888:2 #6 0x197e5e6 in innobase_init(void*) /mariadb/server/storage/innobase/handler/ha_innodb.cc:4475:3
9 years ago
MDEV-12266: Change dict_table_t::space to fil_space_t* InnoDB always keeps all tablespaces in the fil_system cache. The fil_system.LRU is only for closing file handles; the fil_space_t and fil_node_t for all data files will remain in main memory. Between startup to shutdown, they can only be created and removed by DDL statements. Therefore, we can let dict_table_t::space point directly to the fil_space_t. dict_table_t::space_id: A numeric tablespace ID for the corner cases where we do not have a tablespace. The most prominent examples are ALTER TABLE...DISCARD TABLESPACE or a missing or corrupted file. There are a few functional differences; most notably: (1) DROP TABLE will delete matching .ibd and .cfg files, even if they were not attached to the data dictionary. (2) Some error messages will report file names instead of numeric IDs. There still are many functions that use numeric tablespace IDs instead of fil_space_t*, and many functions could be converted to fil_space_t member functions. Also, Tablespace and Datafile should be merged with fil_space_t and fil_node_t. page_id_t and buf_page_get_gen() could use fil_space_t& instead of a numeric ID, and after moving to a single buffer pool (MDEV-15058), buf_pool_t::page_hash could be moved to fil_space_t::page_hash. FilSpace: Remove. Only few calls to fil_space_acquire() will remain, and gradually they should be removed. mtr_t::set_named_space_id(ulint): Renamed from set_named_space(), to prevent accidental calls to this slower function. Very few callers remain. fseg_create(), fsp_reserve_free_extents(): Take fil_space_t* as a parameter instead of a space_id. fil_space_t::rename(): Wrapper for fil_rename_tablespace_check(), fil_name_write_rename(), fil_rename_tablespace(). Mariabackup passes the parameter log=false; InnoDB passes log=true. dict_mem_table_create(): Take fil_space_t* instead of space_id as parameter. dict_process_sys_tables_rec_and_mtr_commit(): Replace the parameter 'status' with 'bool cached'. dict_get_and_save_data_dir_path(): Avoid copying the fil_node_t::name. fil_ibd_open(): Return the tablespace. fil_space_t::set_imported(): Replaces fil_space_set_imported(). truncate_t: Change many member function parameters to fil_space_t*, and remove page_size parameters. row_truncate_prepare(): Merge to its only caller. row_drop_table_from_cache(): Assert that the table is persistent. dict_create_sys_indexes_tuple(): Write SYS_INDEXES.SPACE=FIL_NULL if the tablespace has been discarded. row_import_update_discarded_flag(): Remove a constant parameter.
8 years ago
  1. /*****************************************************************************
  2. Copyright (c) 1996, 2016, Oracle and/or its affiliates. All Rights Reserved.
  3. Copyright (c) 2017, 2018, MariaDB Corporation.
  4. This program is free software; you can redistribute it and/or modify it under
  5. the terms of the GNU General Public License as published by the Free Software
  6. Foundation; version 2 of the License.
  7. This program is distributed in the hope that it will be useful, but WITHOUT
  8. ANY WARRANTY; without even the implied warranty of MERCHANTABILITY or FITNESS
  9. FOR A PARTICULAR PURPOSE. See the GNU General Public License for more details.
  10. You should have received a copy of the GNU General Public License along with
  11. this program; if not, write to the Free Software Foundation, Inc.,
  12. 51 Franklin Street, Suite 500, Boston, MA 02110-1335 USA
  13. *****************************************************************************/
  14. /**************************************************//**
  15. @file include/dict0load.h
  16. Loads to the memory cache database object definitions
  17. from dictionary tables
  18. Created 4/24/1996 Heikki Tuuri
  19. *******************************************************/
  20. #ifndef dict0load_h
  21. #define dict0load_h
  22. #include "univ.i"
  23. #include "dict0types.h"
  24. #include "trx0types.h"
  25. #include "ut0byte.h"
  26. #include "mem0mem.h"
  27. #include "btr0types.h"
  28. #include "ut0new.h"
  29. #include <deque>
  30. /** A stack of table names related through foreign key constraints */
  31. typedef std::deque<const char*, ut_allocator<const char*> > dict_names_t;
  32. /** enum that defines all system table IDs. @see SYSTEM_TABLE_NAME[] */
  33. enum dict_system_id_t {
  34. SYS_TABLES = 0,
  35. SYS_INDEXES,
  36. SYS_COLUMNS,
  37. SYS_FIELDS,
  38. SYS_FOREIGN,
  39. SYS_FOREIGN_COLS,
  40. SYS_TABLESPACES,
  41. SYS_DATAFILES,
  42. SYS_VIRTUAL,
  43. /* This must be last item. Defines the number of system tables. */
  44. SYS_NUM_SYSTEM_TABLES
  45. };
  46. /** Check each tablespace found in the data dictionary.
  47. Look at each table defined in SYS_TABLES that has a space_id > 0.
  48. If the tablespace is not yet in the fil_system cache, look up the
  49. tablespace in SYS_DATAFILES to ensure the correct path.
  50. In a crash recovery we already have some tablespace objects created from
  51. processing the REDO log. Any other tablespace in SYS_TABLESPACES not
  52. previously used in recovery will be opened here. We will compare the
  53. space_id information in the data dictionary to what we find in the
  54. tablespace file. In addition, more validation will be done if recovery
  55. was needed and force_recovery is not set.
  56. We also scan the biggest space id, and store it to fil_system.
  57. @param[in] validate true if recovery was needed */
  58. void
  59. dict_check_tablespaces_and_store_max_id(
  60. bool validate);
  61. /********************************************************************//**
  62. Finds the first table name in the given database.
  63. @return own: table name, NULL if does not exist; the caller must free
  64. the memory in the string! */
  65. char*
  66. dict_get_first_table_name_in_db(
  67. /*============================*/
  68. const char* name); /*!< in: database name which ends to '/' */
  69. /** Get the first filepath from SYS_DATAFILES for a given space_id.
  70. @param[in] space_id Tablespace ID
  71. @return First filepath (caller must invoke ut_free() on it)
  72. @retval NULL if no SYS_DATAFILES entry was found. */
  73. char*
  74. dict_get_first_path(
  75. ulint space_id);
  76. /** Make sure the data_file_name is saved in dict_table_t if needed.
  77. Try to read it from the fil_system first, then from SYS_DATAFILES.
  78. @param[in] table Table object
  79. @param[in] dict_mutex_own true if dict_sys->mutex is owned already */
  80. void
  81. dict_get_and_save_data_dir_path(
  82. dict_table_t* table,
  83. bool dict_mutex_own);
  84. /** Loads a table definition and also all its index definitions, and also
  85. the cluster definition if the table is a member in a cluster. Also loads
  86. all foreign key constraints where the foreign key is in the table or where
  87. a foreign key references columns in this table.
  88. @param[in] name Table name in the dbname/tablename format
  89. @param[in] cached true=add to cache, false=do not
  90. @param[in] ignore_err Error to be ignored when loading
  91. table and its index definition
  92. @return table, NULL if does not exist; if the table is stored in an
  93. .ibd file, but the file does not exist, then we set the file_unreadable
  94. flag in the table object we return. */
  95. dict_table_t*
  96. dict_load_table(
  97. const char* name,
  98. bool cached,
  99. dict_err_ignore_t ignore_err);
  100. /***********************************************************************//**
  101. Loads a table object based on the table id.
  102. @return table; NULL if table does not exist */
  103. dict_table_t*
  104. dict_load_table_on_id(
  105. /*==================*/
  106. table_id_t table_id, /*!< in: table id */
  107. dict_err_ignore_t ignore_err); /*!< in: errors to ignore
  108. when loading the table */
  109. /********************************************************************//**
  110. This function is called when the database is booted.
  111. Loads system table index definitions except for the clustered index which
  112. is added to the dictionary cache at booting before calling this function. */
  113. void
  114. dict_load_sys_table(
  115. /*================*/
  116. dict_table_t* table); /*!< in: system table */
  117. /***********************************************************************//**
  118. Loads foreign key constraints where the table is either the foreign key
  119. holder or where the table is referenced by a foreign key. Adds these
  120. constraints to the data dictionary.
  121. The foreign key constraint is loaded only if the referenced table is also
  122. in the dictionary cache. If the referenced table is not in dictionary
  123. cache, then it is added to the output parameter (fk_tables).
  124. @return DB_SUCCESS or error code */
  125. dberr_t
  126. dict_load_foreigns(
  127. /*===============*/
  128. const char* table_name, /*!< in: table name */
  129. const char** col_names, /*!< in: column names, or NULL
  130. to use table->col_names */
  131. bool check_recursive,/*!< in: Whether to check
  132. recursive load of tables
  133. chained by FK */
  134. bool check_charsets, /*!< in: whether to check
  135. charset compatibility */
  136. dict_err_ignore_t ignore_err, /*!< in: error to be ignored */
  137. dict_names_t& fk_tables) /*!< out: stack of table names
  138. which must be loaded
  139. subsequently to load all the
  140. foreign key constraints. */
  141. MY_ATTRIBUTE((nonnull(1), warn_unused_result));
  142. /********************************************************************//**
  143. This function opens a system table, and return the first record.
  144. @return first record of the system table */
  145. const rec_t*
  146. dict_startscan_system(
  147. /*==================*/
  148. btr_pcur_t* pcur, /*!< out: persistent cursor to
  149. the record */
  150. mtr_t* mtr, /*!< in: the mini-transaction */
  151. dict_system_id_t system_id); /*!< in: which system table to open */
  152. /********************************************************************//**
  153. This function get the next system table record as we scan the table.
  154. @return the record if found, NULL if end of scan. */
  155. const rec_t*
  156. dict_getnext_system(
  157. /*================*/
  158. btr_pcur_t* pcur, /*!< in/out: persistent cursor
  159. to the record */
  160. mtr_t* mtr); /*!< in: the mini-transaction */
  161. /********************************************************************//**
  162. This function processes one SYS_TABLES record and populate the dict_table_t
  163. struct for the table.
  164. @return error message, or NULL on success */
  165. const char*
  166. dict_process_sys_tables_rec_and_mtr_commit(
  167. /*=======================================*/
  168. mem_heap_t* heap, /*!< in: temporary memory heap */
  169. const rec_t* rec, /*!< in: SYS_TABLES record */
  170. dict_table_t** table, /*!< out: dict_table_t to fill */
  171. bool cached, /*!< in: whether to load from cache */
  172. mtr_t* mtr); /*!< in/out: mini-transaction,
  173. will be committed */
  174. /********************************************************************//**
  175. This function parses a SYS_INDEXES record and populate a dict_index_t
  176. structure with the information from the record. For detail information
  177. about SYS_INDEXES fields, please refer to dict_boot() function.
  178. @return error message, or NULL on success */
  179. const char*
  180. dict_process_sys_indexes_rec(
  181. /*=========================*/
  182. mem_heap_t* heap, /*!< in/out: heap memory */
  183. const rec_t* rec, /*!< in: current SYS_INDEXES rec */
  184. dict_index_t* index, /*!< out: dict_index_t to be
  185. filled */
  186. table_id_t* table_id); /*!< out: table id */
  187. /********************************************************************//**
  188. This function parses a SYS_COLUMNS record and populate a dict_column_t
  189. structure with the information from the record.
  190. @return error message, or NULL on success */
  191. const char*
  192. dict_process_sys_columns_rec(
  193. /*=========================*/
  194. mem_heap_t* heap, /*!< in/out: heap memory */
  195. const rec_t* rec, /*!< in: current SYS_COLUMNS rec */
  196. dict_col_t* column, /*!< out: dict_col_t to be filled */
  197. table_id_t* table_id, /*!< out: table id */
  198. const char** col_name, /*!< out: column name */
  199. ulint* nth_v_col); /*!< out: if virtual col, this is
  200. records its sequence number */
  201. /** This function parses a SYS_VIRTUAL record and extract virtual column
  202. information
  203. @param[in,out] heap heap memory
  204. @param[in] rec current SYS_COLUMNS rec
  205. @param[in,out] table_id table id
  206. @param[in,out] pos virtual column position
  207. @param[in,out] base_pos base column position
  208. @return error message, or NULL on success */
  209. const char*
  210. dict_process_sys_virtual_rec(
  211. mem_heap_t* heap,
  212. const rec_t* rec,
  213. table_id_t* table_id,
  214. ulint* pos,
  215. ulint* base_pos);
  216. /********************************************************************//**
  217. This function parses a SYS_FIELDS record and populate a dict_field_t
  218. structure with the information from the record.
  219. @return error message, or NULL on success */
  220. const char*
  221. dict_process_sys_fields_rec(
  222. /*========================*/
  223. mem_heap_t* heap, /*!< in/out: heap memory */
  224. const rec_t* rec, /*!< in: current SYS_FIELDS rec */
  225. dict_field_t* sys_field, /*!< out: dict_field_t to be
  226. filled */
  227. ulint* pos, /*!< out: Field position */
  228. index_id_t* index_id, /*!< out: current index id */
  229. index_id_t last_id); /*!< in: previous index id */
  230. /********************************************************************//**
  231. This function parses a SYS_FOREIGN record and populate a dict_foreign_t
  232. structure with the information from the record. For detail information
  233. about SYS_FOREIGN fields, please refer to dict_load_foreign() function
  234. @return error message, or NULL on success */
  235. const char*
  236. dict_process_sys_foreign_rec(
  237. /*=========================*/
  238. mem_heap_t* heap, /*!< in/out: heap memory */
  239. const rec_t* rec, /*!< in: current SYS_FOREIGN rec */
  240. dict_foreign_t* foreign); /*!< out: dict_foreign_t to be
  241. filled */
  242. /********************************************************************//**
  243. This function parses a SYS_FOREIGN_COLS record and extract necessary
  244. information from the record and return to caller.
  245. @return error message, or NULL on success */
  246. const char*
  247. dict_process_sys_foreign_col_rec(
  248. /*=============================*/
  249. mem_heap_t* heap, /*!< in/out: heap memory */
  250. const rec_t* rec, /*!< in: current SYS_FOREIGN_COLS rec */
  251. const char** name, /*!< out: foreign key constraint name */
  252. const char** for_col_name, /*!< out: referencing column name */
  253. const char** ref_col_name, /*!< out: referenced column name
  254. in referenced table */
  255. ulint* pos); /*!< out: column position */
  256. /********************************************************************//**
  257. This function parses a SYS_TABLESPACES record, extracts necessary
  258. information from the record and returns to caller.
  259. @return error message, or NULL on success */
  260. const char*
  261. dict_process_sys_tablespaces(
  262. /*=========================*/
  263. mem_heap_t* heap, /*!< in/out: heap memory */
  264. const rec_t* rec, /*!< in: current SYS_TABLESPACES rec */
  265. ulint* space, /*!< out: pace id */
  266. const char** name, /*!< out: tablespace name */
  267. ulint* flags); /*!< out: tablespace flags */
  268. /********************************************************************//**
  269. This function parses a SYS_DATAFILES record, extracts necessary
  270. information from the record and returns to caller.
  271. @return error message, or NULL on success */
  272. const char*
  273. dict_process_sys_datafiles(
  274. /*=======================*/
  275. mem_heap_t* heap, /*!< in/out: heap memory */
  276. const rec_t* rec, /*!< in: current SYS_DATAFILES rec */
  277. ulint* space, /*!< out: pace id */
  278. const char** path); /*!< out: datafile path */
  279. /** Update the record for space_id in SYS_TABLESPACES to this filepath.
  280. @param[in] space_id Tablespace ID
  281. @param[in] filepath Tablespace filepath
  282. @return DB_SUCCESS if OK, dberr_t if the insert failed */
  283. dberr_t
  284. dict_update_filepath(
  285. ulint space_id,
  286. const char* filepath);
  287. /** Replace records in SYS_TABLESPACES and SYS_DATAFILES associated with
  288. the given space_id using an independent transaction.
  289. @param[in] space_id Tablespace ID
  290. @param[in] name Tablespace name
  291. @param[in] filepath First filepath
  292. @param[in] fsp_flags Tablespace flags
  293. @return DB_SUCCESS if OK, dberr_t if the insert failed */
  294. dberr_t
  295. dict_replace_tablespace_and_filepath(
  296. ulint space_id,
  297. const char* name,
  298. const char* filepath,
  299. ulint fsp_flags);
  300. #endif