mariadb

Commit Graph

Author	SHA1	Message	Date
Marko Mäkelä	3d4a801533	MDEV-12353 preparation: Replace mtr_x_lock() and friends Apart from page latches (buf_block_t::lock), mini-transactions are keeping track of at most one dict_index_t::lock and fil_space_t::latch at a time, and in a rare case, purge_sys.latch. Let us introduce interfaces for acquiring an index latch or a tablespace latch. In a later version, we may want to introduce mtr_t members for holding a latched dict_index_t* and fil_space_t, and replace the remaining use of mtr_t::m_memo with std::set<buf_block_t> or with a map<buf_block_t,byte> pointing to log records.	6 years ago
Marko Mäkelä	49019dde65	MDEV-17138 follow-up: Optimize index page creation btr_create(), btr_root_raise_and_insert(): Write a MLOG_MEMSET record to set FIL_PAGE_PREV,FIL_PAGE_NEXT to FIL_NULL, instead of writing two MLOG_4BYTES records. For ROW_FORMAT=COMPRESSED pages, we will not use MLOG_MEMSET because we want the crash-downgrade to earlier 10.4 releases to succeed. mlog_parse_nbytes(): Relax the too strict assertion. There is no problem with MLOG_MEMSET records that affect the uncompressed header of ROW_FORMAT=COMPRESSED index pages.	6 years ago
Marko Mäkelä	33f74e8fcf	MDEV-21024: Clean up IMPORT TABLESPACE page_rec_write_field(): Remove. dict_create_index_tree_step(): If the SYS_INDEXES.PAGE does not change, do not update it in the data dictionary. Typically, all index page numbers would be unchanged before and after IMPORT TABLESPACE, except if some secondary indexes were created after loading some data. btr_root_fseg_adjust_on_import(): Remove the redundant mtr_t* parameter. Redo logging is disabled during the page adjustments that IMPORT TABLESPACE is performing.	6 years ago
Marko Mäkelä	dfdd96214b	MDEV-21024: Clean up btr_root_raise_and_insert() The root page must never have any siblings, so it is unnecessary to clear those fields.	6 years ago
Marko Mäkelä	29d67d051a	Cleanup btr_page_get_prev(), btr_page_get_next() Remove the redundant parameter mtr_t*. Make use of page_has_prev(), page_has_next() whenever possible.	6 years ago
Marko Mäkelä	361e8284f3	MDEV-20813 Assertion failure in buf_flush_init_for_writing() for innodb_immediate_scrub_data_uncompressed=ON The assertion that was added in commit `c0c003beb4` to augment the fix of MDEV-20805 turns out to be invalid when innodb_immediate_scrub_data_uncompressed is enabled. In this mode, fsp_init_file_page() will be invoked on data pages that have been freed, causing writes of almost-all-zero pages. btr_page_free(): Adjust the comment. buf_flush_init_for_writing(): Disable the assertion with a note that it should be re-enabled in MDEV-15528.	6 years ago
Marko Mäkelä	4cdb72f237	MDEV-19783: Relax an assertion btr_page_get_split_rec_to_left(): Assert that in the leftmost leaf page, if the metadata record exists, index->is_instant() must hold. The assertion of commit `01f45becd1` could fail during innobase_instant_try().	6 years ago
Marko Mäkelä	01f45becd1	MDEV-19783: Add more assertions btr_page_get_split_rec_to_left(): Assert that in the leftmost leaf page, the metadata record exists if and only if index->is_instant(). page_validate(): Correct the wording of a message. rec_init_offsets(): Assert that whenever a record is in "instant ALTER" format, index->is_instant() must hold.	6 years ago
Marko Mäkelä	6d7a826953	MDEV-20788: Bogus assertion failure for PAGE_FREE list In MDEV-11369 (instant ADD COLUMN) in MariaDB Server 10.3, we introduced the hidden metadata record that must be the first record in the clustered index if and only if index->is_instant() holds. To catch MDEV-19783, in commit `ed0793e096` and commit `99dc40d6ac` we added some assertions to find cases where the metadata record is missing while it should not be, or a record exists when it should not. Those assertions were invalid when traversing the PAGE_FREE list. That list can contain anything; we must only be able to determine the successor and the size of each garbage record in it. page_validate(), page_simple_validate_old(), page_simple_validate_new(): Do not invoke page_rec_get_next_const() for traversing the PAGE_FREE list, but instead use a lower-level accessor that does not attempt to validate the REC_INFO_MIN_REC_FLAG. page_copy_rec_list_end_no_locks(), page_copy_rec_list_start(), page_delete_rec_list_start(): Add assertions. btr_page_get_split_rec_to_left(): Remove a redundant return value, and make the output parameter the return value. btr_page_get_split_rec_to_right(), btr_page_split_and_insert(): Clean up.	6 years ago
Eugene Kosov	ed0793e096	MDEV-19783: Add more REC_INFO_MIN_REC_FLAG checks btr_cur_pessimistic_delete(): code changed in a way that allows to put more REC_INFO_MIN_REC_FLAG assertions inside btr_set_min_rec_mark(). Without that change tests innodb.innodb-table-online, innodb.temp_table_savepoint and innodb_zip.prefix_index_liftedlimit fail. Removed basically duplicated page_zip_validate() calls which fails because of temporary(!) invariant violation. That fixed innodb_zip.wl5522_debug_zip and innodb_zip.prefix_index_liftedlimit	6 years ago
Marko Mäkelä	d480d28f4f	Add page_has_prev(), page_has_next(), page_has_siblings() Until now, InnoDB inefficiently compared the aligned fields FIL_PAGE_PREV, FIL_PAGE_NEXT to the byte-order-agnostic value FIL_NULL. This is a backport of `32170f8c6d` from MariaDB Server 10.3.	6 years ago
Marko Mäkelä	0e1ba364a1	MDEV-19916 Corruption after instant ADD/DROP and shrinking the tree btr_lift_page_up(): Correct the incorrect condition. page_validate(): Validate the page type.	7 years ago
Marko Mäkelä	304ae942f7	MDEV-15528 preparation: Write MLOG_INIT_FREE_PAGE When freeing a file page, write a MLOG_INIT_FREE_PAGE record. This allows us to avoid page flush and instead punch holes later, in the page flushing. To implement that, we may want to make buf_page_t::file_page_was_freed available in non-debug builds. Crash recovery can choose to ignore or apply the record. In BtrBulk::finish() we must not write this record, because redo logging is being disabled for the page.	7 years ago
Marko Mäkelä	ee7a4f4462	MDEV-12266: Pass fil_space_t* to fseg_free_page() fseg_free_page_func(): Avoid an unnecessary tablespace ID lookup. The callers should pass the tablespace that they already know.	7 years ago
Marko Mäkelä	4b822111ef	MDEV-8139: Clean up the freeing of B-tree pages btr_page_free(): Renamed from btr_page_free_low(). If scrubbing is enabled, zero out the page with proper redo logging. Only pass ahi=true to fseg_free_page() if the page is actually indexed. fil_space_t::modify_check(): Renamed from fsp_space_modify_check(). fsp_init_file_page(): Define inline.	7 years ago
Marko Mäkelä	80f29211eb	Fix a crash in CHECK TABLE for corrupted encrypted root page btr_root_get(): Ignore the root->page.encrypted flag. The purpose of this flag is questionable since commit `8c43f96388`. btr_validate_index(): Avoid crash if btr_root_get() returns NULL.	7 years ago
Marko Mäkelä	1d0380e029	MDEV-15528 preparation: Do not modify a freed page btr_free_root(): Add the parameter bool invalidate. btr_free_root_invalidate(): Remove.	7 years ago
Marko Mäkelä	f602385776	Do not pass table_name_t to printf-like functions	7 years ago
Marko Mäkelä	7984ea80de	Remove a useless CHECK TABLE printout for debug builds	7 years ago
Marko Mäkelä	525e79b057	MDEV-19022: InnoDB fails to cleanup useless B-tree pages The test case for reproducing MDEV-14126 demonstrates that InnoDB can end up with an index tree where a non-leaf page has only one child page. The test case innodb.innodb_bug14676111 demonstrates that such pages are sometimes unavoidable, because InnoDB does not implement any sort of B-tree rotation. But, there is no reason to allow a root page with only one child page. btr_cur_node_ptr_delete(): Replaces btr_node_ptr_delete(). btr_page_get_father(): Declare globally. btr_discard_only_page_on_level(): Declare with ATTRIBUTE_COLD. It turns out that this function is not covered by the innodb.innodb_bug14676111 test case after all. btr_discard_page(): If the root page ends up having only one child page, shrink the tree by invoking btr_lift_page_up().	7 years ago
Marko Mäkelä	b59d484696	MDEV-14126: Remove page_is_root() The predicate page_is_root(), which was added in MariaDB Server 10.2.2, is based on a wrong assumption. Under some circumstances, InnoDB can transform B-trees into a degenerate state where a non-leaf page has no sibling pages. Because of this, we cannot assume that a page that has no siblings is the root page. This bug will be tracked as MDEV-19022. Because of the bug that may affect many InnoDB data files, we must remove and replace the wrong predicate. Using the wrong predicate can cause corruption. A leaf page is not allowed to be empty except if it is the root page, and the entire table is empty.	7 years ago
Marko Mäkelä	2151aed44d	Follow-up fix to MDEV-12026: FIL_SPACE_FLAGS trump fil_space_t::flags Whenever we are reading the first page of a data file, we may have to adjust the provisionally created fil_space_t::flags to match what is actually inside the data files. In this way, we will never accidentally change the format of a data file. fil_node_t::read_page0(): After validating the FIL_SPACE_FLAGS, always assign them to space->flags. btr_root_adjust_on_import(), Datafile::validate_to_dd(), fil_space_for_table_exists_in_mem(): Adapt to the fix in fil_node_t::read_page0(). fsp_flags_try_adjust(): Skip the adjustment if full_crc32 is being used. This adjustment was introduced in MDEV-11623 for upgrading from MariaDB 10.1.0 to 10.1.20, which used an accidentally changed format of FIL_SPACE_FLAGS. MariaDB before 10.4.3 never set the flag that now indicates the full_crc32 format.	7 years ago
Marko Mäkelä	4932aba921	MDEV-18649 Assertion supremum[7] == index.n_core_null_bytes failed This is follow-up to MDEV-18048: Relax a too strict debug assertion This assertion should have been relaxed when implementing the first part of MDEV-15563: instant removal of NOT NULL attribute for ROW_FORMAT=REDUNDANT tables. For ROW_FORMAT=REDUNDANT, there is no bitmap of null columns; the null flags are encoded in the end offset of each field. We do not really care about the number of fields that can be NULL.	7 years ago
Marko Mäkelä	2c74799d64	MDEV-18627 Wrong result after instant size change of integer If we instantly change the size of a fixed-length field and treat it as kind-of variable-length, then we will need conversions between old column values and new ones. I tried adding such a conversion to row_build(), but then I noticed that more conversions would be needed, because old values still appeared in a freshly rebuilt secondary index, causing a mismatch when trying to search with the correct longer value that was converted in my provisional fix to row_build(). So, we will revert the essential part of MDEV-15563: Instant ROW_FORMAT=REDUNDANT column extension (commit `22feb179ae`), but not remove any tests.	7 years ago
Marko Mäkelä	22feb179ae	MDEV-15563: Instant ROW_FORMAT=REDUNDANT column extension This was developed by Aleksey Midenkov based on my design. In the original InnoDB storage format (that was retroactively named ROW_FORMAT=REDUNDANT in MySQL 5.0.3), the length of each index field is stored explicitly. Because of this, we can and now will allow instant conversion from VARCHAR to CHAR or VARBINARY to BINARY of equal or greater size, as well as instant conversion of TINYINT to SMALLINT to MEDIUMINT to INT to BIGINT (while not changing between signed and unsigned). Theoretically, we could allow changing from an unsigned integer to a bigger unsigned integer, as well as changing CHAR to VARCHAR, but that would require additional metadata and conversions whenever reading old records. Field_str::is_equal(), Field_varstring::is_equal(), Field_num::is_equal(): Return the new result IS_EQUAL_PACK_LENGTH_EXT if the table advertises HA_EXTENDED_TYPES_CONVERSION capability and we are considering the above-mentioned conversions. ALTER_COLUMN_EQUAL_PACK_LENGTH_EXT: A new ALTER TABLE flag, similar to ALTER_COLUMN_EQUAL_PACK_LENGTH but requiring conversions when reading the data. The Field::is_equal() result IS_EQUAL_PACK_LENGTH_EXT will map to this flag. dtype_get_fixed_size_low(): For BINARY, CHAR and integer columns in ROW_FORMAT=REDUNDANT, return 0 (variable length) from now on. dtype_get_sql_null_size(): Keep returning the current size for BINARY, CHAR and integer columns, so that in ROW_FORMAT=REDUNDANT it will remain possible to update in place between NULL and NOT NULL values. btr_index_rec_validate(): Relax a CHECK TABLE length check for ROW_FORMAT=REDUNDANT tables. btr_cur_instant_init_low(): No longer trust fixed_len for ROW_FORMAT=REDUNDANT tables. We cannot rely on fixed_len anymore because the record can have shorter length from before instant extension. Note that importing such tablespace into earlier MariaDB versions produces ER_TABLE_SCHEMA_MISMATCH when using a .cfg file.	7 years ago
Marko Mäkelä	0a1c3477bf	MDEV-18493 Remove page_size_t MySQL 5.7 introduced the class page_size_t and increased the size of buffer pool page descriptors by introducing this object to them. Maybe the intention of this exercise was to prepare for a future where the buffer pool could accommodate multiple page sizes. But that future never arrived, not even in MySQL 8.0. It is much easier to manage a pool of a single page size, and typically all storage devices of an InnoDB instance benefit from using the same page size. Let us remove page_size_t from MariaDB Server. This will make it easier to remove support for ROW_FORMAT=COMPRESSED (or make it a compile-time option) in the future, just by removing various occurrences of zip_size.	7 years ago
Sergey Vojtovich	c66db377d7	MDEV-17441 - InnoDB transition to C++11 atomics Replaced srv_fatal_semaphore_wait_threshold juggling with btr_validate_index_running counter. Removed last argument of btr_validate_index(): always false. Simplified away btr_validate_spatial_index().	7 years ago
Marko Mäkelä	bf096f2919	Fix btr_corruption_report() declaration on Windows The Microsoft compiler only allows function attributes before the function signature, not after it.	7 years ago
Marko Mäkelä	61967b90d7	btr_corruption_report(): Add ATTRIBUTE_NORETURN	7 years ago
Marko Mäkelä	06e5f28f9f	MDEV-12266: Remove a level of pointer indirection Replace table->space->id with table->space_id.	7 years ago
Marko Mäkelä	ff88e4bb8a	Remove many redundant #include from InnoDB	7 years ago
Eugene Kosov	14be814380	MDEV-17491 micro optimize page_id_t page_id_t: remove m_fold member various places: pass page_id_t by value instead of by reference	7 years ago
Marko Mäkelä	0e5a4ac253	MDEV-15662 Instant DROP COLUMN or changing the order of columns Allow ADD COLUMN anywhere in a table, not only adding as the last column. Allow instant DROP COLUMN and instant changing the order of columns. The added columns will always be added last in clustered index records. In new records, instantly dropped columns will be stored as NULL or empty when possible. Information about dropped and reordered columns will be written in a metadata BLOB (mblob), which is stored before the first 'user' field in the hidden metadata record at the start of the clustered index. The presence of mblob is indicated by setting the delete-mark flag in the metadata record. The metadata BLOB stores the number of clustered index fields, followed by an array of column information for each field. For dropped columns, we store the NOT NULL flag, the fixed length, and for variable-length columns, whether the maximum length exceeded 255 bytes. For non-dropped columns, we store the column position. Unlike with MDEV-11369, when a table becomes empty, it cannot be converted back to the canonical format. The reason for this is that other threads may hold cached objects such as row_prebuilt_t::ins_node that could refer to dropped or reordered index fields. For instant DROP COLUMN and ROW_FORMAT=COMPACT or ROW_FORMAT=DYNAMIC, we must store the n_core_null_bytes in the root page, so that the chain of node pointer records can be followed in order to reach the leftmost leaf page where the metadata record is located. If the mblob is present, we will zero-initialize the strings "infimum" and "supremum" in the root page, and use the last byte of "supremum" for storing the number of null bytes (which are allocated but useless on node pointer pages). This is necessary for btr_cur_instant_init_metadata() to be able to navigate to the mblob. If the PRIMARY KEY contains any variable-length column and some nullable columns were instantly dropped, the dict_index_t::n_nullable in the data dictionary could be smaller than it actually is in the non-leaf pages. Because of this, the non-leaf pages could use more bytes for the null flags than the data dictionary expects, and we could be reading the lengths of the variable-length columns from the wrong offset, and thus reading the child page number from wrong place. This is the result of two design mistakes that involve unnecessary storage of data: First, it is nonsense to store any data fields for the leftmost node pointer records, because the comparisons would be resolved by the MIN_REC_FLAG alone. Second, there cannot be any null fields in the clustered index node pointer fields, but we nevertheless reserve space for all the null flags. Limitations (future work): MDEV-17459 Allow instant ALTER TABLE even if FULLTEXT INDEX exists MDEV-17468 Avoid table rebuild on operations on generated columns MDEV-17494 Refuse ALGORITHM=INSTANT when the row size is too large btr_page_reorganize_low(): Preserve any metadata in the root page. Call lock_move_reorganize_page() only after restoring the "infimum" and "supremum" records, to avoid a memcmp() assertion failure. dict_col_t::DROPPED: Magic value for dict_col_t::ind. dict_col_t::clear_instant(): Renamed from dict_col_t::remove_instant(). Do not assert that the column was instantly added, because we sometimes call this unconditionally for all columns. Convert an instantly added column to a "core column". The old name remove_instant() could be mistaken to refer to "instant DROP COLUMN". dict_col_t::is_added(): Rename from dict_col_t::is_instant(). dtype_t::metadata_blob_init(): Initialize the mblob data type. dtuple_t::is_metadata(), dtuple_t::is_alter_metadata(), upd_t::is_metadata(), upd_t::is_alter_metadata(): Check if info_bits refer to a metadata record. dict_table_t::instant: Metadata about dropped or reordered columns. dict_table_t::prepare_instant(): Prepare ha_innobase_inplace_ctx::instant_table for instant ALTER TABLE. innobase_instant_try() will pass this to dict_table_t::instant_column(). On rollback, dict_table_t::rollback_instant() will be called. dict_table_t::instant_column(): Renamed from instant_add_column(). Add the parameter col_map so that columns can be reordered. Copy and adjust v_cols[] as well. dict_table_t::find(): Find an old column based on a new column number. dict_table_t::serialise_columns(), dict_table_t::deserialise_columns(): Convert the mblob. dict_index_t::instant_metadata(): Create the metadata record for instant ALTER TABLE. Invoke dict_table_t::serialise_columns(). dict_index_t::reconstruct_fields(): Invoked by dict_table_t::deserialise_columns(). dict_index_t::clear_instant_alter(): Move the fields for the dropped columns to the end, and sort the surviving index fields in ascending order of column position. ha_innobase::check_if_supported_inplace_alter(): Do not allow adding a FTS_DOC_ID column if a hidden FTS_DOC_ID column exists due to FULLTEXT INDEX. (This always required ALGORITHM=COPY.) instant_alter_column_possible(): Add a parameter for InnoDB table, to check for additional conditions, such as the maximum number of index fields. ha_innobase_inplace_ctx::first_alter_pos: The first column whose position is affected by instant ADD, DROP, or changing the order of columns. innobase_build_col_map(): Skip added virtual columns. prepare_inplace_add_virtual(): Correctly compute num_to_add_vcol. Remove some unnecessary code. Note that the call to innodb_base_col_setup() should be executed later. commit_try_norebuild(): If ctx->is_instant(), let the virtual columns be added or dropped by innobase_instant_try(). innobase_instant_try(): Fill in a zero default value for the hidden column FTS_DOC_ID (to reduce the work needed in MDEV-17459). If any columns were dropped or reordered (or added not last), delete any SYS_COLUMNS records for the following columns, and insert SYS_COLUMNS records for all subsequent stored columns as well as for all virtual columns. If any virtual column is dropped, rewrite all virtual column metadata. Use a shortcut only for adding virtual columns. This is because innobase_drop_virtual_try() assumes that the dropped virtual columns still exist in ctx->old_table. innodb_update_cols(): Renamed from innodb_update_n_cols(). innobase_add_one_virtual(), innobase_insert_sys_virtual(): Change the return type to bool, and invoke my_error() when detecting an error. innodb_insert_sys_columns(): Insert a record into SYS_COLUMNS. Refactored from innobase_add_one_virtual() and innobase_instant_add_col(). innobase_instant_add_col(): Replace the parameter dfield with type. innobase_instant_drop_cols(): Drop matching columns from SYS_COLUMNS and all columns from SYS_VIRTUAL. innobase_add_virtual_try(), innobase_drop_virtual_try(): Let the caller invoke innodb_update_cols(). innobase_rename_column_try(): Skip dropped columns. commit_cache_norebuild(): Update table->fts->doc_col. dict_mem_table_col_rename_low(): Skip dropped columns. trx_undo_rec_get_partial_row(): Skip dropped columns. trx_undo_update_rec_get_update(): Handle the metadata BLOB correctly. trx_undo_page_report_modify(): Avoid out-of-bounds access to record fields. Log metadata records consistently. Apparently, the first fields of a clustered index may be updated in an update_undo vector when the index is ID_IND of SYS_FOREIGN, as part of renaming the table during ALTER TABLE. Normally, updates of the PRIMARY KEY should be logged as delete-mark and an insert. row_undo_mod_parse_undo_rec(), row_purge_parse_undo_rec(): Use trx_undo_metadata. row_undo_mod_clust_low(): On metadata rollback, roll back the root page too. row_undo_mod_clust(): Relax an assertion. The delete-mark flag was repurposed for ALTER TABLE metadata records. row_rec_to_index_entry_impl(): Add the template parameter mblob and the optional parameter info_bits for specifying the desired new info bits. For the metadata tuple, allow conversion between the original format (ADD COLUMN only) and the generic format (with hidden BLOB). Add the optional parameter "pad" to determine whether the tuple should be padded to the index fields (on ALTER TABLE it should), or whether it should remain at its original size (on rollback). row_build_index_entry_low(): Clean up the code, removing redundant variables and conditions. For instantly dropped columns, generate a dummy value that is NULL, the empty string, or a fixed length of NUL bytes, depending on the type of the dropped column. row_upd_clust_rec_by_insert_inherit_func(): On the update of PRIMARY KEY of a record that contained a dropped column whose value was stored externally, we will be inserting a dummy NULL or empty string value to the field of the dropped column. The externally stored column would eventually be dropped when purge removes the delete-marked record for the old PRIMARY KEY value. btr_index_rec_validate(): Recognize the metadata record. btr_discard_only_page_on_level(): Preserve the generic instant ALTER TABLE metadata. btr_set_instant(): Replaces page_set_instant(). This sets a clustered index root page to the appropriate format, or upgrades from the MDEV-11369 instant ADD COLUMN to generic ALTER TABLE format. btr_cur_instant_init_low(): Read and validate the metadata BLOB page before reconstructing the dictionary information based on it. btr_cur_instant_init_metadata(): Do not read any lengths from the metadata record header before reading the BLOB. At this point, we would not actually know how many nullable fields the metadata record contains. btr_cur_instant_root_init(): Initialize n_core_null_bytes in one of two possible ways. btr_cur_trim(): Handle the mblob record. row_metadata_to_tuple(): Convert a metadata record to a data tuple, based on the new info_bits of the metadata record. btr_cur_pessimistic_update(): Invoke row_metadata_to_tuple() if needed. Invoke dtuple_convert_big_rec() for metadata records if the record is too large, or if the mblob is not yet marked as externally stored. btr_cur_optimistic_delete_func(), btr_cur_pessimistic_delete(): When the last user record is deleted, do not delete the generic instant ALTER TABLE metadata record. Only delete MDEV-11369 instant ADD COLUMN metadata records. btr_cur_optimistic_insert(): Avoid unnecessary computation of rec_size. btr_pcur_store_position(): Allow a logically empty page to contain a metadata record for generic ALTER TABLE. REC_INFO_DEFAULT_ROW_ADD: Renamed from REC_INFO_DEFAULT_ROW. This is for the old instant ADD COLUMN (MDEV-11369) only. REC_INFO_DEFAULT_ROW_ALTER: The more generic metadata record, with additional information for dropped or reordered columns. rec_info_bits_valid(): Remove. The only case when this would fail is when the record is the generic ALTER TABLE metadata record. rec_is_alter_metadata(): Check if a record is the metadata record for instant ALTER TABLE (other than ADD COLUMN). NOTE: This function must not be invoked on node pointer records, because the delete-mark flag in those records may be set (it is garbage), and then a debug assertion could fail because index->is_instant() does not necessarily hold. rec_is_add_metadata(): Check if a record is MDEV-11369 ADD COLUMN metadata record (not more generic instant ALTER TABLE). rec_get_converted_size_comp_prefix_low(): Assume that the metadata field will be stored externally. In dtuple_convert_big_rec() during the rec_get_converted_size() call, it would not be there yet. rec_get_converted_size_comp(): Replace status,fields,n_fields with tuple. rec_init_offsets_comp_ordinary(), rec_get_converted_size_comp_prefix_low(), rec_convert_dtuple_to_rec_comp(): Add template<bool mblob = false>. With mblob=true, process a record with a metadata BLOB. rec_copy_prefix_to_buf(): Assert that no fields beyond the key and system columns are being copied. Exclude the metadata BLOB field. rec_convert_dtuple_to_metadata_comp(): Convert an alter metadata tuple into a record. row_upd_index_replace_metadata(): Apply an update vector to an alter_metadata tuple. row_log_allocate(): Replace dict_index_t::is_instant() with a more appropriate condition that ignores dict_table_t::instant. Only a table on which the MDEV-11369 ADD COLUMN was performed can "lose its instantness" when it becomes empty. After instant DROP COLUMN or reordering columns, we cannot simply convert the table to the canonical format, because the data dictionary cache and all possibly existing references to it from other client connection threads would have to be adjusted. row_quiesce_write_index_fields(): Do not crash when the table contains an instantly dropped column. Thanks to Thirunarayanan Balathandayuthapani for discussing the design and implementing an initial prototype of this. Thanks to Matthias Leich for testing.	7 years ago
Marko Mäkelä	5567a8c936	MDEV-17138 Reduce redo log volume for undo tablespace initialization Implement a 10.4 redo log format, which extends the 10.3 format by introducing the MLOG_MEMSET record. MLOG_MEMSET: A new redo log record type for filling an area with a byte. mlog_memset(): Write the MLOG_MEMSET record. mlog_parse_nbytes(): Handle MLOG_MEMSET as well. trx_rseg_header_create(): Reduce the redo log volume by making use of mlog_memset() and the zero-initialization that happens inside page allocation. fil_addr_null: Remove. flst_init(): Create a variant that takes a zero-initialized buf_block_t* as a parameter, and only writes the FIL_NULL using mlog_memset(). flst_zero_addr(): A variant of flst_write_addr() that writes a null address using mlog_memset() for the FIL_NULL. The following fixes are replacing some use of MLOG_WRITE_STRING with the more compact MLOG_MEMSET record, or eliminating redundant redo log writes: btr_store_big_rec_extern_fields(): Invoke mlog_memset() for zero-initializing the tail of the ROW_FORMAT=COMPRESSED BLOB page. trx_sysf_create(), trx_rseg_format_upgrade(): Invoke mlog_memset() for zero-initializing the page trailer. fsp_header_init(), trx_rseg_header_create(): Remove redundant zero-initializations.	7 years ago
Marko Mäkelä	09af00cbde	MDEV-13564: Remove old crash-upgrade logic in 10.4 Stop supporting the additional trunc.log files that were introduced via MySQL 5.7 to MariaDB Server 10.2 and 10.3. DB_TABLESPACE_TRUNCATED: Remove. purge_sys.truncate: A new structure to track undo tablespace file truncation. srv_start(): Remove the call to buf_pool_invalidate(). It is no longer necessary, given that we no longer access things in ways that violate the ARIES protocol. This call was originally added for innodb_file_format, and it may later have been necessary for the proper function of the MySQL 5.7 TRUNCATE recovery, which we are now removing. trx_purge_cleanse_purge_queue(): Take the undo tablespace as a parameter. trx_purge_truncate_history(): Rewrite everything mostly in a single function, replacing references to undo::Truncate. recv_apply_hashed_log_recs(): If any redo log is to be applied, and if the log_sys.log.subformat indicates that separately logged truncate may have been used, refuse to proceed except if innodb_force_recovery is set. We will still refuse crash-upgrade if TRUNCATE TABLE was logged. Undo tablespace truncation would only be logged in undotrunc.log files, which we are no longer checking for.	7 years ago
Marko Mäkelä	ba43914ec4	Replace dict_table_is_temporary(table) with table->is_temporary()	8 years ago
Marko Mäkelä	2b27ac8282	Fix many -Wunused-parameter Remove unused InnoDB function parameters and functions. i_s_sys_virtual_fill_table(): Do not allocate heap memory. mtr_is_block_fix(): Replace with mtr_memo_contains(). mtr_is_page_fix(): Replace with mtr_memo_contains_page().	8 years ago
Marko Mäkelä	9801715cb0	Use compile_time_assert() in InnoDB Replace most use of #error. Some checks were impossible to evaluate in the preprocessor due to the use of named integer constants or enumerations.	8 years ago
Marko Mäkelä	a90100d756	Replace univ_page_size and UNIV_PAGE_SIZE Try to use one variable (srv_page_size) for innodb_page_size. Also, replace UNIV_PAGE_SIZE_SHIFT with srv_page_size_shift.	8 years ago
Marko Mäkelä	ba19764209	Fix most -Wsign-conversion in InnoDB Change innodb_buffer_pool_size, innodb_fill_factor to unsigned.	8 years ago
Marko Mäkelä	04bac13b30	Revert "MDEV-12266: Remove lookups when dropping indexes" This reverts commit `d266036292`. The patch would cause a crash when DROP TABLE is executed and the .ibd file is missing. It is not easy to fix this; there is some delicate logic in fil_space_get_page_size().	8 years ago
Marko Mäkelä	d266036292	MDEV-12266: Remove lookups when dropping indexes btr_free_root_check(), btr_free_if_exists(): Replace page_id, page_size with fil_space_t* and the root page number. btr_free_but_not_root(): Take fil_space_t* as a parameter, or NULL if the operation is not going to be redo-logged. btr_free(): Pass space=NULL to btr_free_but_not_root().	8 years ago
Marko Mäkelä	4cad42392a	MDEV-12266: Change dict_table_t::space to fil_space_t* InnoDB always keeps all tablespaces in the fil_system cache. The fil_system.LRU is only for closing file handles; the fil_space_t and fil_node_t for all data files will remain in main memory. Between startup to shutdown, they can only be created and removed by DDL statements. Therefore, we can let dict_table_t::space point directly to the fil_space_t. dict_table_t::space_id: A numeric tablespace ID for the corner cases where we do not have a tablespace. The most prominent examples are ALTER TABLE...DISCARD TABLESPACE or a missing or corrupted file. There are a few functional differences; most notably: (1) DROP TABLE will delete matching .ibd and .cfg files, even if they were not attached to the data dictionary. (2) Some error messages will report file names instead of numeric IDs. There still are many functions that use numeric tablespace IDs instead of fil_space_t, and many functions could be converted to fil_space_t member functions. Also, Tablespace and Datafile should be merged with fil_space_t and fil_node_t. page_id_t and buf_page_get_gen() could use fil_space_t& instead of a numeric ID, and after moving to a single buffer pool (MDEV-15058), buf_pool_t::page_hash could be moved to fil_space_t::page_hash. FilSpace: Remove. Only few calls to fil_space_acquire() will remain, and gradually they should be removed. mtr_t::set_named_space_id(ulint): Renamed from set_named_space(), to prevent accidental calls to this slower function. Very few callers remain. fseg_create(), fsp_reserve_free_extents(): Take fil_space_t as a parameter instead of a space_id. fil_space_t::rename(): Wrapper for fil_rename_tablespace_check(), fil_name_write_rename(), fil_rename_tablespace(). Mariabackup passes the parameter log=false; InnoDB passes log=true. dict_mem_table_create(): Take fil_space_t* instead of space_id as parameter. dict_process_sys_tables_rec_and_mtr_commit(): Replace the parameter 'status' with 'bool cached'. dict_get_and_save_data_dir_path(): Avoid copying the fil_node_t::name. fil_ibd_open(): Return the tablespace. fil_space_t::set_imported(): Replaces fil_space_set_imported(). truncate_t: Change many member function parameters to fil_space_t*, and remove page_size parameters. row_truncate_prepare(): Merge to its only caller. row_drop_table_from_cache(): Assert that the table is persistent. dict_create_sys_indexes_tuple(): Write SYS_INDEXES.SPACE=FIL_NULL if the tablespace has been discarded. row_import_update_discarded_flag(): Remove a constant parameter.	8 years ago
Marko Mäkelä	604fea1ad6	MDEV-12266: Remove dict_index_t::space We can rely on the dict_table_t::space. All indexes of a table object are always in the same tablespace. (For fulltext indexes, the data is located in auxiliary tables, and these will continue to have their own table objects, separate from the main table.)	8 years ago
Sergey Vojtovich	131d9a5d0c	Allocate lock_sys statically There is only one lock_sys. Allocate it statically in order to avoid dereferencing a pointer whenever accessing it. Also, align some members to their own cache line in order to avoid false sharing. lock_sys_t::create(): The deferred constructor. lock_sys_t::close(): The early destructor.	8 years ago
Eugene Kosov	af2d260b37	merge btr_page_get_level_low() and btr_page_get_level()	8 years ago
Marko Mäkelä	32170f8c6d	Add page_has_prev(), page_has_next(), page_has_siblings() Until now, InnoDB inefficiently compared the aligned fields FIL_PAGE_PREV, FIL_PAGE_NEXT to the byte-order-agnostic value FIL_NULL.	8 years ago
Marko Mäkelä	7660d8c94e	Remove dict_table_t::is_clust() Replace all occurrences of the is_clust() method with is_primary(), because that is what is actually meant. (Also the change buffer tree would count as a clustered index.)	8 years ago
Marko Mäkelä	9dfe84d5de	Remove a bogus page_is_root() debug assertion on btr_create() failure The predicate page_is_root() would not hold if btr_create() fails before the root page is fully initialized. Move the debug assertion from btr_free_root_invalidate() to its other caller, btr_free_if_exists(). In that caller, we actually already checked for page_is_root().	8 years ago
Marko Mäkelä	b933a8c354	MDEV-12569 InnoDB suggests filing bugs at MySQL bug tracker Replace all references in InnoDB and XtraDB error log messages to bugs.mysql.com with references to https://jira.mariadb.org/. The original merge commit `4274d0bf57` was accidentally reverted by the subsequent merge commit `3b35d745c3`	8 years ago

1 2 3

147 Commits (89ae01fd0085cf0d1af272eca545e49fdadf4538)