mariadb

Commit Graph

Author	SHA1	Message	Date
Marko Mäkelä	041a32abcd	Remove trx_mod_tables_t::vers_by_trx Only invoke set_versioned() on trx_id versioned tables. dict_table_t::versioned_by_id(): New accessor, to determine if a table is system versioned by transaction ID.	8 years ago
Marko Mäkelä	f8882cce93	Replace trx_sys_t* trx_sys with trx_sys_t trx_sys There is only one transaction system object in InnoDB. Allocate the storage for it at link time, not at runtime. lock_rec_fetch_page(): Use the correct fetch mode BUF_GET. Pages may never be deallocated from a tablespace while record locks are pointing to them.	8 years ago
Marko Mäkelä	acd2862e65	MDEV-14848 MariaDB 10.3 refuses InnoDB crash-upgrade from MariaDB 10.2 While the redo log format was changed in MariaDB 10.3.2 and 10.3.3 due to MDEV-12288 and MDEV-11369, it should be technically possible to upgrade from a crashed MariaDB 10.2 instance. On a related note, it should be possible for Mariabackup 10.3 to create a backup from a running MariaDB Server 10.2. mlog_id_t: Put back the 10.2 specific redo log record types MLOG_UNDO_INSERT, MLOG_UNDO_ERASE_END, MLOG_UNDO_INIT, MLOG_UNDO_HDR_REUSE. trx_undo_parse_add_undo_rec(): Parse or apply MLOG_UNDO_INSERT. trx_undo_erase_page_end(): Apply MLOG_UNDO_ERASE_END. trx_undo_parse_page_init(): Parse or apply MLOG_UNDO_INIT. trx_undo_parse_page_header_reuse(): Parse or apply MLOG_UNDO_HDR_REUSE. recv_log_recover_10_2(): Remove. Always parse the redo log from 10.2. recv_find_max_checkpoint(), recv_recovery_from_checkpoint_start(): Always parse the redo log from MariaDB 10.2. recv_parse_or_apply_log_rec_body(): Parse or apply MLOG_UNDO_INSERT, MLOG_UNDO_ERASE_END, MLOG_UNDO_INIT. srv_prepare_to_delete_redo_log_files(), innobase_start_or_create_for_mysql(): Upgrade from a previous (supported) redo log format.	8 years ago
Marko Mäkelä	9eb3fcc9fb	Follow-up fix of MDEV-14717 RENAME TABLE in InnoDB is not crash-safe trx_undo_page_report_rename(): Return a pointer to the start of the undo log record, not to the start of the (not yet written) next free record. The wrong return value would sometimes cause ROLLBACK to crash in an assertion failure (trying to parse garbage from the free area at the end of the insert_undo log page) if the TRX_UNDO_RENAME_TABLE record was the very last thing that was written to the insert_undo log. This would occasionally happen when an ALTER TABLE operation is rolled back due to invalid FOREIGN KEY constraints in the innodb.innodb test. In these tests, the error ER_ERROR_ON_RENAME (1025) would be returned at the end of the ALGORITHM=COPY operation of ALTER TABLE.	8 years ago
Marko Mäkelä	0bc36758ba	MDEV-14717 RENAME TABLE in InnoDB is not crash-safe InnoDB in MariaDB 10.2 appears to only write MLOG_FILE_RENAME2 redo log records during table-rebuilding ALGORITHM=INPLACE operations. We must write the records for any .ibd file renames, so that the operations are crash-safe. If InnoDB is killed during a RENAME TABLE operation, it can happen that the transaction for updating the data dictionary will be rolled back. But, nothing will roll back the renaming of the .ibd file (the MLOG_FILE_RENAME2 only guarantees roll-forward), or for that matter, the renaming of the dict_table_t::name in the dict_sys cache. We introduce the undo log record TRX_UNDO_RENAME_TABLE to fix this. fil_space_for_table_exists_in_mem(): Remove the parameters adjust_space, table_id and some code that was trying to work around these deficiencies. fil_name_write_rename(): Write a MLOG_FILE_RENAME2 record. dict_table_rename_in_cache(): Invoke fil_name_write_rename(). trx_undo_rec_copy(): Set the first 2 bytes to the length of the copied undo log record. trx_undo_page_report_rename(), trx_undo_report_rename(): Write a TRX_UNDO_RENAME_TABLE record with the old table name. row_rename_table_for_mysql(): Invoke trx_undo_report_rename() before modifying any data dictionary tables. row_undo_ins_parse_undo_rec(): Roll back TRX_UNDO_RENAME_TABLE by invoking dict_table_rename_in_cache(), which will take care of both renaming the table and the file.	8 years ago
Marko Mäkelä	f7f5c710e4	Correct a function comment The comment became stale in commit `9f57e595b4` which removed the parameter "flags".	8 years ago
Marko Mäkelä	0fd3def284	Remove MLOG_UNDO_ERASE_END	8 years ago
Marko Mäkelä	9ee8917dfd	Replace MLOG_UNDO_INSERT with MLOG_WRITE_STRING, MLOG_2BYTES	8 years ago
Aleksey Midenkov	b55a149194	Timestamp-based versioning for InnoDB [closes #209 ] * Removed integer_fields check * Reworked Vers_parse_info::check_sys_fields() * Misc renames * versioned as vers_sys_type_t * Removed versioned_by_sql(), versioned_by_engine() versioned() works as before; versioned(VERS_TIMESTAMP) is versioned_by_sql(); versioned(VERS_TRX_ID) is versioned_by_engine(). * create_tmp_table() fix * Foreign constraints for timestamp-based * Range auto-specifier fix * SQL: 1-row partition rotation fix [fixes #260] * Fix 'drop system versioning, algorithm=inplace'	8 years ago
Marko Mäkelä	0b89a42ffc	Remove the flag vers_update_trt THD::vers_update_trt, trx_t::vers_update_trt, trx_savept_t::vers_update_trt: Remove. Instead, determine from trx_t::mod_tables whether versioned columns were affected by the transaction. handlerton::prepare_commit_versioned: Replaces vers_get_trt_data. Return the transaction start ID and also the commit ID, in case the transaction modified any system-versioned columns (0 if not). TR_table::store_data(): Remove (merge with update() below). TR_table::update(): Add the parameters start_id, end_id. ha_commit_trans(): Remove a condition on SQLCOM_ALTER_TABLE. If we need something special for ALTER TABLE...ALGORITHM=INPLACE, that can be done inside InnoDB by modifying trx_t::mod_tables. innodb_prepare_commit_versioned(): Renamed from innodb_get_trt_data(). Check trx_t::mod_tables to see if any changes to versioned columns are present. trx_mod_table_time_t: A pair of logical timestamps, replacing the undo_no_t in trx_mod_tables_t. Keep track of not only the first modification to a persistent table in each transaction, but also the first modification of a versioned column in a table. dtype_t, dict_col_t: Add the accessor is_any_versioned(), to check if the type refers to a system-versioned user or system column. upd_t::affects_versioned(): Check if an update affects a versioned column. trx_undo_report_row_operation(): If a versioned column is affected by the update, invoke trx_mod_table_time_t::set_versioned(). trx_rollback_to_savepoint_low(): If all changes to versioned columns were rolled back, invoke trx_mod_table_time_t::rollback_versioned(), so that trx_mod_table_time_t::is_versioned() will no longer hold.	8 years ago
Marko Mäkelä	d8ada08152	MDEV-14477 InnoDB update_time is wrongly updated after partial rollback or internal COMMIT The non-persistent UPDATE_TIME for InnoDB tables was not being updated consistently at transaction commit. If a transaction is partly rolled back so that in the end it will not modify a table that it intended to modify, the update_time will be updated nevertheless. This will also happen when InnoDB fails to write an undo log record for the intended modification. If a transaction is committed internally in InnoDB, instead of being committed from the SQL interface, then the trx_t::mod_tables will not be applied to the update_time of the tables. trx_t::mod_tables: Replace the std::set<dict_table_t> with std::map<dict_table_t,undo_no_t>, so that the very first modification within the transaction is identified. trx_undo_report_row_operation(): Update mod_tables for every operation after the undo log record was successfully written. trx_rollback_to_savepoint_low(): After partial rollback, erase from trx_t::mod_tables any tables for which all changes were rolled back. trx_commit_in_memory(): Tighten some assertions and simplify conditions. Invoke trx_update_mod_tables_timestamp() if persistent tables were affected. trx_commit_for_mysql(): Remove the call to trx_update_mod_tables_timestamp(), as it is now invoked at the lower level, in trx_commit_in_memory(). trx_rollback_finish(): Clear mod_tables before invoking trx_commit(), because the trx_commit_in_memory() would otherwise wrongly process mod_tables after a full ROLLBACK.	8 years ago
Marko Mäkelä	fda4fabe9c	Correct a comment	8 years ago
Marko Mäkelä	5d142f9958	MDEV-13795/MDEV-14332 Corruption during online table-rebuilding ALTER when VIRTUAL columns exist When MySQL 5.7 introduced indexed virtual columns, it introduced several bugs into the online table-rebuilding ALTER, that is, the row_log_table_apply() family of functions. The online_log format that was introduced for online table-rebuilding ALTER in MySQL 5.6 should be sufficient. Ideally, any indexed virtual column values would be evaluated based on the log records in the temporary file. There is no need to log virtual column values. (For ADD INDEX, that is row_log_apply(), we always must log the values of the keys, no matter if the columns are virtual.) Because omitting the virtual column values removes any chance of row_log_table_apply() working with indexed virtual columns, we will for now refuse LOCK=NONE in table-rebuilding ALTER operations when indexes on virtual columns exist. This restriction would be lifted in MDEV-14341. innobase_indexed_virtual_exist(): New predicate, to determine if indexed virtual columns exist in a table definition. ha_innobase::check_if_supported_inplace_alter(): Refuse online rebuild if indexed virtual columns exist. rec_get_converted_size_temp_v(), rec_convert_dtuple_to_temp_v(): Remove. row_log_table_delete(), row_log_table_update(, row_log_table_insert(): Remove parameters for virtual columns. trx_undo_read_v_rows(): Remove the col_map parameter. row_log_table_apply(): Do not deal with virtual columns.	8 years ago
Marko Mäkelä	b933a8c354	MDEV-12569 InnoDB suggests filing bugs at MySQL bug tracker Replace all references in InnoDB and XtraDB error log messages to bugs.mysql.com with references to https://jira.mariadb.org/. The original merge commit `4274d0bf57` was accidentally reverted by the subsequent merge commit `3b35d745c3`	8 years ago
Marko Mäkelä	a4948dafcd	MDEV-11369 Instant ADD COLUMN for InnoDB For InnoDB tables, adding, dropping and reordering columns has required a rebuild of the table and all its indexes. Since MySQL 5.6 (and MariaDB 10.0) this has been supported online (LOCK=NONE), allowing concurrent modification of the tables. This work revises the InnoDB ROW_FORMAT=REDUNDANT, ROW_FORMAT=COMPACT and ROW_FORMAT=DYNAMIC so that columns can be appended instantaneously, with only minor changes performed to the table structure. The counter innodb_instant_alter_column in INFORMATION_SCHEMA.GLOBAL_STATUS is incremented whenever a table rebuild operation is converted into an instant ADD COLUMN operation. ROW_FORMAT=COMPRESSED tables will not support instant ADD COLUMN. Some usability limitations will be addressed in subsequent work: MDEV-13134 Introduce ALTER TABLE attributes ALGORITHM=NOCOPY and ALGORITHM=INSTANT MDEV-14016 Allow instant ADD COLUMN, ADD INDEX, LOCK=NONE The format of the clustered index (PRIMARY KEY) is changed as follows: (1) The FIL_PAGE_TYPE of the root page will be FIL_PAGE_TYPE_INSTANT, and a new field PAGE_INSTANT will contain the original number of fields in the clustered index ('core' fields). If instant ADD COLUMN has not been used or the table becomes empty, or the very first instant ADD COLUMN operation is rolled back, the fields PAGE_INSTANT and FIL_PAGE_TYPE will be reset to 0 and FIL_PAGE_INDEX. (2) A special 'default row' record is inserted into the leftmost leaf, between the page infimum and the first user record. This record is distinguished by the REC_INFO_MIN_REC_FLAG, and it is otherwise in the same format as records that contain values for the instantly added columns. This 'default row' always has the same number of fields as the clustered index according to the table definition. The values of 'core' fields are to be ignored. For other fields, the 'default row' will contain the default values as they were during the ALTER TABLE statement. (If the column default values are changed later, those values will only be stored in the .frm file. The 'default row' will contain the original evaluated values, which must be the same for every row.) The 'default row' must be completely hidden from higher-level access routines. Assertions have been added to ensure that no 'default row' is ever present in the adaptive hash index or in locked records. The 'default row' is never delete-marked. (3) In clustered index leaf page records, the number of fields must reside between the number of 'core' fields (dict_index_t::n_core_fields introduced in this work) and dict_index_t::n_fields. If the number of fields is less than dict_index_t::n_fields, the missing fields are replaced with the column value of the 'default row'. Note: The number of fields in the record may shrink if some of the last instantly added columns are updated to the value that is in the 'default row'. The function btr_cur_trim() implements this 'compression' on update and rollback; dtuple::trim() implements it on insert. (4) In ROW_FORMAT=COMPACT and ROW_FORMAT=DYNAMIC records, the new status value REC_STATUS_COLUMNS_ADDED will indicate the presence of a new record header that will encode n_fields-n_core_fields-1 in 1 or 2 bytes. (In ROW_FORMAT=REDUNDANT records, the record header always explicitly encodes the number of fields.) We introduce the undo log record type TRX_UNDO_INSERT_DEFAULT for covering the insert of the 'default row' record when instant ADD COLUMN is used for the first time. Subsequent instant ADD COLUMN can use TRX_UNDO_UPD_EXIST_REC. This is joint work with Vin Chen (陈福荣) from Tencent. The design that was discussed in April 2017 would not have allowed import or export of data files, because instead of the 'default row' it would have introduced a data dictionary table. The test rpl.rpl_alter_instant is exactly as contributed in pull request #408. The test innodb.instant_alter is based on a contributed test. The redo log record format changes for ROW_FORMAT=DYNAMIC and ROW_FORMAT=COMPACT are as contributed. (With this change present, crash recovery from MariaDB 10.3.1 will fail in spectacular ways!) Also the semantics of higher-level redo log records that modify the PAGE_INSTANT field is changed. The redo log format version identifier was already changed to LOG_HEADER_FORMAT_CURRENT=103 in MariaDB 10.3.1. Everything else has been rewritten by me. Thanks to Elena Stepanova, the code has been tested extensively. When rolling back an instant ADD COLUMN operation, we must empty the PAGE_FREE list after deleting or shortening the 'default row' record, by calling either btr_page_empty() or btr_page_reorganize(). We must know the size of each entry in the PAGE_FREE list. If rollback left a freed copy of the 'default row' in the PAGE_FREE list, we would be unable to determine its size (if it is in ROW_FORMAT=COMPACT or ROW_FORMAT=DYNAMIC) because it would contain more fields than the rolled-back definition of the clustered index. UNIV_SQL_DEFAULT: A new special constant that designates an instantly added column that is not present in the clustered index record. len_is_stored(): Check if a length is an actual length. There are two magic length values: UNIV_SQL_DEFAULT, UNIV_SQL_NULL. dict_col_t::def_val: The 'default row' value of the column. If the column is not added instantly, def_val.len will be UNIV_SQL_DEFAULT. dict_col_t: Add the accessors is_virtual(), is_nullable(), is_instant(), instant_value(). dict_col_t::remove_instant(): Remove the 'instant ADD' status of a column. dict_col_t::name(const dict_table_t& table): Replaces dict_table_get_col_name(). dict_index_t::n_core_fields: The original number of fields. For secondary indexes and if instant ADD COLUMN has not been used, this will be equal to dict_index_t::n_fields. dict_index_t::n_core_null_bytes: Number of bytes needed to represent the null flags; usually equal to UT_BITS_IN_BYTES(n_nullable). dict_index_t::NO_CORE_NULL_BYTES: Magic value signalling that n_core_null_bytes was not initialized yet from the clustered index root page. dict_index_t: Add the accessors is_instant(), is_clust(), get_n_nullable(), instant_field_value(). dict_index_t::instant_add_field(): Adjust clustered index metadata for instant ADD COLUMN. dict_index_t::remove_instant(): Remove the 'instant ADD' status of a clustered index when the table becomes empty, or the very first instant ADD COLUMN operation is rolled back. dict_table_t: Add the accessors is_instant(), is_temporary(), supports_instant(). dict_table_t::instant_add_column(): Adjust metadata for instant ADD COLUMN. dict_table_t::rollback_instant(): Adjust metadata on the rollback of instant ADD COLUMN. prepare_inplace_alter_table_dict(): First create the ctx->new_table, and only then decide if the table really needs to be rebuilt. We must split the creation of table or index metadata from the creation of the dictionary table records and the creation of the data. In this way, we can transform a table-rebuilding operation into an instant ADD COLUMN operation. Dictionary objects will only be added to cache when table rebuilding or index creation is needed. The ctx->instant_table will never be added to cache. dict_table_t::add_to_cache(): Modified and renamed from dict_table_add_to_cache(). Do not modify the table metadata. Let the callers invoke dict_table_add_system_columns() and if needed, set can_be_evicted. dict_create_sys_tables_tuple(), dict_create_table_step(): Omit the system columns (which will now exist in the dict_table_t object already at this point). dict_create_table_step(): Expect the callers to invoke dict_table_add_system_columns(). pars_create_table(): Before creating the table creation execution graph, invoke dict_table_add_system_columns(). row_create_table_for_mysql(): Expect all callers to invoke dict_table_add_system_columns(). create_index_dict(): Replaces row_merge_create_index_graph(). innodb_update_n_cols(): Renamed from innobase_update_n_virtual(). Call my_error() if an error occurs. btr_cur_instant_init(), btr_cur_instant_init_low(), btr_cur_instant_root_init(): Load additional metadata from the clustered index and set dict_index_t::n_core_null_bytes. This is invoked when table metadata is first loaded into the data dictionary. dict_boot(): Initialize n_core_null_bytes for the four hard-coded dictionary tables. dict_create_index_step(): Initialize n_core_null_bytes. This is executed as part of CREATE TABLE. dict_index_build_internal_clust(): Initialize n_core_null_bytes to NO_CORE_NULL_BYTES if table->supports_instant(). row_create_index_for_mysql(): Initialize n_core_null_bytes for CREATE TEMPORARY TABLE. commit_cache_norebuild(): Call the code to rename or enlarge columns in the cache only if instant ADD COLUMN is not being used. (Instant ADD COLUMN would copy all column metadata from instant_table to old_table, including the names and lengths.) PAGE_INSTANT: A new 13-bit field for storing dict_index_t::n_core_fields. This is repurposing the 16-bit field PAGE_DIRECTION, of which only the least significant 3 bits were used. The original byte containing PAGE_DIRECTION will be accessible via the new constant PAGE_DIRECTION_B. page_get_instant(), page_set_instant(): Accessors for the PAGE_INSTANT. page_ptr_get_direction(), page_get_direction(), page_ptr_set_direction(): Accessors for PAGE_DIRECTION. page_direction_reset(): Reset PAGE_DIRECTION, PAGE_N_DIRECTION. page_direction_increment(): Increment PAGE_N_DIRECTION and set PAGE_DIRECTION. rec_get_offsets(): Use the 'leaf' parameter for non-debug purposes, and assume that heap_no is always set. Initialize all dict_index_t::n_fields for ROW_FORMAT=REDUNDANT records, even if the record contains fewer fields. rec_offs_make_valid(): Add the parameter 'leaf'. rec_copy_prefix_to_dtuple(): Assert that the tuple is only built on the core fields. Instant ADD COLUMN only applies to the clustered index, and we should never build a search key that has more than the PRIMARY KEY and possibly DB_TRX_ID,DB_ROLL_PTR. All these columns are always present. dict_index_build_data_tuple(): Remove assertions that would be duplicated in rec_copy_prefix_to_dtuple(). rec_init_offsets(): Support ROW_FORMAT=REDUNDANT records whose number of fields is between n_core_fields and n_fields. cmp_rec_rec_with_match(): Implement the comparison between two MIN_REC_FLAG records. trx_t::in_rollback: Make the field available in non-debug builds. trx_start_for_ddl_low(): Remove dangerous error-tolerance. A dictionary transaction must be flagged as such before it has generated any undo log records. This is because trx_undo_assign_undo() will mark the transaction as a dictionary transaction in the undo log header right before the very first undo log record is being written. btr_index_rec_validate(): Account for instant ADD COLUMN row_undo_ins_remove_clust_rec(): On the rollback of an insert into SYS_COLUMNS, revert instant ADD COLUMN in the cache by removing the last column from the table and the clustered index. row_search_on_row_ref(), row_undo_mod_parse_undo_rec(), row_undo_mod(), trx_undo_update_rec_get_update(): Handle the 'default row' as a special case. dtuple_t::trim(index): Omit a redundant suffix of an index tuple right before insert or update. After instant ADD COLUMN, if the last fields of a clustered index tuple match the 'default row', there is no need to store them. While trimming the entry, we must hold a page latch, so that the table cannot be emptied and the 'default row' be deleted. btr_cur_optimistic_update(), btr_cur_pessimistic_update(), row_upd_clust_rec_by_insert(), row_ins_clust_index_entry_low(): Invoke dtuple_t::trim() if needed. row_ins_clust_index_entry(): Restore dtuple_t::n_fields after calling row_ins_clust_index_entry_low(). rec_get_converted_size(), rec_get_converted_size_comp(): Allow the number of fields to be between n_core_fields and n_fields. Do not support infimum,supremum. They are never supposed to be stored in dtuple_t, because page creation nowadays uses a lower-level method for initializing them. rec_convert_dtuple_to_rec_comp(): Assign the status bits based on the number of fields. btr_cur_trim(): In an update, trim the index entry as needed. For the 'default row', handle rollback specially. For user records, omit fields that match the 'default row'. btr_cur_optimistic_delete_func(), btr_cur_pessimistic_delete(): Skip locking and adaptive hash index for the 'default row'. row_log_table_apply_convert_mrec(): Replace 'default row' values if needed. In the temporary file that is applied by row_log_table_apply(), we must identify whether the records contain the extra header for instantly added columns. For now, we will allocate an additional byte for this for ROW_T_INSERT and ROW_T_UPDATE records when the source table has been subject to instant ADD COLUMN. The ROW_T_DELETE records are fine, as they will be converted and will only contain 'core' columns (PRIMARY KEY and some system columns) that are converted from dtuple_t. rec_get_converted_size_temp(), rec_init_offsets_temp(), rec_convert_dtuple_to_temp(): Add the parameter 'status'. REC_INFO_DEFAULT_ROW = REC_INFO_MIN_REC_FLAG \| REC_STATUS_COLUMNS_ADDED: An info_bits constant for distinguishing the 'default row' record. rec_comp_status_t: An enum of the status bit values. rec_leaf_format: An enum that replaces the bool parameter of rec_init_offsets_comp_ordinary().	8 years ago
Marko Mäkelä	7cd4a66de6	Remove unused parameters and dead code upd_field_set_field_no(): Remove the unused parameter trx, and remove a debug message. In debug builds, the out-of-bounds access would still be caught by dict_index_get_nth_col(). trx_undo_update_rec_get_update(): Remove the unnecessary parameter trx.	8 years ago
Marko Mäkelä	48192f963a	Add the parameter bool leaf to rec_get_offsets() This should affect debug builds only. Debug builds will check that the status bits of ROW_FORMAT!=REDUNDANT records match the is_leaf parameter. The only observable change to non-debug should be the addition of the is_leaf parameter to the function rec_copy_prefix_to_dtuple(), and the removal of some calls to update the adaptive hash index (it is only built for the leaf pages). This change should have been made in MySQL 5.0.3, instead of introducing the status flags in the ROW_FORMAT=COMPACT record header.	8 years ago
Marko Mäkelä	c3399d799f	Add some const qualifiers	8 years ago
Marko Mäkelä	f4b379d6ca	MDEV-13536 DB_TRX_ID is not actually being reset when the history is removed This should have been part of MDEV-12288. trx_undo_t::del_marks: Remove. Purge needs to process all undo log records in order to reset the DB_TRX_ID. Before MDEV-12288, it sufficed to only delete the purgeable delete-marked records, and it ignore other undo log. trx_rseg_t::needs_purge: Renamed from trx_rseg_t::last_del_marks. Indicates whether a rollback segment needs to be processed by purge. TRX_UNDO_NEEDS_PURGE: Renamed from TRX_UNDO_DEL_MARKS. Indicates whether a rollback segment needs to be processed by purge. This will be 1 until trx_purge_free_segment() has been invoked. row_purge_record_func(): Set the is_insert flag for TRX_UNDO_INSERT_REC, so that the DB_ROLL_PTR will match in row_purge_reset_trx_id(). trx_purge_fetch_next_rec(): Add a comment about row_purge_record_func() going to set the is_insert flag. trx_purge_read_undo_rec(): Always attempt to read the undo log record. trx_purge_get_next_rec(): Do not skip any undo log records. Even when no clustered index record is going to be removed, we may want to reset some DB_TRX_ID,DB_ROLL_PTR. trx_undo_rec_get_cmpl_info(), trx_undo_rec_get_extern_storage(): Remove. trx_purge_add_undo_to_history(): Set the TRX_UNDO_NEEDS_PURGE flag so that the resetting will work on undo logs that were originally created before MDEV-12288 (MariaDB 10.3.1). trx_undo_roll_ptr_is_insert(), trx_purge_free_segment(): Cleanup (should be no functional change).	8 years ago
Marko Mäkelä	73297f532f	MDEV-13476 TRX_UNDO_PAGE_TYPE mismatch when writing undo log after upgrade When undo log pages pre-exist from an upgrade, the TRX_UNDO_PAGE_TYPE could be TRX_UNDO_INSERT==1 (for insert_undo) TRX_UNDO_UPDATE==2 (for update_undo), instead of the new unified page type 0 that was introduced in MDEV-12288. The undo log page type does not really matter much, because the undo log record type identifies the records independently of the page type. So, the debug assertions can simply allow any potential value of the TRX_UNDO_PAGE_TYPE (0, 1, or 2).	8 years ago
Marko Mäkelä	3c09f148f3	MDEV-12288 Reset DB_TRX_ID when the history is removed, to speed up MVCC Let InnoDB purge reset DB_TRX_ID,DB_ROLL_PTR when the history is removed. [TODO: It appears that the resetting is not taking place as often as it could be. We should test that a simple INSERT should eventually cause row_purge_reset_trx_id() to be invoked unless DROP TABLE is invoked soon enough.] The InnoDB clustered index record system columns DB_TRX_ID,DB_ROLL_PTR are used by multi-versioning. After the history is no longer needed, these columns can safely be reset to 0 and 1<<55 (to indicate a fresh insert). When a reader sees 0 in the DB_TRX_ID column, it can instantly determine that the record is present the read view. There is no need to acquire the transaction system mutex to check if the transaction exists, because writes can never be conducted by a transaction whose ID is 0. The persistent InnoDB undo log used to be split into two parts: insert_undo and update_undo. The insert_undo log was discarded at transaction commit or rollback, and the update_undo log was processed by the purge subsystem. As part of this change, we will only generate a single undo log for new transactions, and the purge subsystem will reset the DB_TRX_ID whenever a clustered index record is touched. That is, all persistent undo log will be preserved at transaction commit or rollback, to be removed by purge. The InnoDB redo log format is changed in two ways: We remove the redo log record type MLOG_UNDO_HDR_REUSE, and we introduce the MLOG_ZIP_WRITE_TRX_ID record for updating the DB_TRX_ID,DB_ROLL_PTR in a ROW_FORMAT=COMPRESSED table. This is also changing the format of persistent InnoDB data files: undo log and clustered index leaf page records. It will still be possible via import and export to exchange data files with earlier versions of MariaDB. The change to clustered index leaf page records is simple: we allow DB_TRX_ID to be 0. When it comes to the undo log, we must be able to upgrade from earlier MariaDB versions after a clean shutdown (no redo log to apply). While it would be nice to perform a slow shutdown (innodb_fast_shutdown=0) before an upgrade, to empty the undo logs, we cannot assume that this has been done. So, separate insert_undo log may exist for recovered uncommitted transactions. These transactions may be automatically rolled back, or they may be in XA PREPARE state, in which case InnoDB will preserve the transaction until an explicit XA COMMIT or XA ROLLBACK. Upgrade has been tested by starting up MariaDB 10.2 with ./mysql-test-run --manual-gdb innodb.read_only_recovery and then starting up this patched server with and without --innodb-read-only. trx_undo_ptr_t::undo: Renamed from update_undo. trx_undo_ptr_t::old_insert: Renamed from insert_undo. trx_rseg_t::undo_list: Renamed from update_undo_list. trx_rseg_t::undo_cached: Merged from update_undo_cached and insert_undo_cached. trx_rseg_t::old_insert_list: Renamed from insert_undo_list. row_purge_reset_trx_id(): New function to reset the columns. This will be called for all undo processing in purge that does not remove the clustered index record. trx_undo_update_rec_get_update(): Allow trx_id=0 when copying the old DB_TRX_ID of the record to the undo log. ReadView::changes_visible(): Allow id==0. (Return true for it. This is what speeds up the MVCC.) row_vers_impl_x_locked_low(), row_vers_build_for_semi_consistent_read(): Implement a fast path for DB_TRX_ID=0. Always initialize the TRX_UNDO_PAGE_TYPE to 0. Remove undo->type. MLOG_UNDO_HDR_REUSE: Remove. This changes the redo log format! innobase_start_or_create_for_mysql(): Set srv_undo_sources before starting any transactions. The parsing of the MLOG_ZIP_WRITE_TRX_ID record was successfully tested by running the following: ./mtr --parallel=auto --mysqld=--debug=d,ib_log innodb_zip.bug56680 grep MLOG_ZIP_WRITE_TRX_ID var/*/log/mysqld.1.err	8 years ago
Marko Mäkelä	c436338d9d	Assert that DB_TRX_ID must be set on delete-marked records This is preparation for MDEV-12288, which would set DB_TRX_ID=0 when purging history. Also with that change in place, delete-marked records must always refer to an undo log record via a nonzero DB_TRX_ID column. (The DB_TRX_ID is only present in clustered index leaf page records.) btr_cur_parse_del_mark_set_clust_rec(), rec_get_trx_id(): Statically allocate the offsets (should never use the heap). Add some debug assertions. Replace some use of rec_get_trx_id() with row_get_rec_trx_id(). trx_undo_report_row_operation(): Add some sanity checks that are common for all operations that produce undo log.	8 years ago
Marko Mäkelä	0c92794db3	Remove deprecated InnoDB file format parameters The following options will be removed: innodb_file_format innodb_file_format_check innodb_file_format_max innodb_large_prefix They have been deprecated in MySQL 5.7.7 (and MariaDB 10.2.2) in WL#7703. The file_format column in two INFORMATION_SCHEMA tables will be removed: innodb_sys_tablespaces innodb_sys_tables Code to update the file format tag at the end of page 0:5 (TRX_SYS_PAGE in the InnoDB system tablespace) will be removed. When initializing a new database, the bytes will remain 0. All references to the Barracuda file format will be removed. Some references to the Antelope file format (meaning ROW_FORMAT=REDUNDANT or ROW_FORMAT=COMPACT) will remain. This basically ports WL#7704 from MySQL 8.0.0 to MariaDB 10.3.1: commit `4a69dc2a95` Author: Marko Mäkelä <marko.makela@oracle.com> Date: Wed Mar 11 22:19:49 2015 +0200	9 years ago
Marko Mäkelä	9f57e595b4	Refactor trx_undo_report_row_operation() Fix a -fsanitizer=undefined warning that trx_undo_report_row_operation() was being passed thr=NULL when the BTR_NO_UNDO_LOG_FLAG flag was set. trx_undo_report_row_operation(): Remove the first two parameters. The parameter clust_entry!=NULL distinguishes inserts from updates. This should be a non-functional change (no observable change in behaviour; slightly smaller code).	9 years ago
Marko Mäkelä	124bae082b	MDEV-12289 Keep 128 persistent rollback segments for compatibility and performance InnoDB divides the allocation of undo logs into rollback segments. The DB_ROLL_PTR system column of clustered indexes can address up to 128 rollback segments (TRX_SYS_N_RSEGS). Originally, InnoDB only created one rollback segment. In MySQL 5.5 or in the InnoDB Plugin for MySQL 5.1, all 128 rollback segments were created. MySQL 5.7 hard-codes the rollback segment IDs 1..32 for temporary undo logs. On upgrade, unless a slow shutdown (innodb_fast_shutdown=0) was performed on the old server instance, these rollback segments could be in use by transactions that are in XA PREPARE state or transactions that were left behind by a server kill followed by a normal shutdown immediately after restart. Persistent tables cannot refer to temporary undo logs or vice versa. Therefore, we should keep two distinct sets of rollback segments: one for persistent tables and another for temporary tables. In this way, all 128 rollback segments will be available for both types of tables, which could improve performance. Also, MariaDB 10.2 will remain more compatible than MySQL 5.7 with data files from earlier versions of MySQL or MariaDB. trx_sys_t::temp_rsegs[TRX_SYS_N_RSEGS]: A new array of temporary rollback segments. The trx_sys_t::rseg_array[TRX_SYS_N_RSEGS] will be solely for persistent undo logs. srv_tmp_undo_logs. Remove. Use the constant TRX_SYS_N_RSEGS. srv_available_undo_logs: Change the type to ulong. trx_rseg_get_on_id(): Remove. Instead, let the callers refer to trx_sys directly. trx_rseg_create(), trx_sysf_rseg_find_free(): Remove unneeded parameters. These functions only deal with persistent undo logs. trx_temp_rseg_create(): New function, to create all temporary rollback segments at server startup. trx_rseg_t::is_persistent(): Determine if the rollback segment is for persistent tables. trx_sys_is_noredo_rseg_slot(): Remove. The callers must know based on context (such as table handle) whether the DB_ROLL_PTR is referring to a persistent undo log. trx_sys_create_rsegs(): Remove all parameters, which were always passed as global variables. Instead, modify the global variables directly. enum trx_rseg_type_t: Remove. trx_t::get_temp_rseg(): A method to ensure that a temporary rollback segment has been assigned for the transaction. trx_t::assign_temp_rseg(): Replaces trx_assign_rseg(). trx_purge_free_segment(), trx_purge_truncate_rseg_history(): Remove the redundant variable noredo=false. Temporary undo logs are discarded immediately at transaction commit or rollback, not lazily by purge. trx_purge_mark_undo_for_truncate(): Remove references to the temporary rollback segments. trx_purge_mark_undo_for_truncate(): Remove a check for temporary rollback segments. Only the dedicated persistent undo log tablespaces can be truncated. trx_undo_get_undo_rec_low(), trx_undo_get_undo_rec(): Add the parameter is_temp. trx_rseg_mem_restore(): Split from trx_rseg_mem_create(). Initialize the undo log and the rollback segment from the file data structures. trx_sysf_get_n_rseg_slots(): Renamed from trx_sysf_used_slots_for_redo_rseg(). Count the persistent rollback segment headers that have been initialized. trx_sys_close(): Also free trx_sys->temp_rsegs[]. get_next_redo_rseg(): Merged to trx_assign_rseg_low(). trx_assign_rseg_low(): Remove the parameters and access the global variables directly. Revert to simple round-robin, now that the whole trx_sys->rseg_array[] is for persistent undo log again. get_next_noredo_rseg(): Moved to trx_t::assign_temp_rseg(). srv_undo_tablespaces_init(): Remove some parameters and use the global variables directly. Clarify some error messages. Adjust the test innodb.log_file. Apparently, before these changes, InnoDB somehow ignored missing dedicated undo tablespace files that are pointed by the TRX_SYS header page, possibly losing part of essential transaction system state.	9 years ago
Marko Mäkelä	a06da5c848	MDEV-12258 InnoDB: Fix the bogus debug assertion introduced in MDEV-12219 After a partial rollback, an undo log segment that is empty may carry a duplicate-looking top_undo_no. Adjust the debug assertions and add a test.	9 years ago
Marko Mäkelä	97acc4a1c3	MDEV-12270 Port MySQL 8.0 Bug#21141390 REMOVE UNUSED FUNCTIONS AND CONVERT GLOBAL SYMBOLS TO STATIC InnoDB defines some functions that are not called at all. Other functions are called, but only from the same compilation unit. Remove some function declarations and definitions, and add 'static' keywords. Some symbols must be kept for separately compiled tools, such as innochecksum.	11 years ago
Marko Mäkelä	4e1116b2c6	MDEV-12271 Port MySQL 8.0 Bug#23150562 REMOVE UNIV_MUST_NOT_INLINE AND UNIV_NONINL Also, remove empty .ic files that were not removed by my MySQL commit. Problem: InnoDB used to support a compilation mode that allowed to choose whether the function definitions in .ic files are to be inlined or not. This stopped making sense when InnoDB moved to C++ in MySQL 5.6 (and ha_innodb.cc started to #include .ic files), and more so in MySQL 5.7 when inline methods and functions were introduced in .h files. Solution: Remove all references to UNIV_NONINL and UNIV_MUST_NOT_INLINE from all files, assuming that the symbols are never defined. Remove the files fut0fut.cc and ut0byte.cc which only mattered when UNIV_NONINL was defined.	10 years ago
Marko Mäkelä	13e5c9de80	MDEV-12219 Discard temporary undo logs at transaction commit Starting with MySQL 5.7, temporary tables in InnoDB are handled differently from persistent tables. Because temporary tables are private to a connection, concurrency control and multi-versioning (MVCC) are not applicable. For performance reasons, purge is disabled as well. Rollback is supported for temporary tables; that is why we have the temporary undo logs in the first place. Because MVCC and purge are disabled for temporary tables, we should discard all temporary undo logs already at transaction commit, just like we discard the persistent insert_undo logs. Before this change, update_undo logs were being preserved. trx_temp_undo_t: A wrapper for temporary undo logs, comprising a rollback segment and a single temporary undo log. trx_rsegs_t::m_noredo: Use trx_temp_undo_t. (Instead of insert_undo, update_undo, there will be a single undo.) trx_is_noredo_rseg_updated(), trx_is_rseg_assigned(): Remove. trx_undo_add_page(): Remove the parameter undo_ptr. Acquire and release the rollback segment mutex inside the function. trx_undo_free_last_page(): Remove the parameter trx. trx_undo_truncate_end(): Remove the parameter trx, and add the parameter is_temp. Clean up the code a bit. trx_undo_assign_undo(): Split the parameter undo_ptr into rseg, undo. trx_undo_commit_cleanup(): Renamed from trx_undo_insert_cleanup(). Replace the parameter undo_ptr with undo. This will discard the temporary undo or insert_undo log at commit/rollback. trx_purge_add_update_undo_to_history(), trx_undo_update_cleanup(): Remove 3 parameters. Always operate on the persistent update_undo. trx_serialise(): Renamed from trx_serialisation_number_get(). trx_write_serialisation_history(): Simplify the code flow. If there are no persistent changes, do not update MONITOR_TRX_COMMIT_UNDO. trx_commit_in_memory(): Simplify the logic, and add assertions. trx_undo_page_report_modify(): Keep a direct reference to the persistent update_undo log. trx_undo_report_row_operation(): Simplify some code. Always assign TRX_UNDO_INSERT for temporary undo logs. trx_prepare_low(): Keep only one parameter. Prepare all 3 undo logs. trx_roll_try_truncate(): Remove the parameter undo_ptr. Try to truncate all 3 undo logs of the transaction. trx_roll_pop_top_rec_of_trx_low(): Remove. trx_roll_pop_top_rec_of_trx(): Remove the redundant parameter trx->roll_limit. Clear roll_limit when exhausting the undo logs. Consider all 3 undo logs at once, prioritizing the persistent undo logs. row_undo(): Minor cleanup. Let trx_roll_pop_top_rec_of_trx() reset the trx->roll_limit.	9 years ago
Marko Mäkelä	a20340cf85	Hard-code innodb_page_size as the undo log page size. InnoDB undo logs currently always use the innodb_page_size, whether they are stored in the system tablespace, in a dedicated undo tablespace, or in the temporary tablespace. Remove redundant page_size parameters. TrxUndoRsegsIterator::set_next(): return bool instead of page_size.	9 years ago
Marko Mäkelä	15bdfeeba8	Remove trx_sys_t::pending_purge_rseg_array. In MySQL 5.7, there is some redundant code for supposedly handling an upgrade from an earlier version of InnoDB. An upgrade of InnoDB between major versions should include a slow shutdown (innodb_fast_shutdown=0) of the previous version. A comment in trx_lists_init_at_db_start() confused clean shutdown and slow shutdown. A clean shutdown does not necessarily guarantee that there are no active transactions. A slow shutdown guarantees that. Because there was no code to handle rollback of recovered transactions that happened to use the rollback segment slots that MySQL 5.7.2 repurposed for temporary undo logs, the upgrade is not working in all cases, and we may as well remove the code to handle purging. trx_sys_t::pending_purge_rseg_array: Remove. trx_undo_get_undo_rec_low(): Define as static. Remove the parameter is_redo_rseg. trx_undo_get_undo_rec(), trx_rseg_get_on_id(): Remove the parameter is_redo_rseg. trx_rseg_mem_free(): Remove the second parameter. trx_sys_get_nth_rseg(): Replace with trx_rseg_get_on_id(). trx_rseg_schedule_pending_purge(): Remove.	9 years ago
Marko Mäkelä	8780b89529	MDEV-11831 Make InnoDB mini-transaction memo checks stricter InnoDB keeps track of buffer-fixed buf_block_t or acquired rw_lock_t within a mini-transaction. There are some memo_contains assertions in the code that document when certain blocks or rw_locks must be held. But, these assertions only check the mini-transaction memo, not the fact whether the rw_lock_t are actually being held by the caller. btr_pcur_store_position(): Remove #ifdef, and assert that the block is always buffer-fixed. rtr_pcur_getnext_from_path(), rtr_pcur_open_low(), ibuf_rec_get_page_no_func(), ibuf_rec_get_space_func(), ibuf_rec_get_info_func(), ibuf_rec_get_op_type_func(), ibuf_build_entry_from_ibuf_rec_func(), ibuf_rec_get_volume_func(), ibuf_get_merge_page_nos_func(), ibuf_get_volume_buffered_count_func() ibuf_get_entry_counter_low_func(), page_set_ssn_id(), row_vers_old_has_index_entry(), row_vers_build_for_consistent_read(), row_vers_build_for_semi_consistent_read(), trx_undo_prev_version_build(): Make use of mtr_memo_contains_page_flagged(). mtr_t::memo_contains(): Take a const memo. Assert rw_lock_own(). FindPage, FlaggedCheck: Assert rw_lock_own_flagged().	9 years ago
Marko Mäkelä	63574f1275	MDEV-11690 Remove UNIV_HOTBACKUP The InnoDB source code contains quite a few references to a closed-source hot backup tool which was originally called InnoDB Hot Backup (ibbackup) and later incorporated in MySQL Enterprise Backup. The open source backup tool XtraBackup uses the full database for recovery. So, the references to UNIV_HOTBACKUP are only cluttering the source code.	9 years ago
Jan Lindström	fec844aca8	Merge InnoDB 5.7 from mysql-5.7.14. Contains also: MDEV-10549 mysqld: sql/handler.cc:2692: int handler::ha_index_first(uchar*): Assertion `table_share->tmp_table != NO_TMP_TABLE \|\| m_lock_type != 2' failed. (branch bb-10.2-jan) Unlike MySQL, InnoDB still uses THR_LOCK in MariaDB MDEV-10548 Some of the debug sync waits do not work with InnoDB 5.7 (branch bb-10.2-jan) enable tests that were fixed in MDEV-10549 MDEV-10548 Some of the debug sync waits do not work with InnoDB 5.7 (branch bb-10.2-jan) fix main.innodb_mysql_sync - re-enable online alter for partitioned innodb tables	9 years ago
Jan Lindström	2e814d4702	Merge InnoDB 5.7 from mysql-5.7.9. Contains also MDEV-10547: Test multi_update_innodb fails with InnoDB 5.7 The failure happened because 5.7 has changed the signature of the bool handler::primary_key_is_clustered() const virtual function ("const" was added). InnoDB was using the old signature which caused the function not to be used. MDEV-10550: Parallel replication lock waits/deadlock handling does not work with InnoDB 5.7 Fixed mutexing problem on lock_trx_handle_wait. Note that rpl_parallel and rpl_optimistic_parallel tests still fail. MDEV-10156 : Group commit tests fail on 10.2 InnoDB (branch bb-10.2-jan) Reason: incorrect merge MDEV-10550: Parallel replication can't sync with master in InnoDB 5.7 (branch bb-10.2-jan) Reason: incorrect merge	9 years ago
Sergei Golubchik	720e04ff67	5.6.31	9 years ago
Sergei Golubchik	6d06fbbd1d	move to storage/innobase	11 years ago
Sergei Golubchik	14a142fca6	move to storage/xtradb	11 years ago
Sergei Golubchik	0175860925	initial commit into innodb-5.6 mergetree	12 years ago
Sergei Golubchik	52c26f7a1f	Percona-Server-5.6.14-rel62.0.tar.gz	12 years ago
Sergei Golubchik	63efc652b5	rename .c -> .cc	12 years ago
Sergei Golubchik	086b54c281	Percona-Server-5.5.30-rel30.1.tar.gz	13 years ago
unknown	1d812468d5	Updated with changes from Percona Server 5.1.66-rel14.1 tarball.	13 years ago
Sergei Golubchik	6de579241e	XtraDB 1.1.8-20.1 from Percona-Server 5.5.24-rel26.0	14 years ago
unknown	6afbf295f0	Updated with XtraDB from Percona Server 5.5.17-rel22.1 Files copied from Percona-Server-5.5.17-rel22.1.tar.gz source tarball.	14 years ago
unknown	190ccd244b	Updated with changes from Percona Server 5.1.56-13, from lp:percona-server/5.1, tag Percona-Server-5.1.59-13.0. Merged: revid:ignacio.nin@percona.com-20111016133841-fzpr5s89n13ft1s1	14 years ago
Sergei Golubchik	d4d7a8fa62	applied percona patches to InnoDB as of 5.5.15	14 years ago
Sergei Golubchik	54ab9ea20f	xtradb 5.5.13	14 years ago
unknown	fecd255a16	Updated with changes from Percona Server 5.1.56-12.7, from lp:~percona-dev/percona-server/release-5.1.56-12.7 percona-server-5.1.56-12.7 as of April 29, 2011. Merged: revid:ignacio.nin@percona.com-20110427224434-e5a4kpyfwvj641q3	15 years ago
unknown	0c6afe17dc	updated with changes from Percona-Server-5.1.47-11.2	15 years ago

1 2

69 Commits (041a32abcdd9791761a15d93e77ff2ea7dbc9d7c)