mariadb

Commit Graph

Author	SHA1	Message	Date
Marko Mäkelä	fde5386d16	Add dict_table_t::not_redundant()	7 years ago
Marko Mäkelä	3ea7de9a78	MDEV-15562 post-push cleanup dict_instant_t::non_pk_col_map: Change the type to uint16_t The persistent storage for this metadata is only 16 bits. dict_table_t::serialise_columns(): Define inline.	7 years ago
Marko Mäkelä	0e5a4ac253	MDEV-15662 Instant DROP COLUMN or changing the order of columns Allow ADD COLUMN anywhere in a table, not only adding as the last column. Allow instant DROP COLUMN and instant changing the order of columns. The added columns will always be added last in clustered index records. In new records, instantly dropped columns will be stored as NULL or empty when possible. Information about dropped and reordered columns will be written in a metadata BLOB (mblob), which is stored before the first 'user' field in the hidden metadata record at the start of the clustered index. The presence of mblob is indicated by setting the delete-mark flag in the metadata record. The metadata BLOB stores the number of clustered index fields, followed by an array of column information for each field. For dropped columns, we store the NOT NULL flag, the fixed length, and for variable-length columns, whether the maximum length exceeded 255 bytes. For non-dropped columns, we store the column position. Unlike with MDEV-11369, when a table becomes empty, it cannot be converted back to the canonical format. The reason for this is that other threads may hold cached objects such as row_prebuilt_t::ins_node that could refer to dropped or reordered index fields. For instant DROP COLUMN and ROW_FORMAT=COMPACT or ROW_FORMAT=DYNAMIC, we must store the n_core_null_bytes in the root page, so that the chain of node pointer records can be followed in order to reach the leftmost leaf page where the metadata record is located. If the mblob is present, we will zero-initialize the strings "infimum" and "supremum" in the root page, and use the last byte of "supremum" for storing the number of null bytes (which are allocated but useless on node pointer pages). This is necessary for btr_cur_instant_init_metadata() to be able to navigate to the mblob. If the PRIMARY KEY contains any variable-length column and some nullable columns were instantly dropped, the dict_index_t::n_nullable in the data dictionary could be smaller than it actually is in the non-leaf pages. Because of this, the non-leaf pages could use more bytes for the null flags than the data dictionary expects, and we could be reading the lengths of the variable-length columns from the wrong offset, and thus reading the child page number from wrong place. This is the result of two design mistakes that involve unnecessary storage of data: First, it is nonsense to store any data fields for the leftmost node pointer records, because the comparisons would be resolved by the MIN_REC_FLAG alone. Second, there cannot be any null fields in the clustered index node pointer fields, but we nevertheless reserve space for all the null flags. Limitations (future work): MDEV-17459 Allow instant ALTER TABLE even if FULLTEXT INDEX exists MDEV-17468 Avoid table rebuild on operations on generated columns MDEV-17494 Refuse ALGORITHM=INSTANT when the row size is too large btr_page_reorganize_low(): Preserve any metadata in the root page. Call lock_move_reorganize_page() only after restoring the "infimum" and "supremum" records, to avoid a memcmp() assertion failure. dict_col_t::DROPPED: Magic value for dict_col_t::ind. dict_col_t::clear_instant(): Renamed from dict_col_t::remove_instant(). Do not assert that the column was instantly added, because we sometimes call this unconditionally for all columns. Convert an instantly added column to a "core column". The old name remove_instant() could be mistaken to refer to "instant DROP COLUMN". dict_col_t::is_added(): Rename from dict_col_t::is_instant(). dtype_t::metadata_blob_init(): Initialize the mblob data type. dtuple_t::is_metadata(), dtuple_t::is_alter_metadata(), upd_t::is_metadata(), upd_t::is_alter_metadata(): Check if info_bits refer to a metadata record. dict_table_t::instant: Metadata about dropped or reordered columns. dict_table_t::prepare_instant(): Prepare ha_innobase_inplace_ctx::instant_table for instant ALTER TABLE. innobase_instant_try() will pass this to dict_table_t::instant_column(). On rollback, dict_table_t::rollback_instant() will be called. dict_table_t::instant_column(): Renamed from instant_add_column(). Add the parameter col_map so that columns can be reordered. Copy and adjust v_cols[] as well. dict_table_t::find(): Find an old column based on a new column number. dict_table_t::serialise_columns(), dict_table_t::deserialise_columns(): Convert the mblob. dict_index_t::instant_metadata(): Create the metadata record for instant ALTER TABLE. Invoke dict_table_t::serialise_columns(). dict_index_t::reconstruct_fields(): Invoked by dict_table_t::deserialise_columns(). dict_index_t::clear_instant_alter(): Move the fields for the dropped columns to the end, and sort the surviving index fields in ascending order of column position. ha_innobase::check_if_supported_inplace_alter(): Do not allow adding a FTS_DOC_ID column if a hidden FTS_DOC_ID column exists due to FULLTEXT INDEX. (This always required ALGORITHM=COPY.) instant_alter_column_possible(): Add a parameter for InnoDB table, to check for additional conditions, such as the maximum number of index fields. ha_innobase_inplace_ctx::first_alter_pos: The first column whose position is affected by instant ADD, DROP, or changing the order of columns. innobase_build_col_map(): Skip added virtual columns. prepare_inplace_add_virtual(): Correctly compute num_to_add_vcol. Remove some unnecessary code. Note that the call to innodb_base_col_setup() should be executed later. commit_try_norebuild(): If ctx->is_instant(), let the virtual columns be added or dropped by innobase_instant_try(). innobase_instant_try(): Fill in a zero default value for the hidden column FTS_DOC_ID (to reduce the work needed in MDEV-17459). If any columns were dropped or reordered (or added not last), delete any SYS_COLUMNS records for the following columns, and insert SYS_COLUMNS records for all subsequent stored columns as well as for all virtual columns. If any virtual column is dropped, rewrite all virtual column metadata. Use a shortcut only for adding virtual columns. This is because innobase_drop_virtual_try() assumes that the dropped virtual columns still exist in ctx->old_table. innodb_update_cols(): Renamed from innodb_update_n_cols(). innobase_add_one_virtual(), innobase_insert_sys_virtual(): Change the return type to bool, and invoke my_error() when detecting an error. innodb_insert_sys_columns(): Insert a record into SYS_COLUMNS. Refactored from innobase_add_one_virtual() and innobase_instant_add_col(). innobase_instant_add_col(): Replace the parameter dfield with type. innobase_instant_drop_cols(): Drop matching columns from SYS_COLUMNS and all columns from SYS_VIRTUAL. innobase_add_virtual_try(), innobase_drop_virtual_try(): Let the caller invoke innodb_update_cols(). innobase_rename_column_try(): Skip dropped columns. commit_cache_norebuild(): Update table->fts->doc_col. dict_mem_table_col_rename_low(): Skip dropped columns. trx_undo_rec_get_partial_row(): Skip dropped columns. trx_undo_update_rec_get_update(): Handle the metadata BLOB correctly. trx_undo_page_report_modify(): Avoid out-of-bounds access to record fields. Log metadata records consistently. Apparently, the first fields of a clustered index may be updated in an update_undo vector when the index is ID_IND of SYS_FOREIGN, as part of renaming the table during ALTER TABLE. Normally, updates of the PRIMARY KEY should be logged as delete-mark and an insert. row_undo_mod_parse_undo_rec(), row_purge_parse_undo_rec(): Use trx_undo_metadata. row_undo_mod_clust_low(): On metadata rollback, roll back the root page too. row_undo_mod_clust(): Relax an assertion. The delete-mark flag was repurposed for ALTER TABLE metadata records. row_rec_to_index_entry_impl(): Add the template parameter mblob and the optional parameter info_bits for specifying the desired new info bits. For the metadata tuple, allow conversion between the original format (ADD COLUMN only) and the generic format (with hidden BLOB). Add the optional parameter "pad" to determine whether the tuple should be padded to the index fields (on ALTER TABLE it should), or whether it should remain at its original size (on rollback). row_build_index_entry_low(): Clean up the code, removing redundant variables and conditions. For instantly dropped columns, generate a dummy value that is NULL, the empty string, or a fixed length of NUL bytes, depending on the type of the dropped column. row_upd_clust_rec_by_insert_inherit_func(): On the update of PRIMARY KEY of a record that contained a dropped column whose value was stored externally, we will be inserting a dummy NULL or empty string value to the field of the dropped column. The externally stored column would eventually be dropped when purge removes the delete-marked record for the old PRIMARY KEY value. btr_index_rec_validate(): Recognize the metadata record. btr_discard_only_page_on_level(): Preserve the generic instant ALTER TABLE metadata. btr_set_instant(): Replaces page_set_instant(). This sets a clustered index root page to the appropriate format, or upgrades from the MDEV-11369 instant ADD COLUMN to generic ALTER TABLE format. btr_cur_instant_init_low(): Read and validate the metadata BLOB page before reconstructing the dictionary information based on it. btr_cur_instant_init_metadata(): Do not read any lengths from the metadata record header before reading the BLOB. At this point, we would not actually know how many nullable fields the metadata record contains. btr_cur_instant_root_init(): Initialize n_core_null_bytes in one of two possible ways. btr_cur_trim(): Handle the mblob record. row_metadata_to_tuple(): Convert a metadata record to a data tuple, based on the new info_bits of the metadata record. btr_cur_pessimistic_update(): Invoke row_metadata_to_tuple() if needed. Invoke dtuple_convert_big_rec() for metadata records if the record is too large, or if the mblob is not yet marked as externally stored. btr_cur_optimistic_delete_func(), btr_cur_pessimistic_delete(): When the last user record is deleted, do not delete the generic instant ALTER TABLE metadata record. Only delete MDEV-11369 instant ADD COLUMN metadata records. btr_cur_optimistic_insert(): Avoid unnecessary computation of rec_size. btr_pcur_store_position(): Allow a logically empty page to contain a metadata record for generic ALTER TABLE. REC_INFO_DEFAULT_ROW_ADD: Renamed from REC_INFO_DEFAULT_ROW. This is for the old instant ADD COLUMN (MDEV-11369) only. REC_INFO_DEFAULT_ROW_ALTER: The more generic metadata record, with additional information for dropped or reordered columns. rec_info_bits_valid(): Remove. The only case when this would fail is when the record is the generic ALTER TABLE metadata record. rec_is_alter_metadata(): Check if a record is the metadata record for instant ALTER TABLE (other than ADD COLUMN). NOTE: This function must not be invoked on node pointer records, because the delete-mark flag in those records may be set (it is garbage), and then a debug assertion could fail because index->is_instant() does not necessarily hold. rec_is_add_metadata(): Check if a record is MDEV-11369 ADD COLUMN metadata record (not more generic instant ALTER TABLE). rec_get_converted_size_comp_prefix_low(): Assume that the metadata field will be stored externally. In dtuple_convert_big_rec() during the rec_get_converted_size() call, it would not be there yet. rec_get_converted_size_comp(): Replace status,fields,n_fields with tuple. rec_init_offsets_comp_ordinary(), rec_get_converted_size_comp_prefix_low(), rec_convert_dtuple_to_rec_comp(): Add template<bool mblob = false>. With mblob=true, process a record with a metadata BLOB. rec_copy_prefix_to_buf(): Assert that no fields beyond the key and system columns are being copied. Exclude the metadata BLOB field. rec_convert_dtuple_to_metadata_comp(): Convert an alter metadata tuple into a record. row_upd_index_replace_metadata(): Apply an update vector to an alter_metadata tuple. row_log_allocate(): Replace dict_index_t::is_instant() with a more appropriate condition that ignores dict_table_t::instant. Only a table on which the MDEV-11369 ADD COLUMN was performed can "lose its instantness" when it becomes empty. After instant DROP COLUMN or reordering columns, we cannot simply convert the table to the canonical format, because the data dictionary cache and all possibly existing references to it from other client connection threads would have to be adjusted. row_quiesce_write_index_fields(): Do not crash when the table contains an instantly dropped column. Thanks to Thirunarayanan Balathandayuthapani for discussing the design and implementing an initial prototype of this. Thanks to Matthias Leich for testing.	7 years ago
Marko Mäkelä	f545e3cfa9	MDEV-15562: Remove dict_table_t::rollback_instant(unsigned n) On the rollback of changes to SYS_COLUMNS, MDEV-15562 will break the assumption that the only instantaneous changes to columns are the addition to the end of the column list. The function dict_table_t::rollback_instant(unsigned n) is inherently incompatible with instantly dropping or reordering columns. When a change to SYS_COLUMNS is rolled back, we must simply evict the affected table definition, at the end of the rollback. We cannot free the table object immediately, because the current transaction that is being rolled back may be holding a lock on the table and its metadata record. dict_table_remove_from_cache_low(): Replaced by dict_table_remove_from_cache(). dict_table_remove_from_cache(): Add a third parameter keep=false, so that the table can be freed by the caller. trx_lock_t::evicted_tables: List of tables on which trx_t::evict_table() was invoked. trx_t::evict_table(): Evict a table definition during rollback. trx_commit_in_memory(): Empty the trx->lock.evicted_tables list after the locks were released, by freeing the table objects. row_undo_ins_remove_clust_rec(), row_undo_mod_clust_low(): Invoke trx_t::evict_table() on the affected table if a change to SYS_COLUMNS is being rolled back.	7 years ago
Marko Mäkelä	cd08173490	MDEV-15562: Add dict_index_t::first_user_field() dict_index_t::first_user_field(): Return the first data field in a clustered index, that is, the field after the PRIMARY KEY and the two system columns DB_TRX_ID, DB_ROLL_PTR. dtuple_convert_big_rec(): Remove some local variables.	7 years ago
Marko Mäkelä	85cdb63f77	Correct a function comment	7 years ago
Marko Mäkelä	754727bb8a	MDEV-14378 In ALGORITHM=INPLACE, use a common name for the intermediate tables or partitions This is a backport of commit `07e9ff1fe1`. Allow DROP TABLE `#mysql50##sql-...._.` to drop tables that were being rebuilt by ALGORITHM=INPLACE NOTE: If the server is killed after the table-rebuilding ALGORITHM=INPLACE commits inside InnoDB but before the .frm file has been replaced, then the recovery will involve something else than DROP TABLE. NOTE: If the server is killed in a true inplace ALTER TABLE commits inside InnoDB but before the .frm file has been replaced, then we are really out of luck. To properly handle that situation, we would need a transactional mysql.ddl_fixup table that directs recovery to rename or remove files. prepare_inplace_alter_table_dict(): Use the altered_table->s->table_name for generating the new_table_name. table_name_t::part_suffix: The start of the partition name suffix. table_name_t::dbend(): Return the end of the schema name. table_name_t::dblen(): Return the length of the schema name, in bytes. table_name_t::basename(): Return the name without the schema name. table_name_t::part(): Return the partition name, or NULL if none. row_drop_table_for_mysql(): Assert for #sql, not #sql-ib.	7 years ago
Marko Mäkelä	9dfef6e29b	Fix -Wclass-memaccess warnings in InnoDB,XtraDB	7 years ago
Marko Mäkelä	255328d393	MDEV-16131 Assertion failed in dict_index_t::instant_field_value() During a table-rebuilding online ALTER TABLE, if dict_index_t::remove_instant() was invoked on the source table (because it became empty), we would inadvertently change the way how log records are written and parsed. We must keep the online_log format unchanged throughout the whole table-rebuilding operation. dict_col_t::def_t: Name the type of dict_col_t::def_val. rec_get_n_add_field_len(), rec_set_n_add_field(): Define globally, because these will be needed in row_log_table_low(). rec_init_offsets_temp(), rec_init_offsets_comp_ordinary(): Add the parameter def_val for explicitly passing the default values of the instantly added columns of the source table, so that dict_index_t::instant_field_value() will not be called during row_log_table_apply(). This allows us to consistently parse the online_log records, even if the source table was converted to the canonical non-instant format during the rebuild operation. row_log_t::non_core_fields[]: The default values of the instantly added columns on the source table; copied during ha_innobase::prepare_inplace_alter_table() while the table is exclusively locked. row_log_t::instant_field_value(): Accessor to non_core_fields[], analogous to dict_index_t::instant_field_value(). row_log_table_low(): Add fake_extra_size bytes to the record header if the source table was converted to the canonical format during the operation. row_log_allocate(): Initialize row_log_t::non_core_fields.	7 years ago
Thirunarayanan Balathandayuthapani	d897b4dc25	MDEV-15855: Use atomics for dict_table_t::n_ref_count Make dict_table_t::n_ref_count private, and protect it with a combination of dict_sys->mutex and atomics. We want to be able to invoke dict_table_t::release() without holding dict_sys->mutex.	7 years ago
Thirunarayanan Balathandayuthapani	eb77f8cf8d	MDEV-16087 Inconsistent SELECT results when query cache is enabled The following conditions will decide the query cache retrieval or storing inside innodb: (1) There should not be any locks on the table. (2) Some other trx shouldn't invalidated the cache before the transaction started. (3) Read view shouldn't exist. If exists then the view low_limit_id should be greater than or equal to the transaction that invalidates the cache for the particular table. For read-only transaction: should satisfy the above (1) and (3) For read-write transaction: should satisfy the above (1), (2), (3). - Changed the variable from query_cache_inv_id to query_cache_inv_trx_id. - Moved the function row_search_check_if_query_cache_permitted from row0sel.h and made it as static function in ha_innodb.cc	8 years ago
Marko Mäkelä	8dc70c862b	MDEV-16376 ASAN: heap-use-after-free in gcol.innodb_virtual_debug After a failed ADD INDEX, dict_index_remove_from_cache_low() could iterate the index fields and dereference a freed virtual column object when trying to remove the index from the v_indexes of the virtual column. This regression was caused by a merge of MDEV-16119 InnoDB lock->index refers to a freed object. ha_innobase_inplace_ctx::clear_added_indexes(): Detach the indexes of uncommitted indexes from virtual columns, so that the iteration in dict_index_remove_from_cache_low() can be avoided. ha_innobase::prepare_inplace_alter_table(): Ignore uncommitted corrupted indexes when rejecting ALTER TABLE. (This minor bug was revealed by the extension of the test case.) dict_index_t::detach_columns(): Detach an index from virtual columns. Invoked by both dict_index_remove_from_cache_low() and ha_innobase_inplace_ctx::clear_added_indexes(). dict_col_t::detach(const dict_index_t& index): Detach an index from a column. dict_col_t::is_virtual(): Replaces dict_col_is_virtual(). dict_index_t::has_virtual(): Replaces dict_index_has_virtual().	8 years ago
Marko Mäkelä	5e84ea9634	MDEV-12266: Remove dict_table_is_discarded() The predicate dict_table_is_discarded() checks whether ALTER TABLE…DISCARD TABLESPACE has been executed. Replace most occurrences of dict_table_is_discarded() with checks of dict_table_t::space. A few checks for the flag DICT_TF2_DISCARDED are necessary; write them inline. Because !is_readable() implies !space, some checks for dict_table_is_discarded() were redundant.	8 years ago
Marko Mäkelä	e44ca6cc9c	MDEV-14825 Assertion `col->ord_part' in row_build_index_entry_low upon ROLLBACK or DELETE with concurrent ALTER on partitioned table If creating a secondary index fails (typically, ADD UNIQUE INDEX fails due to duplicate key), it is possible that concurrently running UPDATE or DELETE will access the index stub and hit the debug assertion. It does not make any sense to keep updating an uncommitted index whose creation has failed. dict_index_t::is_corrupted(): Replaces dict_index_is_corrupted(). Also take online_status into account. Replace some calls to dict_index_is_clust() with calls to dict_index_t::is_primary().	8 years ago
Marko Mäkelä	6c43068d63	MDEV-15060 Assertion in row_log_table_apply_op after instant ADD when the table is emptied during subsequent ALTER TABLE During an online table rebuild, a table could be emptied and converted from 'instant ADD' format to plain (pre-10.3) format. All online_log records for rebuilding the table must be written and parsed in the format of the table that existed at the start of the operation. row_log_t::n_core_fields: A new field for recording index->n_core_fields when online ALTER is initiated in row_log_allocate(). row_log_t::is_instant(): Determine if the log is in the instant format. Only invoked by the row_log_table_ family of functions. dict_index_t::get_n_nullable(): Remove is_instant() debug assertions. Because a table can be converted to non-instant format during a table-rebuilding ALTER TABLE, these assertions would be bogus when executing row_log_table_apply(). rec_init_offsets_temp(): Add the parameter n_core for passing the original index->n_core_fields. rec_init_offsets_temp(): Add a 3-parameter variant. rec_init_offsets_comp_ordinary(): Add the parameter n_core for passing the index->n_core_fields.	8 years ago
Marko Mäkelä	ba19764209	Fix most -Wsign-conversion in InnoDB Change innodb_buffer_pool_size, innodb_fill_factor to unsigned.	8 years ago
Monty	2ccd6716fc	Fix a lot of compiler warnings found by -Wunused	8 years ago
Marko Mäkelä	4cad42392a	MDEV-12266: Change dict_table_t::space to fil_space_t* InnoDB always keeps all tablespaces in the fil_system cache. The fil_system.LRU is only for closing file handles; the fil_space_t and fil_node_t for all data files will remain in main memory. Between startup to shutdown, they can only be created and removed by DDL statements. Therefore, we can let dict_table_t::space point directly to the fil_space_t. dict_table_t::space_id: A numeric tablespace ID for the corner cases where we do not have a tablespace. The most prominent examples are ALTER TABLE...DISCARD TABLESPACE or a missing or corrupted file. There are a few functional differences; most notably: (1) DROP TABLE will delete matching .ibd and .cfg files, even if they were not attached to the data dictionary. (2) Some error messages will report file names instead of numeric IDs. There still are many functions that use numeric tablespace IDs instead of fil_space_t, and many functions could be converted to fil_space_t member functions. Also, Tablespace and Datafile should be merged with fil_space_t and fil_node_t. page_id_t and buf_page_get_gen() could use fil_space_t& instead of a numeric ID, and after moving to a single buffer pool (MDEV-15058), buf_pool_t::page_hash could be moved to fil_space_t::page_hash. FilSpace: Remove. Only few calls to fil_space_acquire() will remain, and gradually they should be removed. mtr_t::set_named_space_id(ulint): Renamed from set_named_space(), to prevent accidental calls to this slower function. Very few callers remain. fseg_create(), fsp_reserve_free_extents(): Take fil_space_t as a parameter instead of a space_id. fil_space_t::rename(): Wrapper for fil_rename_tablespace_check(), fil_name_write_rename(), fil_rename_tablespace(). Mariabackup passes the parameter log=false; InnoDB passes log=true. dict_mem_table_create(): Take fil_space_t* instead of space_id as parameter. dict_process_sys_tables_rec_and_mtr_commit(): Replace the parameter 'status' with 'bool cached'. dict_get_and_save_data_dir_path(): Avoid copying the fil_node_t::name. fil_ibd_open(): Return the tablespace. fil_space_t::set_imported(): Replaces fil_space_set_imported(). truncate_t: Change many member function parameters to fil_space_t*, and remove page_size parameters. row_truncate_prepare(): Merge to its only caller. row_drop_table_from_cache(): Assert that the table is persistent. dict_create_sys_indexes_tuple(): Write SYS_INDEXES.SPACE=FIL_NULL if the tablespace has been discarded. row_import_update_discarded_flag(): Remove a constant parameter.	8 years ago
Marko Mäkelä	428e02895b	MDEV-12266: Remove dict_index_t::table_name Replace index->table_name with index->table->name.	8 years ago
Marko Mäkelä	604fea1ad6	MDEV-12266: Remove dict_index_t::space We can rely on the dict_table_t::space. All indexes of a table object are always in the same tablespace. (For fulltext indexes, the data is located in auxiliary tables, and these will continue to have their own table objects, separate from the main table.)	8 years ago
Marko Mäkelä	e7980f9cee	MDEV-12266: Add dict_index_t::set_modified(mtr) This is a non-functional change (pure refactoring).	8 years ago
Marko Mäkelä	60ef478c2e	Remove an unnecessary global declaration	8 years ago
Eugene Kosov	19a182b1ea	MDEV-15364 FOREIGN CASCADE operations in system versioned referenced tables Make foreign system versioning tables work in CASCADE UPDATE/SET NULL. In that case basically row update is performed. This patch makes insert of a historical row performed too. row_update_versioned_insert(): restores btr_pcur_t, reads row from it, makes row historical and inserts to table. row_ins_check_foreign_constraint(): disable constraint check for historical rows because it has no sense. Also check will fail always, because referenced table is updated at that point. row_update_cascade_for_mysql(): insert historical row for system versioning tables before updating current row. revert DATA_VERSIONED -> DATA_UNVERSIONED	8 years ago
Eugene Kosov	fd73c6dda4	Vers IB: Mark unversioned fields in system versioning tables Some fields in system-versioned table may be unversioned. SQL layer marks unversioned. And this patch makes InnoDB mark unversioned too because of two reasons: 1) by default fields are versioned 2) most of fields are expected to be versioned dtype_t::vers_sys_field(): fixed return true on row_start/row_end dict_col_t::vers_sys_field(): fixed return true on row_start/row_end	8 years ago
Aleksey Midenkov	0cf97ad5b9	IB: CASCADE operation for DELETE * Removed "Not supported for TIMESTAMP-based" error * Fixed code duplication with node->vers_set_fields() * Recovered foreign.test [closes tempesta-tech#473]	8 years ago
Sergey Vojtovich	131d9a5d0c	Allocate lock_sys statically There is only one lock_sys. Allocate it statically in order to avoid dereferencing a pointer whenever accessing it. Also, align some members to their own cache line in order to avoid false sharing. lock_sys_t::create(): The deferred constructor. lock_sys_t::close(): The early destructor.	8 years ago
Marko Mäkelä	db25305780	MDEV-14407 Assertion failure during rollback Rollback attempted to dereference DB_ROLL_PTR=0, which cannot possibly be a valid undo log pointer. A safe canonical value would be roll_ptr_t(1) << ROLL_PTR_INSERT_FLAG_POS which is what was chosen in MDEV-12288. This bug was reproduced in 10.3 only. Potentially, the problem could have been introduced by MDEV-11415, which suppresses undo logging for ALGORITHM=COPY operations. In those operations, we should actually have written the safe value of DB_ROLL_PTR instead of writing 0. However, the test in commit `5421e3aee7` demonstrates that access to the rebuilt table by earlier-started transactions should actually have been refused with ER_TABLE_DEF_CHANGED. btr_cur_ins_lock_and_undo(): When undo logging is disabled, use the safe value of DB_ROLL_PTR. btr_cur_optimistic_insert(): Validate the DB_TRX_ID,DB_ROLL_PTR before inserting into a clustered index leaf page. ins_node_t::sys_buf[]: Replaces row_id_buf and trx_id_buf and some heap usage. row_ins_alloc_sys_fields(): Initialize ins_node_t::sys_buf[]. trx_undo_page_report_modify(): Assert that the DB_ROLL_PTR is not 0. trx_undo_get_undo_rec_low(): Assert that the roll_ptr is valid before trying to dereference it. dict_index_t::is_primary(): Check if the index is the primary key.	8 years ago
Marko Mäkelä	7660d8c94e	Remove dict_table_t::is_clust() Replace all occurrences of the is_clust() method with is_primary(), because that is what is actually meant. (Also the change buffer tree would count as a clustered index.)	8 years ago
Marko Mäkelä	609d0a9194	MDEV-14407 Assertion failure during rollback Rollback attempted to dereference DB_ROLL_PTR=0, which cannot possibly be a valid undo log pointer. A safer canonical value would be roll_ptr_t(1) << ROLL_PTR_INSERT_FLAG_POS which is what was chosen in MDEV-12288, corresponding to reset_trx_id. No deterministic test case for the bug was found. The simplest test cases may be related to MDEV-11415, which suppresses undo logging for ALGORITHM=COPY operations. In those operations, in the spirit of MDEV-12288, we should actually have written reset_trx_id instead of using the transaction identifier of the current transaction (and a bogus value of DB_ROLL_PTR=0). However, thanks to MySQL Bug#28432 which I had fixed in MySQL 5.6.8 as part of WL#6255, access to the rebuilt table by earlier-started transactions should actually have been refused with ER_TABLE_DEF_CHANGED. reset_trx_id: Move the definition to data0type.cc and the declaration to data0type.h. btr_cur_ins_lock_and_undo(): When undo logging is disabled, use the safe value that corresponds to reset_trx_id. btr_cur_optimistic_insert(): Validate the DB_TRX_ID,DB_ROLL_PTR before inserting into a clustered index leaf page. ins_node_t::sys_buf[]: Replaces row_id_buf and trx_id_buf and some heap usage. row_ins_alloc_sys_fields(): Init ins_node_t::sys_buf[] to reset_trx_id. row_ins_buf(): Only if undo logging is enabled, copy trx->id to node->sys_buf. Otherwise, rely on the initialization in row_ins_alloc_sys_fields(). row_purge_reset_trx_id(): Invoke mlog_write_string() with reset_trx_id directly. (No functional change.) trx_undo_page_report_modify(): Assert that the DB_ROLL_PTR is not 0. trx_undo_get_undo_rec_low(): Assert that the roll_ptr is valid before trying to dereference it. dict_index_t::is_primary(): Check if the index is the primary key. PageConverter::adjust_cluster_record(): Fix MDEV-15249 Crash in MVCC read after IMPORT TABLESPACE by resetting the system fields to reset_trx_id instead of writing the current transaction ID (which will be committed at the end of the IMPORT TABLESPACE) and DB_ROLL_PTR=0. This can partially be viewed as a follow-up fix of MDEV-12288, because IMPORT should already then have written DB_TRX_ID=0 and DB_ROLL_PTR=1<<55 to prevent unnecessary DB_TRX_ID lookups in subsequent accesses to the table.	8 years ago
Marko Mäkelä	10590dd39c	MDEV-15199 Referential integrity broken in ON DELETE CASCADE MDEV-14222 Unnecessary 'cascade' memory allocation for every updated row when there is no FOREIGN KEY This reverts the MySQL 5.7.2 change `377774689b` which introduced these problems. MariaDB 10.2.2 inherited these problems in commit `2e814d4702`. The FOREIGN KEY CASCADE and SET NULL operations implemented as procedural recursion are consuming more than 8 kilobytes of stack (9 stack frames) per iteration in a non-debug GNU/Linux AMD64 build. This is why we need to limit the maximum recursion depth to 15 steps instead of the 255 that it used to be in MySQL 5.7 and MariaDB 10.2. A corresponding change was made in MySQL 5.7.21 in `7b26dc98a6`	8 years ago
Vladislav Vaintroub	6c279ad6a7	MDEV-15091 : Windows, 64bit: reenable and fix warning C4267 (conversion from 'size_t' to 'type', possible loss of data) Handle string length as size_t, consistently (almost always:)) Change function prototypes to accept size_t, where in the past ulong or uint were used. change local/member variables to size_t when appropriate. This fix excludes rocksdb, spider,spider, sphinx and connect for now.	8 years ago
Marko Mäkelä	0ba6aaf030	MDEV-11415 Remove excessive undo logging during ALTER TABLE…ALGORITHM=COPY If a crash occurs during ALTER TABLE…ALGORITHM=COPY, InnoDB would spend a lot of time rolling back writes to the intermediate copy of the table. To reduce the amount of busy work done, a work-around was introduced in commit `fd069e2bb3` in MySQL 4.1.8 and 5.0.2, to commit the transaction after every 10,000 inserted rows. A proper fix would have been to disable the undo logging altogether and to simply drop the intermediate copy of the table on subsequent server startup. This is what happens in MariaDB 10.3 with MDEV-14717,MDEV-14585. In MariaDB 10.2, the intermediate copy of the table would be left behind with a name starting with the string #sql. This is a backport of a bug fix from MySQL 8.0.0 to MariaDB, contributed by jixianliang <271365745@qq.com>. Unlike recent MySQL, MariaDB supports ALTER IGNORE. For that operation InnoDB must for now keep the undo logging enabled, so that the latest row can be rolled back in case of an error. In Galera cluster, the LOAD DATA statement will retain the existing behaviour and commit the transaction after every 10,000 rows if the parameter wsrep_load_data_splitting=ON is set. The logic to do so (the wsrep_load_data_split() function and the call handler::extra(HA_EXTRA_FAKE_START_STMT)) are joint work by Ji Xianliang and Marko Mäkelä. The original fix: Author: Thirunarayanan Balathandayuthapani <thirunarayanan.balathandayuth@oracle.com> Date: Wed Dec 2 16:09:15 2015 +0530 Bug#17479594 AVOID INTERMEDIATE COMMIT WHILE DOING ALTER TABLE ALGORITHM=COPY Problem: During ALTER TABLE, we commit and restart the transaction for every 10,000 rows, so that the rollback after recovery would not take so long. Fix: Suppress the undo logging during copy alter operation. If fts_index is present then insert directly into fts auxiliary table rather than doing at commit time. ha_innobase::num_write_row: Remove the variable. ha_innobase::write_row(): Remove the hack for committing every 10000 rows. row_lock_table_for_mysql(): Remove the extra 2 parameters. lock_get_src_table(), lock_is_table_exclusive(): Remove. Reviewed-by: Marko Mäkelä <marko.makela@oracle.com> Reviewed-by: Shaohua Wang <shaohua.wang@oracle.com> Reviewed-by: Jon Olav Hauglid <jon.hauglid@oracle.com>	8 years ago
Marko Mäkelä	041a32abcd	Remove trx_mod_tables_t::vers_by_trx Only invoke set_versioned() on trx_id versioned tables. dict_table_t::versioned_by_id(): New accessor, to determine if a table is system versioned by transaction ID.	8 years ago
Marko Mäkelä	92d233a512	MDEV-15061 TRUNCATE must honor InnoDB table locks Traditionally, DROP TABLE and TRUNCATE TABLE discarded any locks that may have been held on the table. This feels like an ACID violation. Probably most occurrences of it were prevented by meta-data locks (MDL) which were introduced in MySQL 5.5. dict_table_t::n_foreign_key_checks_running: Reduce the number of non-debug checks. lock_remove_all_on_table(), lock_remove_all_on_table_for_trx(): Remove. ha_innobase::truncate(): Acquire an exclusive InnoDB table lock before proceeding. DROP TABLE and DISCARD/IMPORT were already doing this. row_truncate_table_for_mysql(): Convert the already started transaction into a dictionary operation, and do not invoke lock_remove_all_on_table(). row_mysql_table_id_reassign(): Do not call lock_remove_all_on_table(). This function is only used in ALTER TABLE...DISCARD/IMPORT TABLESPACE, which is already holding an exclusive InnoDB table lock. TODO: Make n_foreign_key_checks running a debug-only variable. This would require two fixes: (1) DROP TABLE: Exclusively lock the table beforehand, to prevent the possibility of concurrently running foreign key checks (which would acquire a table IS lock and then record S locks). (2) RENAME TABLE: Find out if n_foreign_key_checks_running>0 actually constitutes a potential problem.	8 years ago
Marko Mäkelä	906ce0962d	MDEV-7049 MySQL#74585 - InnoDB: Failing assertion: *mbmaxlen < 5 in file ha_innodb.cc line 1904 InnoDB limited the maximum number of bytes per character to 4. But, the filename character set that was introduced in MySQL 5.1 uses up to 5 bytes per character. To allow InnoDB tables to be created with wider characters, let us split the mbminmaxlen fields into mbminlen, mbmaxlen, and increase the limit to 7 bytes per character. This will increase the payload size of dtype_t and dict_col_t by one bit. The storage size will be unchanged (54 bits and 77 bits will use the same number of bytes as the previous sizes 53 and 76 bits).	8 years ago
Aleksey Midenkov	b55a149194	Timestamp-based versioning for InnoDB [closes #209 ] * Removed integer_fields check * Reworked Vers_parse_info::check_sys_fields() * Misc renames * versioned as vers_sys_type_t * Removed versioned_by_sql(), versioned_by_engine() versioned() works as before; versioned(VERS_TIMESTAMP) is versioned_by_sql(); versioned(VERS_TRX_ID) is versioned_by_engine(). * create_tmp_table() fix * Foreign constraints for timestamp-based * Range auto-specifier fix * SQL: 1-row partition rotation fix [fixes #260] * Fix 'drop system versioning, algorithm=inplace'	8 years ago
Marko Mäkelä	07e9ff1fe1	MDEV-14378 In ALGORITHM=INPLACE, use a common name for the intermediate tables or partitions Allow DROP TABLE `#mysql50##sql-...._.` to drop tables that were being rebuilt by ALGORITHM=INPLACE NOTE: If the server is killed after the table-rebuilding ALGORITHM=INPLACE commits inside InnoDB but before the .frm file has been replaced, then the recovery will involve something else than DROP TABLE. NOTE: If the server is killed in a true inplace ALTER TABLE commits inside InnoDB but before the .frm file has been replaced, then we are really out of luck. To properly handle that situation, we would need a transactional mysql.ddl_fixup table that directs recovery to rename or remove files. prepare_inplace_alter_table_dict(): Use the altered_table->s->table_name for generating the new_table_name. table_name_t::part_suffix: The start of the partition name suffix. table_name_t::dbend(): Return the end of the schema name. table_name_t::dblen(): Return the length of the schema name, in bytes. table_name_t::basename(): Return the name without the schema name. table_name_t::part(): Return the partition name, or NULL if none. row_drop_table_for_mysql(): Assert for #sql, not #sql-ib.	8 years ago
Marko Mäkelä	e9bc0f75ef	MDEV-5834 cleanup: Inline two tiny functions	8 years ago
Marko Mäkelä	8be7548085	Follow-up to MDEV-12698: Adjust some comments The function dict_stats_update_if_needed() replaced row_update_statistics_if_needed(). Adjust the comments accordingly.	8 years ago
Marko Mäkelä	0b89a42ffc	Remove the flag vers_update_trt THD::vers_update_trt, trx_t::vers_update_trt, trx_savept_t::vers_update_trt: Remove. Instead, determine from trx_t::mod_tables whether versioned columns were affected by the transaction. handlerton::prepare_commit_versioned: Replaces vers_get_trt_data. Return the transaction start ID and also the commit ID, in case the transaction modified any system-versioned columns (0 if not). TR_table::store_data(): Remove (merge with update() below). TR_table::update(): Add the parameters start_id, end_id. ha_commit_trans(): Remove a condition on SQLCOM_ALTER_TABLE. If we need something special for ALTER TABLE...ALGORITHM=INPLACE, that can be done inside InnoDB by modifying trx_t::mod_tables. innodb_prepare_commit_versioned(): Renamed from innodb_get_trt_data(). Check trx_t::mod_tables to see if any changes to versioned columns are present. trx_mod_table_time_t: A pair of logical timestamps, replacing the undo_no_t in trx_mod_tables_t. Keep track of not only the first modification to a persistent table in each transaction, but also the first modification of a versioned column in a table. dtype_t, dict_col_t: Add the accessor is_any_versioned(), to check if the type refers to a system-versioned user or system column. upd_t::affects_versioned(): Check if an update affects a versioned column. trx_undo_report_row_operation(): If a versioned column is affected by the update, invoke trx_mod_table_time_t::set_versioned(). trx_rollback_to_savepoint_low(): If all changes to versioned columns were rolled back, invoke trx_mod_table_time_t::rollback_versioned(), so that trx_mod_table_time_t::is_versioned() will no longer hold.	8 years ago
Marko Mäkelä	03fbfeef66	Identify system-versioned columns in the InnoDB dictionary Both bits DATA_VERSIONED will be set in prtype if the column is system-versioned. The bits will be 0 for normal unversioned columns. For the special columns identifying the logical start and end times of versions, only one bit will be set: DATA_VERS_START or DATA_VERS_END. create_table_info_t::create_table_def(), prepare_inplace_alter_table_dict(): Set prtype \|= DATA_VERSIONED for system-versioned columns. dfield_t::is_version_historical_end(): Determine if a data tuple field is_version_end() and contains a timestamp in the past (not TRX_ID_MAX). dtype_t, dict_col_t: Add the accessors is_versioned(), is_version_start(), is_version_end(). trx_id_max_bytes[]: The bit pattern of TRX_ID_MAX, for use with memcmp().	8 years ago
Aleksey Midenkov	33085349e9	IB, SQL: removed VTQ, added TRT on SQL layer [closes #305 ]	8 years ago
Aleksey Midenkov	8972291ac1	IB: misc fixes [#305 ] * System fields renamed * Removed row0ins.ic * Removed row_update_for_mysql() wrapper	8 years ago
Eugene Kosov	e674806282	IB: DICT_TF2_VERSIONED flag removed Reverted DICT_TF2_BITS (midenok)	8 years ago
Marko Mäkelä	e94c9d24f6	MDEV-14378 In ALGORITHM=INPLACE, use a common name for the intermediate tables or partitions Allow DROP TABLE `#mysql50##sql-...._.` to drop tables that were being rebuilt by ALGORITHM=INPLACE NOTE: If the server is killed after the table-rebuilding ALGORITHM=INPLACE commits inside InnoDB but before the .frm file has been replaced, then the recovery will involve something else than DROP TABLE. NOTE: If the server is killed in a true inplace ALTER TABLE commits inside InnoDB but before the .frm file has been replaced, then we are really out of luck. To properly handle that situation, we would need a transactional mysql.ddl_fixup table that directs recovery to rename or remove files. prepare_inplace_alter_table_dict(): Use the altered_table->s->table_name for generating the new_table_name. table_name_t::part_suffix: The start of the partition name suffix. table_name_t::dbend(): Return the end of the schema name. table_name_t::dblen(): Return the length of the schema name, in bytes. table_name_t::basename(): Return the name without the schema name. table_name_t::part(): Return the partition name, or NULL if none. row_drop_table_for_mysql(): Assert for #sql, not #sql-ib.	8 years ago
Vicențiu Ciorbaru	4ef64e01a7	5.6.38	8 years ago
Marko Mäkelä	c65fdcf7c8	MDEV-14062 Upgrade from 10.0 fails on assertion during ALTER TABLE The fix in MDEV-14022 was incomplete. We must adjust some more code for the fact that SYS_INDEXES records may lack the column MERGE_THRESHOLD that was "instantly" added in MySQL 5.7 and MariaDB 10.2.2. dict_boot(): Hard-core SYS_INDEXES.MERGE_THRESHOLD as DEFAULT NULL. dict_index_t::instant_field_value(), rec_offs_make_valid(): Tolerate the missing SYS_INDEXES.MERGE_THRESHOLD column.	8 years ago
Marko Mäkelä	a4948dafcd	MDEV-11369 Instant ADD COLUMN for InnoDB For InnoDB tables, adding, dropping and reordering columns has required a rebuild of the table and all its indexes. Since MySQL 5.6 (and MariaDB 10.0) this has been supported online (LOCK=NONE), allowing concurrent modification of the tables. This work revises the InnoDB ROW_FORMAT=REDUNDANT, ROW_FORMAT=COMPACT and ROW_FORMAT=DYNAMIC so that columns can be appended instantaneously, with only minor changes performed to the table structure. The counter innodb_instant_alter_column in INFORMATION_SCHEMA.GLOBAL_STATUS is incremented whenever a table rebuild operation is converted into an instant ADD COLUMN operation. ROW_FORMAT=COMPRESSED tables will not support instant ADD COLUMN. Some usability limitations will be addressed in subsequent work: MDEV-13134 Introduce ALTER TABLE attributes ALGORITHM=NOCOPY and ALGORITHM=INSTANT MDEV-14016 Allow instant ADD COLUMN, ADD INDEX, LOCK=NONE The format of the clustered index (PRIMARY KEY) is changed as follows: (1) The FIL_PAGE_TYPE of the root page will be FIL_PAGE_TYPE_INSTANT, and a new field PAGE_INSTANT will contain the original number of fields in the clustered index ('core' fields). If instant ADD COLUMN has not been used or the table becomes empty, or the very first instant ADD COLUMN operation is rolled back, the fields PAGE_INSTANT and FIL_PAGE_TYPE will be reset to 0 and FIL_PAGE_INDEX. (2) A special 'default row' record is inserted into the leftmost leaf, between the page infimum and the first user record. This record is distinguished by the REC_INFO_MIN_REC_FLAG, and it is otherwise in the same format as records that contain values for the instantly added columns. This 'default row' always has the same number of fields as the clustered index according to the table definition. The values of 'core' fields are to be ignored. For other fields, the 'default row' will contain the default values as they were during the ALTER TABLE statement. (If the column default values are changed later, those values will only be stored in the .frm file. The 'default row' will contain the original evaluated values, which must be the same for every row.) The 'default row' must be completely hidden from higher-level access routines. Assertions have been added to ensure that no 'default row' is ever present in the adaptive hash index or in locked records. The 'default row' is never delete-marked. (3) In clustered index leaf page records, the number of fields must reside between the number of 'core' fields (dict_index_t::n_core_fields introduced in this work) and dict_index_t::n_fields. If the number of fields is less than dict_index_t::n_fields, the missing fields are replaced with the column value of the 'default row'. Note: The number of fields in the record may shrink if some of the last instantly added columns are updated to the value that is in the 'default row'. The function btr_cur_trim() implements this 'compression' on update and rollback; dtuple::trim() implements it on insert. (4) In ROW_FORMAT=COMPACT and ROW_FORMAT=DYNAMIC records, the new status value REC_STATUS_COLUMNS_ADDED will indicate the presence of a new record header that will encode n_fields-n_core_fields-1 in 1 or 2 bytes. (In ROW_FORMAT=REDUNDANT records, the record header always explicitly encodes the number of fields.) We introduce the undo log record type TRX_UNDO_INSERT_DEFAULT for covering the insert of the 'default row' record when instant ADD COLUMN is used for the first time. Subsequent instant ADD COLUMN can use TRX_UNDO_UPD_EXIST_REC. This is joint work with Vin Chen (陈福荣) from Tencent. The design that was discussed in April 2017 would not have allowed import or export of data files, because instead of the 'default row' it would have introduced a data dictionary table. The test rpl.rpl_alter_instant is exactly as contributed in pull request #408. The test innodb.instant_alter is based on a contributed test. The redo log record format changes for ROW_FORMAT=DYNAMIC and ROW_FORMAT=COMPACT are as contributed. (With this change present, crash recovery from MariaDB 10.3.1 will fail in spectacular ways!) Also the semantics of higher-level redo log records that modify the PAGE_INSTANT field is changed. The redo log format version identifier was already changed to LOG_HEADER_FORMAT_CURRENT=103 in MariaDB 10.3.1. Everything else has been rewritten by me. Thanks to Elena Stepanova, the code has been tested extensively. When rolling back an instant ADD COLUMN operation, we must empty the PAGE_FREE list after deleting or shortening the 'default row' record, by calling either btr_page_empty() or btr_page_reorganize(). We must know the size of each entry in the PAGE_FREE list. If rollback left a freed copy of the 'default row' in the PAGE_FREE list, we would be unable to determine its size (if it is in ROW_FORMAT=COMPACT or ROW_FORMAT=DYNAMIC) because it would contain more fields than the rolled-back definition of the clustered index. UNIV_SQL_DEFAULT: A new special constant that designates an instantly added column that is not present in the clustered index record. len_is_stored(): Check if a length is an actual length. There are two magic length values: UNIV_SQL_DEFAULT, UNIV_SQL_NULL. dict_col_t::def_val: The 'default row' value of the column. If the column is not added instantly, def_val.len will be UNIV_SQL_DEFAULT. dict_col_t: Add the accessors is_virtual(), is_nullable(), is_instant(), instant_value(). dict_col_t::remove_instant(): Remove the 'instant ADD' status of a column. dict_col_t::name(const dict_table_t& table): Replaces dict_table_get_col_name(). dict_index_t::n_core_fields: The original number of fields. For secondary indexes and if instant ADD COLUMN has not been used, this will be equal to dict_index_t::n_fields. dict_index_t::n_core_null_bytes: Number of bytes needed to represent the null flags; usually equal to UT_BITS_IN_BYTES(n_nullable). dict_index_t::NO_CORE_NULL_BYTES: Magic value signalling that n_core_null_bytes was not initialized yet from the clustered index root page. dict_index_t: Add the accessors is_instant(), is_clust(), get_n_nullable(), instant_field_value(). dict_index_t::instant_add_field(): Adjust clustered index metadata for instant ADD COLUMN. dict_index_t::remove_instant(): Remove the 'instant ADD' status of a clustered index when the table becomes empty, or the very first instant ADD COLUMN operation is rolled back. dict_table_t: Add the accessors is_instant(), is_temporary(), supports_instant(). dict_table_t::instant_add_column(): Adjust metadata for instant ADD COLUMN. dict_table_t::rollback_instant(): Adjust metadata on the rollback of instant ADD COLUMN. prepare_inplace_alter_table_dict(): First create the ctx->new_table, and only then decide if the table really needs to be rebuilt. We must split the creation of table or index metadata from the creation of the dictionary table records and the creation of the data. In this way, we can transform a table-rebuilding operation into an instant ADD COLUMN operation. Dictionary objects will only be added to cache when table rebuilding or index creation is needed. The ctx->instant_table will never be added to cache. dict_table_t::add_to_cache(): Modified and renamed from dict_table_add_to_cache(). Do not modify the table metadata. Let the callers invoke dict_table_add_system_columns() and if needed, set can_be_evicted. dict_create_sys_tables_tuple(), dict_create_table_step(): Omit the system columns (which will now exist in the dict_table_t object already at this point). dict_create_table_step(): Expect the callers to invoke dict_table_add_system_columns(). pars_create_table(): Before creating the table creation execution graph, invoke dict_table_add_system_columns(). row_create_table_for_mysql(): Expect all callers to invoke dict_table_add_system_columns(). create_index_dict(): Replaces row_merge_create_index_graph(). innodb_update_n_cols(): Renamed from innobase_update_n_virtual(). Call my_error() if an error occurs. btr_cur_instant_init(), btr_cur_instant_init_low(), btr_cur_instant_root_init(): Load additional metadata from the clustered index and set dict_index_t::n_core_null_bytes. This is invoked when table metadata is first loaded into the data dictionary. dict_boot(): Initialize n_core_null_bytes for the four hard-coded dictionary tables. dict_create_index_step(): Initialize n_core_null_bytes. This is executed as part of CREATE TABLE. dict_index_build_internal_clust(): Initialize n_core_null_bytes to NO_CORE_NULL_BYTES if table->supports_instant(). row_create_index_for_mysql(): Initialize n_core_null_bytes for CREATE TEMPORARY TABLE. commit_cache_norebuild(): Call the code to rename or enlarge columns in the cache only if instant ADD COLUMN is not being used. (Instant ADD COLUMN would copy all column metadata from instant_table to old_table, including the names and lengths.) PAGE_INSTANT: A new 13-bit field for storing dict_index_t::n_core_fields. This is repurposing the 16-bit field PAGE_DIRECTION, of which only the least significant 3 bits were used. The original byte containing PAGE_DIRECTION will be accessible via the new constant PAGE_DIRECTION_B. page_get_instant(), page_set_instant(): Accessors for the PAGE_INSTANT. page_ptr_get_direction(), page_get_direction(), page_ptr_set_direction(): Accessors for PAGE_DIRECTION. page_direction_reset(): Reset PAGE_DIRECTION, PAGE_N_DIRECTION. page_direction_increment(): Increment PAGE_N_DIRECTION and set PAGE_DIRECTION. rec_get_offsets(): Use the 'leaf' parameter for non-debug purposes, and assume that heap_no is always set. Initialize all dict_index_t::n_fields for ROW_FORMAT=REDUNDANT records, even if the record contains fewer fields. rec_offs_make_valid(): Add the parameter 'leaf'. rec_copy_prefix_to_dtuple(): Assert that the tuple is only built on the core fields. Instant ADD COLUMN only applies to the clustered index, and we should never build a search key that has more than the PRIMARY KEY and possibly DB_TRX_ID,DB_ROLL_PTR. All these columns are always present. dict_index_build_data_tuple(): Remove assertions that would be duplicated in rec_copy_prefix_to_dtuple(). rec_init_offsets(): Support ROW_FORMAT=REDUNDANT records whose number of fields is between n_core_fields and n_fields. cmp_rec_rec_with_match(): Implement the comparison between two MIN_REC_FLAG records. trx_t::in_rollback: Make the field available in non-debug builds. trx_start_for_ddl_low(): Remove dangerous error-tolerance. A dictionary transaction must be flagged as such before it has generated any undo log records. This is because trx_undo_assign_undo() will mark the transaction as a dictionary transaction in the undo log header right before the very first undo log record is being written. btr_index_rec_validate(): Account for instant ADD COLUMN row_undo_ins_remove_clust_rec(): On the rollback of an insert into SYS_COLUMNS, revert instant ADD COLUMN in the cache by removing the last column from the table and the clustered index. row_search_on_row_ref(), row_undo_mod_parse_undo_rec(), row_undo_mod(), trx_undo_update_rec_get_update(): Handle the 'default row' as a special case. dtuple_t::trim(index): Omit a redundant suffix of an index tuple right before insert or update. After instant ADD COLUMN, if the last fields of a clustered index tuple match the 'default row', there is no need to store them. While trimming the entry, we must hold a page latch, so that the table cannot be emptied and the 'default row' be deleted. btr_cur_optimistic_update(), btr_cur_pessimistic_update(), row_upd_clust_rec_by_insert(), row_ins_clust_index_entry_low(): Invoke dtuple_t::trim() if needed. row_ins_clust_index_entry(): Restore dtuple_t::n_fields after calling row_ins_clust_index_entry_low(). rec_get_converted_size(), rec_get_converted_size_comp(): Allow the number of fields to be between n_core_fields and n_fields. Do not support infimum,supremum. They are never supposed to be stored in dtuple_t, because page creation nowadays uses a lower-level method for initializing them. rec_convert_dtuple_to_rec_comp(): Assign the status bits based on the number of fields. btr_cur_trim(): In an update, trim the index entry as needed. For the 'default row', handle rollback specially. For user records, omit fields that match the 'default row'. btr_cur_optimistic_delete_func(), btr_cur_pessimistic_delete(): Skip locking and adaptive hash index for the 'default row'. row_log_table_apply_convert_mrec(): Replace 'default row' values if needed. In the temporary file that is applied by row_log_table_apply(), we must identify whether the records contain the extra header for instantly added columns. For now, we will allocate an additional byte for this for ROW_T_INSERT and ROW_T_UPDATE records when the source table has been subject to instant ADD COLUMN. The ROW_T_DELETE records are fine, as they will be converted and will only contain 'core' columns (PRIMARY KEY and some system columns) that are converted from dtuple_t. rec_get_converted_size_temp(), rec_init_offsets_temp(), rec_convert_dtuple_to_temp(): Add the parameter 'status'. REC_INFO_DEFAULT_ROW = REC_INFO_MIN_REC_FLAG \| REC_STATUS_COLUMNS_ADDED: An info_bits constant for distinguishing the 'default row' record. rec_comp_status_t: An enum of the status bit values. rec_leaf_format: An enum that replaces the bool parameter of rec_init_offsets_comp_ordinary().	8 years ago
Marko Mäkelä	770231f355	Remove dict_disable_redo_if_temporary() The function dict_disable_redo_if_temporary() was supposed to disable redo logging for temporary tables. It was invoked unnecessarily for two read-only operations: row_undo_search_clust_to_pcur() and dict_stats_update_transient_for_index(). When a table is not temporary and not in the system tablespace, the tablespace should be flagged for MLOG_FILE_NAME logging. We do not need this overhead for temporary tables. Therefore, either mtr_t::set_log_mode() or mtr_t::set_named_space() should be invoked. dict_table_t::is_temporary(): Determine if a table is temporary. dict_table_is_temporary(): Redefined as a macro wrapper for dict_table_t::is_temporary(). dict_disable_redo_if_temporary(): Remove.	8 years ago
Marko Mäkelä	cc3057fde7	Remove dict_table_t::big_rows The field dict_table_t::big_rows was only used for determining if the adaptive hash index should be used when the internal InnoDB SQL parser is used. That parser is only used for modifying the InnoDB data dictionary, updating persistent tables, and for fulltext indexes.	8 years ago

1 2 3 4 5 ...

377 Commits (4b80c11f52a3da189bafd7a772bcbf3519ceb41e)