From 0e5a4ac2532c64a545796c787354dc41d61d0e62 Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?Marko=20M=C3=A4kel=C3=A4?= Date: Fri, 19 Oct 2018 16:49:54 +0300 Subject: [PATCH] MDEV-15662 Instant DROP COLUMN or changing the order of columns Allow ADD COLUMN anywhere in a table, not only adding as the last column. Allow instant DROP COLUMN and instant changing the order of columns. The added columns will always be added last in clustered index records. In new records, instantly dropped columns will be stored as NULL or empty when possible. Information about dropped and reordered columns will be written in a metadata BLOB (mblob), which is stored before the first 'user' field in the hidden metadata record at the start of the clustered index. The presence of mblob is indicated by setting the delete-mark flag in the metadata record. The metadata BLOB stores the number of clustered index fields, followed by an array of column information for each field. For dropped columns, we store the NOT NULL flag, the fixed length, and for variable-length columns, whether the maximum length exceeded 255 bytes. For non-dropped columns, we store the column position. Unlike with MDEV-11369, when a table becomes empty, it cannot be converted back to the canonical format. The reason for this is that other threads may hold cached objects such as row_prebuilt_t::ins_node that could refer to dropped or reordered index fields. For instant DROP COLUMN and ROW_FORMAT=COMPACT or ROW_FORMAT=DYNAMIC, we must store the n_core_null_bytes in the root page, so that the chain of node pointer records can be followed in order to reach the leftmost leaf page where the metadata record is located. If the mblob is present, we will zero-initialize the strings "infimum" and "supremum" in the root page, and use the last byte of "supremum" for storing the number of null bytes (which are allocated but useless on node pointer pages). This is necessary for btr_cur_instant_init_metadata() to be able to navigate to the mblob. If the PRIMARY KEY contains any variable-length column and some nullable columns were instantly dropped, the dict_index_t::n_nullable in the data dictionary could be smaller than it actually is in the non-leaf pages. Because of this, the non-leaf pages could use more bytes for the null flags than the data dictionary expects, and we could be reading the lengths of the variable-length columns from the wrong offset, and thus reading the child page number from wrong place. This is the result of two design mistakes that involve unnecessary storage of data: First, it is nonsense to store any data fields for the leftmost node pointer records, because the comparisons would be resolved by the MIN_REC_FLAG alone. Second, there cannot be any null fields in the clustered index node pointer fields, but we nevertheless reserve space for all the null flags. Limitations (future work): MDEV-17459 Allow instant ALTER TABLE even if FULLTEXT INDEX exists MDEV-17468 Avoid table rebuild on operations on generated columns MDEV-17494 Refuse ALGORITHM=INSTANT when the row size is too large btr_page_reorganize_low(): Preserve any metadata in the root page. Call lock_move_reorganize_page() only after restoring the "infimum" and "supremum" records, to avoid a memcmp() assertion failure. dict_col_t::DROPPED: Magic value for dict_col_t::ind. dict_col_t::clear_instant(): Renamed from dict_col_t::remove_instant(). Do not assert that the column was instantly added, because we sometimes call this unconditionally for all columns. Convert an instantly added column to a "core column". The old name remove_instant() could be mistaken to refer to "instant DROP COLUMN". dict_col_t::is_added(): Rename from dict_col_t::is_instant(). dtype_t::metadata_blob_init(): Initialize the mblob data type. dtuple_t::is_metadata(), dtuple_t::is_alter_metadata(), upd_t::is_metadata(), upd_t::is_alter_metadata(): Check if info_bits refer to a metadata record. dict_table_t::instant: Metadata about dropped or reordered columns. dict_table_t::prepare_instant(): Prepare ha_innobase_inplace_ctx::instant_table for instant ALTER TABLE. innobase_instant_try() will pass this to dict_table_t::instant_column(). On rollback, dict_table_t::rollback_instant() will be called. dict_table_t::instant_column(): Renamed from instant_add_column(). Add the parameter col_map so that columns can be reordered. Copy and adjust v_cols[] as well. dict_table_t::find(): Find an old column based on a new column number. dict_table_t::serialise_columns(), dict_table_t::deserialise_columns(): Convert the mblob. dict_index_t::instant_metadata(): Create the metadata record for instant ALTER TABLE. Invoke dict_table_t::serialise_columns(). dict_index_t::reconstruct_fields(): Invoked by dict_table_t::deserialise_columns(). dict_index_t::clear_instant_alter(): Move the fields for the dropped columns to the end, and sort the surviving index fields in ascending order of column position. ha_innobase::check_if_supported_inplace_alter(): Do not allow adding a FTS_DOC_ID column if a hidden FTS_DOC_ID column exists due to FULLTEXT INDEX. (This always required ALGORITHM=COPY.) instant_alter_column_possible(): Add a parameter for InnoDB table, to check for additional conditions, such as the maximum number of index fields. ha_innobase_inplace_ctx::first_alter_pos: The first column whose position is affected by instant ADD, DROP, or changing the order of columns. innobase_build_col_map(): Skip added virtual columns. prepare_inplace_add_virtual(): Correctly compute num_to_add_vcol. Remove some unnecessary code. Note that the call to innodb_base_col_setup() should be executed later. commit_try_norebuild(): If ctx->is_instant(), let the virtual columns be added or dropped by innobase_instant_try(). innobase_instant_try(): Fill in a zero default value for the hidden column FTS_DOC_ID (to reduce the work needed in MDEV-17459). If any columns were dropped or reordered (or added not last), delete any SYS_COLUMNS records for the following columns, and insert SYS_COLUMNS records for all subsequent stored columns as well as for all virtual columns. If any virtual column is dropped, rewrite all virtual column metadata. Use a shortcut only for adding virtual columns. This is because innobase_drop_virtual_try() assumes that the dropped virtual columns still exist in ctx->old_table. innodb_update_cols(): Renamed from innodb_update_n_cols(). innobase_add_one_virtual(), innobase_insert_sys_virtual(): Change the return type to bool, and invoke my_error() when detecting an error. innodb_insert_sys_columns(): Insert a record into SYS_COLUMNS. Refactored from innobase_add_one_virtual() and innobase_instant_add_col(). innobase_instant_add_col(): Replace the parameter dfield with type. innobase_instant_drop_cols(): Drop matching columns from SYS_COLUMNS and all columns from SYS_VIRTUAL. innobase_add_virtual_try(), innobase_drop_virtual_try(): Let the caller invoke innodb_update_cols(). innobase_rename_column_try(): Skip dropped columns. commit_cache_norebuild(): Update table->fts->doc_col. dict_mem_table_col_rename_low(): Skip dropped columns. trx_undo_rec_get_partial_row(): Skip dropped columns. trx_undo_update_rec_get_update(): Handle the metadata BLOB correctly. trx_undo_page_report_modify(): Avoid out-of-bounds access to record fields. Log metadata records consistently. Apparently, the first fields of a clustered index may be updated in an update_undo vector when the index is ID_IND of SYS_FOREIGN, as part of renaming the table during ALTER TABLE. Normally, updates of the PRIMARY KEY should be logged as delete-mark and an insert. row_undo_mod_parse_undo_rec(), row_purge_parse_undo_rec(): Use trx_undo_metadata. row_undo_mod_clust_low(): On metadata rollback, roll back the root page too. row_undo_mod_clust(): Relax an assertion. The delete-mark flag was repurposed for ALTER TABLE metadata records. row_rec_to_index_entry_impl(): Add the template parameter mblob and the optional parameter info_bits for specifying the desired new info bits. For the metadata tuple, allow conversion between the original format (ADD COLUMN only) and the generic format (with hidden BLOB). Add the optional parameter "pad" to determine whether the tuple should be padded to the index fields (on ALTER TABLE it should), or whether it should remain at its original size (on rollback). row_build_index_entry_low(): Clean up the code, removing redundant variables and conditions. For instantly dropped columns, generate a dummy value that is NULL, the empty string, or a fixed length of NUL bytes, depending on the type of the dropped column. row_upd_clust_rec_by_insert_inherit_func(): On the update of PRIMARY KEY of a record that contained a dropped column whose value was stored externally, we will be inserting a dummy NULL or empty string value to the field of the dropped column. The externally stored column would eventually be dropped when purge removes the delete-marked record for the old PRIMARY KEY value. btr_index_rec_validate(): Recognize the metadata record. btr_discard_only_page_on_level(): Preserve the generic instant ALTER TABLE metadata. btr_set_instant(): Replaces page_set_instant(). This sets a clustered index root page to the appropriate format, or upgrades from the MDEV-11369 instant ADD COLUMN to generic ALTER TABLE format. btr_cur_instant_init_low(): Read and validate the metadata BLOB page before reconstructing the dictionary information based on it. btr_cur_instant_init_metadata(): Do not read any lengths from the metadata record header before reading the BLOB. At this point, we would not actually know how many nullable fields the metadata record contains. btr_cur_instant_root_init(): Initialize n_core_null_bytes in one of two possible ways. btr_cur_trim(): Handle the mblob record. row_metadata_to_tuple(): Convert a metadata record to a data tuple, based on the new info_bits of the metadata record. btr_cur_pessimistic_update(): Invoke row_metadata_to_tuple() if needed. Invoke dtuple_convert_big_rec() for metadata records if the record is too large, or if the mblob is not yet marked as externally stored. btr_cur_optimistic_delete_func(), btr_cur_pessimistic_delete(): When the last user record is deleted, do not delete the generic instant ALTER TABLE metadata record. Only delete MDEV-11369 instant ADD COLUMN metadata records. btr_cur_optimistic_insert(): Avoid unnecessary computation of rec_size. btr_pcur_store_position(): Allow a logically empty page to contain a metadata record for generic ALTER TABLE. REC_INFO_DEFAULT_ROW_ADD: Renamed from REC_INFO_DEFAULT_ROW. This is for the old instant ADD COLUMN (MDEV-11369) only. REC_INFO_DEFAULT_ROW_ALTER: The more generic metadata record, with additional information for dropped or reordered columns. rec_info_bits_valid(): Remove. The only case when this would fail is when the record is the generic ALTER TABLE metadata record. rec_is_alter_metadata(): Check if a record is the metadata record for instant ALTER TABLE (other than ADD COLUMN). NOTE: This function must not be invoked on node pointer records, because the delete-mark flag in those records may be set (it is garbage), and then a debug assertion could fail because index->is_instant() does not necessarily hold. rec_is_add_metadata(): Check if a record is MDEV-11369 ADD COLUMN metadata record (not more generic instant ALTER TABLE). rec_get_converted_size_comp_prefix_low(): Assume that the metadata field will be stored externally. In dtuple_convert_big_rec() during the rec_get_converted_size() call, it would not be there yet. rec_get_converted_size_comp(): Replace status,fields,n_fields with tuple. rec_init_offsets_comp_ordinary(), rec_get_converted_size_comp_prefix_low(), rec_convert_dtuple_to_rec_comp(): Add template. With mblob=true, process a record with a metadata BLOB. rec_copy_prefix_to_buf(): Assert that no fields beyond the key and system columns are being copied. Exclude the metadata BLOB field. rec_convert_dtuple_to_metadata_comp(): Convert an alter metadata tuple into a record. row_upd_index_replace_metadata(): Apply an update vector to an alter_metadata tuple. row_log_allocate(): Replace dict_index_t::is_instant() with a more appropriate condition that ignores dict_table_t::instant. Only a table on which the MDEV-11369 ADD COLUMN was performed can "lose its instantness" when it becomes empty. After instant DROP COLUMN or reordering columns, we cannot simply convert the table to the canonical format, because the data dictionary cache and all possibly existing references to it from other client connection threads would have to be adjusted. row_quiesce_write_index_fields(): Do not crash when the table contains an instantly dropped column. Thanks to Thirunarayanan Balathandayuthapani for discussing the design and implementing an initial prototype of this. Thanks to Matthias Leich for testing. --- .../gcol/r/innodb_virtual_rebuild.result | 16 +- .../suite/gcol/t/innodb_virtual_rebuild.test | 5 +- mysql-test/suite/innodb/r/innodb-alter.result | 2 + .../suite/innodb/r/instant_alter.result | 125 +- .../suite/innodb/r/instant_alter_crash.result | 62 +- .../suite/innodb/r/instant_alter_debug.result | 26 +- .../suite/innodb/r/instant_alter_limit.result | 35 + .../innodb/r/instant_alter_rollback.result | 27 +- mysql-test/suite/innodb/r/instant_drop.result | 190 ++ mysql-test/suite/innodb/t/innodb-alter.test | 4 +- mysql-test/suite/innodb/t/instant_alter.test | 36 + .../suite/innodb/t/instant_alter_crash.test | 70 +- .../suite/innodb/t/instant_alter_debug.test | 31 +- .../suite/innodb/t/instant_alter_limit.test | 42 + .../innodb/t/instant_alter_rollback.test | 24 +- mysql-test/suite/innodb/t/instant_drop.test | 98 + .../innodb_gis/r/alter_spatial_index.result | 2 +- .../innodb_gis/t/alter_spatial_index.test | 3 +- mysql-test/suite/versioning/r/online.result | 3 +- mysql-test/suite/versioning/t/online.test | 3 +- storage/innobase/btr/btr0btr.cc | 197 +- storage/innobase/btr/btr0cur.cc | 332 ++- storage/innobase/btr/btr0pcur.cc | 11 +- storage/innobase/btr/btr0sea.cc | 6 +- storage/innobase/data/data0data.cc | 31 +- storage/innobase/dict/dict0mem.cc | 308 ++- storage/innobase/fts/fts0fts.cc | 2 +- storage/innobase/handler/ha_innodb.cc | 8 +- storage/innobase/handler/handler0alter.cc | 1797 ++++++++++++----- storage/innobase/include/btr0btr.h | 6 + storage/innobase/include/data0data.h | 29 +- storage/innobase/include/data0type.h | 44 + storage/innobase/include/dict0dict.ic | 3 +- storage/innobase/include/dict0mem.h | 260 ++- storage/innobase/include/page0cur.ic | 1 + storage/innobase/include/page0page.h | 7 - storage/innobase/include/page0page.ic | 18 - storage/innobase/include/page0size.h | 2 +- storage/innobase/include/rem0rec.h | 153 +- storage/innobase/include/rem0rec.ic | 39 +- storage/innobase/include/row0row.h | 21 + storage/innobase/include/row0upd.h | 9 +- storage/innobase/lock/lock0lock.cc | 24 +- storage/innobase/page/page0cur.cc | 6 +- storage/innobase/page/page0page.cc | 2 + storage/innobase/page/page0zip.cc | 15 +- storage/innobase/rem/rem0rec.cc | 510 +++-- storage/innobase/row/row0import.cc | 4 +- storage/innobase/row/row0ins.cc | 46 +- storage/innobase/row/row0log.cc | 21 +- storage/innobase/row/row0merge.cc | 2 +- storage/innobase/row/row0mysql.cc | 4 +- storage/innobase/row/row0purge.cc | 8 +- storage/innobase/row/row0quiesce.cc | 7 +- storage/innobase/row/row0row.cc | 388 +++- storage/innobase/row/row0sel.cc | 10 +- storage/innobase/row/row0umod.cc | 95 +- storage/innobase/row/row0undo.cc | 6 +- storage/innobase/row/row0upd.cc | 78 +- storage/innobase/trx/trx0rec.cc | 184 +- 60 files changed, 4026 insertions(+), 1472 deletions(-) create mode 100644 mysql-test/suite/innodb/r/instant_alter_limit.result create mode 100644 mysql-test/suite/innodb/r/instant_drop.result create mode 100644 mysql-test/suite/innodb/t/instant_alter_limit.test create mode 100644 mysql-test/suite/innodb/t/instant_drop.test diff --git a/mysql-test/suite/gcol/r/innodb_virtual_rebuild.result b/mysql-test/suite/gcol/r/innodb_virtual_rebuild.result index 2e35698e47e..35f37034721 100644 --- a/mysql-test/suite/gcol/r/innodb_virtual_rebuild.result +++ b/mysql-test/suite/gcol/r/innodb_virtual_rebuild.result @@ -7,31 +7,31 @@ ROW_FORMAT=REDUNDANT; INSERT INTO t4 SET i=1; ALTER TABLE t4 ADD INDEX(v), LOCK=NONE; ALTER TABLE t4 ADD COLUMN k INT, LOCK=NONE; -ALTER TABLE t4 DROP k, LOCK=NONE; ERROR 0A000: LOCK=NONE is not supported. Reason: online rebuild with indexed virtual columns. Try LOCK=SHARED -ALTER TABLE t4 DROP INDEX v, LOCK=NONE; ALTER TABLE t4 DROP k, LOCK=NONE; +ERROR 42000: Can't DROP COLUMN `k`; check that it exists +ALTER TABLE t4 DROP INDEX v, LOCK=NONE; INSERT INTO t3 SET i=1; ALTER TABLE t3 ADD INDEX(v), LOCK=NONE; ALTER TABLE t3 ADD COLUMN k INT, LOCK=NONE; -ALTER TABLE t3 DROP k, LOCK=NONE; ERROR 0A000: LOCK=NONE is not supported. Reason: online rebuild with indexed virtual columns. Try LOCK=SHARED -ALTER TABLE t3 DROP INDEX v, LOCK=NONE; ALTER TABLE t3 DROP k, LOCK=NONE; +ERROR 42000: Can't DROP COLUMN `k`; check that it exists +ALTER TABLE t3 DROP INDEX v, LOCK=NONE; INSERT INTO t2 SET i=1; ALTER TABLE t2 ADD INDEX(v), LOCK=NONE; ALTER TABLE t2 ADD COLUMN k INT, LOCK=NONE; -ALTER TABLE t2 DROP k, LOCK=NONE; ERROR 0A000: LOCK=NONE is not supported. Reason: online rebuild with indexed virtual columns. Try LOCK=SHARED -ALTER TABLE t2 DROP INDEX v, LOCK=NONE; ALTER TABLE t2 DROP k, LOCK=NONE; +ERROR 42000: Can't DROP COLUMN `k`; check that it exists +ALTER TABLE t2 DROP INDEX v, LOCK=NONE; INSERT INTO t1 SET i=1; ALTER TABLE t1 ADD INDEX(v), LOCK=NONE; ALTER TABLE t1 ADD COLUMN k INT, LOCK=NONE; -ALTER TABLE t1 DROP k, LOCK=NONE; ERROR 0A000: LOCK=NONE is not supported. Reason: online rebuild with indexed virtual columns. Try LOCK=SHARED -ALTER TABLE t1 DROP INDEX v, LOCK=NONE; ALTER TABLE t1 DROP k, LOCK=NONE; +ERROR 42000: Can't DROP COLUMN `k`; check that it exists +ALTER TABLE t1 DROP INDEX v, LOCK=NONE; connect ddl,localhost,root,,test; connection default; connection ddl; diff --git a/mysql-test/suite/gcol/t/innodb_virtual_rebuild.test b/mysql-test/suite/gcol/t/innodb_virtual_rebuild.test index 37ab82c46db..fe4f5e307b3 100644 --- a/mysql-test/suite/gcol/t/innodb_virtual_rebuild.test +++ b/mysql-test/suite/gcol/t/innodb_virtual_rebuild.test @@ -14,11 +14,12 @@ while ($n) { eval INSERT INTO t$n SET i=1; eval ALTER TABLE t$n ADD INDEX(v), LOCK=NONE; -eval ALTER TABLE t$n ADD COLUMN k INT, LOCK=NONE; +# MDEV-17468 FIXME: Fix this, and remove the 2 --error below. --error ER_ALTER_OPERATION_NOT_SUPPORTED_REASON +eval ALTER TABLE t$n ADD COLUMN k INT, LOCK=NONE; +--error ER_CANT_DROP_FIELD_OR_KEY eval ALTER TABLE t$n DROP k, LOCK=NONE; eval ALTER TABLE t$n DROP INDEX v, LOCK=NONE; -eval ALTER TABLE t$n DROP k, LOCK=NONE; dec $n; } diff --git a/mysql-test/suite/innodb/r/innodb-alter.result b/mysql-test/suite/innodb/r/innodb-alter.result index afdeac1c22a..8f8a30d832d 100644 --- a/mysql-test/suite/innodb/r/innodb-alter.result +++ b/mysql-test/suite/innodb/r/innodb-alter.result @@ -449,6 +449,7 @@ FULLTEXT INDEX(t) ALTER TABLE tu ADD COLUMN c CHAR(1) NOT NULL FIRST, LOCK=NONE; ERROR 0A000: LOCK=NONE is not supported. Reason: InnoDB presently supports one FULLTEXT index creation at a time. Try LOCK=SHARED ALTER TABLE tu ADD COLUMN c CHAR(1) NOT NULL, LOCK=NONE; +ERROR 0A000: LOCK=NONE is not supported. Reason: InnoDB presently supports one FULLTEXT index creation at a time. Try LOCK=SHARED DROP TABLE tu; CREATE TABLE tv ( pk INT PRIMARY KEY, FTS_DOC_ID BIGINT UNSIGNED NOT NULL, t TEXT, @@ -458,6 +459,7 @@ FULLTEXT INDEX(t) ALTER TABLE tv ADD COLUMN c CHAR(1) NOT NULL FIRST, LOCK=NONE; ERROR 0A000: LOCK=NONE is not supported. Reason: InnoDB presently supports one FULLTEXT index creation at a time. Try LOCK=SHARED ALTER TABLE tv ADD COLUMN c CHAR(1) NOT NULL, LOCK=NONE; +ERROR 0A000: LOCK=NONE is not supported. Reason: InnoDB presently supports one FULLTEXT index creation at a time. Try LOCK=SHARED DROP TABLE tv; ALTER TABLE t1o CHANGE c1 dB_row_Id INT, ALGORITHM=COPY; ERROR 42000: Incorrect column name 'dB_row_Id' diff --git a/mysql-test/suite/innodb/r/instant_alter.result b/mysql-test/suite/innodb/r/instant_alter.result index 3ebc161caf1..5f89b31c142 100644 --- a/mysql-test/suite/innodb/r/instant_alter.result +++ b/mysql-test/suite/innodb/r/instant_alter.result @@ -182,8 +182,8 @@ affected rows: 0 info: Records: 0 Duplicates: 0 Warnings: 0 INSERT INTO t1 SET id=9; ALTER TABLE t1 DROP c3; -affected rows: 9 -info: Records: 9 Duplicates: 0 Warnings: 0 +affected rows: 0 +info: Records: 0 Duplicates: 0 Warnings: 0 SHOW CREATE TABLE t1; Table Create Table t1 CREATE TABLE `t1` ( @@ -486,6 +486,43 @@ DELETE FROM t1; COMMIT; InnoDB 0 transactions not purged DROP TABLE t1; +CREATE TABLE t1 (a INT, b INT UNIQUE) ENGINE=InnoDB ROW_FORMAT=REDUNDANT; +INSERT INTO t1 (a) VALUES (NULL), (NULL); +ALTER TABLE t1 DROP a, ADD COLUMN a INT; +DELETE FROM t1; +BEGIN; +INSERT INTO t1 SET a=NULL; +ROLLBACK; +DELETE FROM t1; +DROP TABLE t1; +CREATE TABLE t1 (a INT PRIMARY KEY, t VARCHAR(33101) NOT NULL) ENGINE=InnoDB ROW_FORMAT=REDUNDANT; +INSERT INTO t1 VALUES(347,''); +ALTER TABLE t1 DROP COLUMN t, ALGORITHM=INSTANT; +SELECT * FROM t1; +a +347 +DROP TABLE t1; +CREATE TABLE t1 (a INT) ENGINE=InnoDB ROW_FORMAT=REDUNDANT; +INSERT INTO t1() VALUES(); +ALTER TABLE t1 ADD COLUMN b INT FIRST, ADD COLUMN c INT AFTER b; +SELECT * FROM t1; +b c a +NULL NULL NULL +DROP TABLE t1; +CREATE TABLE t1 (t TEXT) ENGINE=InnoDB ROW_FORMAT=REDUNDANT; +SET @t=REPEAT('x',@@innodb_page_size / 2); +INSERT INTO t1 VALUES (@t),(@t),(@t),(@t),(@t),(@t),(NULL),(@t),(@t),(@t),(@t); +ALTER TABLE t1 ADD COLUMN a INT FIRST; +UPDATE t1 SET a = 0; +DROP TABLE t1; +CREATE TABLE t1 (t TEXT) ENGINE=InnoDB ROW_FORMAT=REDUNDANT; +INSERT INTO t1 SET t = @x; +ALTER TABLE t1 DROP COLUMN t, ADD COLUMN i INT NOT NULL DEFAULT 1; +ALTER TABLE t1 ADD COLUMN t TEXT; +SELECT * FROM t1; +i t +1 NULL +DROP TABLE t1; CREATE TABLE t1 (id INT PRIMARY KEY, c2 INT UNIQUE, c3 POINT NOT NULL DEFAULT ST_GeomFromText('POINT(3 4)'), @@ -614,8 +651,8 @@ affected rows: 0 info: Records: 0 Duplicates: 0 Warnings: 0 INSERT INTO t1 SET id=9; ALTER TABLE t1 DROP c3; -affected rows: 9 -info: Records: 9 Duplicates: 0 Warnings: 0 +affected rows: 0 +info: Records: 0 Duplicates: 0 Warnings: 0 SHOW CREATE TABLE t1; Table Create Table t1 CREATE TABLE `t1` ( @@ -918,6 +955,43 @@ DELETE FROM t1; COMMIT; InnoDB 0 transactions not purged DROP TABLE t1; +CREATE TABLE t1 (a INT, b INT UNIQUE) ENGINE=InnoDB ROW_FORMAT=COMPACT; +INSERT INTO t1 (a) VALUES (NULL), (NULL); +ALTER TABLE t1 DROP a, ADD COLUMN a INT; +DELETE FROM t1; +BEGIN; +INSERT INTO t1 SET a=NULL; +ROLLBACK; +DELETE FROM t1; +DROP TABLE t1; +CREATE TABLE t1 (a INT PRIMARY KEY, t VARCHAR(33101) NOT NULL) ENGINE=InnoDB ROW_FORMAT=COMPACT; +INSERT INTO t1 VALUES(347,''); +ALTER TABLE t1 DROP COLUMN t, ALGORITHM=INSTANT; +SELECT * FROM t1; +a +347 +DROP TABLE t1; +CREATE TABLE t1 (a INT) ENGINE=InnoDB ROW_FORMAT=COMPACT; +INSERT INTO t1() VALUES(); +ALTER TABLE t1 ADD COLUMN b INT FIRST, ADD COLUMN c INT AFTER b; +SELECT * FROM t1; +b c a +NULL NULL NULL +DROP TABLE t1; +CREATE TABLE t1 (t TEXT) ENGINE=InnoDB ROW_FORMAT=COMPACT; +SET @t=REPEAT('x',@@innodb_page_size / 2); +INSERT INTO t1 VALUES (@t),(@t),(@t),(@t),(@t),(@t),(NULL),(@t),(@t),(@t),(@t); +ALTER TABLE t1 ADD COLUMN a INT FIRST; +UPDATE t1 SET a = 0; +DROP TABLE t1; +CREATE TABLE t1 (t TEXT) ENGINE=InnoDB ROW_FORMAT=COMPACT; +INSERT INTO t1 SET t = @x; +ALTER TABLE t1 DROP COLUMN t, ADD COLUMN i INT NOT NULL DEFAULT 1; +ALTER TABLE t1 ADD COLUMN t TEXT; +SELECT * FROM t1; +i t +1 NULL +DROP TABLE t1; CREATE TABLE t1 (id INT PRIMARY KEY, c2 INT UNIQUE, c3 POINT NOT NULL DEFAULT ST_GeomFromText('POINT(3 4)'), @@ -1046,8 +1120,8 @@ affected rows: 0 info: Records: 0 Duplicates: 0 Warnings: 0 INSERT INTO t1 SET id=9; ALTER TABLE t1 DROP c3; -affected rows: 9 -info: Records: 9 Duplicates: 0 Warnings: 0 +affected rows: 0 +info: Records: 0 Duplicates: 0 Warnings: 0 SHOW CREATE TABLE t1; Table Create Table t1 CREATE TABLE `t1` ( @@ -1350,10 +1424,47 @@ DELETE FROM t1; COMMIT; InnoDB 0 transactions not purged DROP TABLE t1; +CREATE TABLE t1 (a INT, b INT UNIQUE) ENGINE=InnoDB ROW_FORMAT=DYNAMIC; +INSERT INTO t1 (a) VALUES (NULL), (NULL); +ALTER TABLE t1 DROP a, ADD COLUMN a INT; +DELETE FROM t1; +BEGIN; +INSERT INTO t1 SET a=NULL; +ROLLBACK; +DELETE FROM t1; +DROP TABLE t1; +CREATE TABLE t1 (a INT PRIMARY KEY, t VARCHAR(33101) NOT NULL) ENGINE=InnoDB ROW_FORMAT=DYNAMIC; +INSERT INTO t1 VALUES(347,''); +ALTER TABLE t1 DROP COLUMN t, ALGORITHM=INSTANT; +SELECT * FROM t1; +a +347 +DROP TABLE t1; +CREATE TABLE t1 (a INT) ENGINE=InnoDB ROW_FORMAT=DYNAMIC; +INSERT INTO t1() VALUES(); +ALTER TABLE t1 ADD COLUMN b INT FIRST, ADD COLUMN c INT AFTER b; +SELECT * FROM t1; +b c a +NULL NULL NULL +DROP TABLE t1; +CREATE TABLE t1 (t TEXT) ENGINE=InnoDB ROW_FORMAT=DYNAMIC; +SET @t=REPEAT('x',@@innodb_page_size / 2); +INSERT INTO t1 VALUES (@t),(@t),(@t),(@t),(@t),(@t),(NULL),(@t),(@t),(@t),(@t); +ALTER TABLE t1 ADD COLUMN a INT FIRST; +UPDATE t1 SET a = 0; +DROP TABLE t1; +CREATE TABLE t1 (t TEXT) ENGINE=InnoDB ROW_FORMAT=DYNAMIC; +INSERT INTO t1 SET t = @x; +ALTER TABLE t1 DROP COLUMN t, ADD COLUMN i INT NOT NULL DEFAULT 1; +ALTER TABLE t1 ADD COLUMN t TEXT; +SELECT * FROM t1; +i t +1 NULL +DROP TABLE t1; disconnect analyze; SELECT variable_value-@old_instant instants FROM information_schema.global_status WHERE variable_name = 'innodb_instant_alter_column'; instants -51 +78 SET GLOBAL innodb_purge_rseg_truncate_frequency= @saved_frequency; diff --git a/mysql-test/suite/innodb/r/instant_alter_crash.result b/mysql-test/suite/innodb/r/instant_alter_crash.result index d9e57b397f3..807a4091822 100644 --- a/mysql-test/suite/innodb/r/instant_alter_crash.result +++ b/mysql-test/suite/innodb/r/instant_alter_crash.result @@ -29,10 +29,11 @@ BEGIN; DELETE FROM t1; ROLLBACK; InnoDB 0 transactions not purged -INSERT INTO t2 VALUES (64,42,'De finibus bonorum'), (347,33101,' et malorum'); +INSERT INTO t2 VALUES +(16,1551,'Omnium enim rerum'),(128,1571,' principia parva sunt'); connect ddl, localhost, root; SET DEBUG_SYNC='innodb_alter_inplace_before_commit SIGNAL ddl WAIT_FOR ever'; -ALTER TABLE t2 ADD COLUMN (c4 TEXT NOT NULL DEFAULT ' et malorum'); +ALTER TABLE t2 DROP COLUMN c3, ADD COLUMN c5 TEXT DEFAULT 'naturam abhorrere'; connection default; SET DEBUG_SYNC='now WAIT_FOR ddl'; SET GLOBAL innodb_flush_log_at_trx_commit=1; @@ -46,8 +47,35 @@ id c2 SELECT * FROM t2; id c2 c3 2 1 De finibus bonorum -64 42 De finibus bonorum -347 33101 et malorum +16 1551 Omnium enim rerum +128 1571 principia parva sunt +BEGIN; +INSERT INTO t1 SET id=1; +DELETE FROM t2; +ROLLBACK; +InnoDB 0 transactions not purged +INSERT INTO t2 VALUES (64,42,'De finibus bonorum'), (347,33101,' et malorum'); +connect ddl, localhost, root; +ALTER TABLE t2 DROP COLUMN c3; +SET DEBUG_SYNC='innodb_alter_inplace_before_commit SIGNAL ddl WAIT_FOR ever'; +ALTER TABLE t2 ADD COLUMN (c4 TEXT NOT NULL DEFAULT ' et malorum'); +connection default; +SET DEBUG_SYNC='now WAIT_FOR ddl'; +SET GLOBAL innodb_flush_log_at_trx_commit=1; +DELETE FROM t1; +# Kill the server +disconnect ddl; +SET @saved_frequency= @@GLOBAL.innodb_purge_rseg_truncate_frequency; +SET GLOBAL innodb_purge_rseg_truncate_frequency=1; +SELECT * FROM t1; +id c2 +SELECT * FROM t2; +id c2 +2 1 +64 42 +16 1551 +128 1571 +347 33101 BEGIN; INSERT INTO t1 SET id=1; DELETE FROM t2; @@ -59,27 +87,38 @@ N_RECS=0; LEVEL=0 header=0x010000030074 (id=0x696e66696d756d00) header=0x010008030000 (id=0x73757072656d756d00) t2 clustered index root page(type 18): -N_RECS=4; LEVEL=0 -header=0x010000030088 (id=0x696e66696d756d00) -header=0x1000100b00b9 (id=0x80000000, +N_RECS=6; LEVEL=0 +header=0x01000003008f (id=0x0000000000000000) +header=0x3000100c00d4 (id=0x80000000, DB_TRX_ID=0x000000000000, DB_ROLL_PTR=0x80000000000000, + BLOB=0x000000260000000000000008, c2=NULL(4 bytes), c3=0x44652066696e6962757320626f6e6f72756d) -header=0x0000180900d8 (id=0x80000002, +header=0x0000180900f4 (id=0x80000002, DB_TRX_ID=0x000000000000, DB_ROLL_PTR=0x80000000000000, c2=0x80000001) -header=0x0000200900f8 (id=0x80000040, +header=0x0000200b0124 (id=0x80000010, + DB_TRX_ID=0x000000000000, + DB_ROLL_PTR=0x80000000000000, + c2=0x8000060f, + c3=0x4f6d6e69756d20656e696d20726572756d) +header=0x000028090144 (id=0x80000040, DB_TRX_ID=0x000000000000, DB_ROLL_PTR=0x80000000000000, c2=0x8000002a) -header=0x0000280b0074 (id=0x8000015b, +header=0x0000300b0179 (id=0x80000080, + DB_TRX_ID=0x000000000000, + DB_ROLL_PTR=0x80000000000000, + c2=0x80000623, + c3=0x207072696e63697069612070617276612073756e74) +header=0x0000380b0074 (id=0x8000015b, DB_TRX_ID=0x000000000000, DB_ROLL_PTR=0x80000000000000, c2=0x8000814d, c3=0x206574206d616c6f72756d) -header=0x050008030000 (id=0x73757072656d756d00) +header=0x070008030000 (id=0x000000000000000100) UNLOCK TABLES; DELETE FROM t2; InnoDB 0 transactions not purged @@ -96,7 +135,6 @@ Table Create Table t2 CREATE TABLE `t2` ( `id` int(11) NOT NULL, `c2` int(11) DEFAULT NULL, - `c3` text NOT NULL DEFAULT 'De finibus bonorum', PRIMARY KEY (`id`), UNIQUE KEY `c2` (`c2`) ) ENGINE=InnoDB DEFAULT CHARSET=latin1 ROW_FORMAT=REDUNDANT diff --git a/mysql-test/suite/innodb/r/instant_alter_debug.result b/mysql-test/suite/innodb/r/instant_alter_debug.result index 72e42763569..c017a466516 100644 --- a/mysql-test/suite/innodb/r/instant_alter_debug.result +++ b/mysql-test/suite/innodb/r/instant_alter_debug.result @@ -197,7 +197,8 @@ DELETE FROM t1; connection ddl; SET DEBUG_SYNC='row_log_table_apply1_before SIGNAL copied WAIT_FOR logged'; ALTER TABLE t1 FORCE; -disconnect stop_purge; +connection stop_purge; +COMMIT; connection default; SET DEBUG_SYNC = 'now WAIT_FOR copied'; InnoDB 1 transactions not purged @@ -211,6 +212,29 @@ SELECT * FROM t1; a b c 1 2 NULL 2 3 4 +ALTER TABLE t1 DROP b, ALGORITHM=INSTANT; +connection stop_purge; +START TRANSACTION WITH CONSISTENT SNAPSHOT; +connection default; +DELETE FROM t1; +connection ddl; +SET DEBUG_SYNC='row_log_table_apply1_before SIGNAL copied WAIT_FOR logged'; +ALTER TABLE t1 ADD COLUMN b INT NOT NULL DEFAULT 2 AFTER a, FORCE; +disconnect stop_purge; +connection default; +SET DEBUG_SYNC = 'now WAIT_FOR copied'; +InnoDB 1 transactions not purged +INSERT INTO t1 SET a=1; +INSERT INTO t1 SET a=2,c=4; +SET DEBUG_SYNC = 'now SIGNAL logged'; +connection ddl; +UPDATE t1 SET b = b + 1 WHERE a = 2; +connection default; +SET DEBUG_SYNC = RESET; +SELECT * FROM t1; +a b c +1 2 NULL +2 3 4 # # MDEV-15872 Crash in online ALTER TABLE...ADD PRIMARY KEY # after instant ADD COLUMN ... NULL diff --git a/mysql-test/suite/innodb/r/instant_alter_limit.result b/mysql-test/suite/innodb/r/instant_alter_limit.result new file mode 100644 index 00000000000..e66579e4a94 --- /dev/null +++ b/mysql-test/suite/innodb/r/instant_alter_limit.result @@ -0,0 +1,35 @@ +SET @old_instant= +(SELECT variable_value FROM information_schema.global_status +WHERE variable_name = 'innodb_instant_alter_column'); +CREATE TABLE t(a INT PRIMARY KEY, b INT, c INT, d INT, e INT) +ENGINE=InnoDB; +INSERT INTO t VALUES(1,2,3,4,5); +SELECT * FROM t; +b d a c e +NULL NULL 1 NULL NULL +ALTER TABLE t DROP b, DROP c, DROP d, DROP e, +ADD COLUMN b INT, ALGORITHM=INSTANT; +ERROR 0A000: ALGORITHM=INSTANT is not supported for this operation. Try ALGORITHM=INPLACE +ALTER TABLE t CHANGE COLUMN b beta INT AFTER a, ALGORITHM=INSTANT; +ALTER TABLE t DROP e, DROP c, DROP d, ALGORITHM=INSTANT; +SELECT * FROM t; +a beta +1 NULL +ALTER TABLE t DROP COLUMN beta, ALGORITHM=INSTANT; +ALTER TABLE t ADD COLUMN b INT NOT NULL, ALGORITHM=INSTANT; +ERROR 0A000: ALGORITHM=INSTANT is not supported for this operation. Try ALGORITHM=INPLACE +SELECT variable_value-@old_instant instants +FROM information_schema.global_status +WHERE variable_name = 'innodb_instant_alter_column'; +instants +256 +ALTER TABLE t ADD COLUMN b INT NOT NULL; +SELECT variable_value-@old_instant instants +FROM information_schema.global_status +WHERE variable_name = 'innodb_instant_alter_column'; +instants +256 +SELECT * FROM t; +a b +1 0 +DROP TABLE t; diff --git a/mysql-test/suite/innodb/r/instant_alter_rollback.result b/mysql-test/suite/innodb/r/instant_alter_rollback.result index 2c2083adc46..ecf63a20b4a 100644 --- a/mysql-test/suite/innodb/r/instant_alter_rollback.result +++ b/mysql-test/suite/innodb/r/instant_alter_rollback.result @@ -1,24 +1,46 @@ FLUSH TABLES; # # MDEV-11369: Instant ADD COLUMN for InnoDB +# MDEV-15562: Instant DROP COLUMN or changing the order of columns # connect to_be_killed, localhost, root; +SET @old_instant= +(SELECT variable_value FROM information_schema.global_status +WHERE variable_name = 'innodb_instant_alter_column'); CREATE TABLE empty (id INT PRIMARY KEY, c2 INT UNIQUE) ENGINE=InnoDB; CREATE TABLE once LIKE empty; CREATE TABLE twice LIKE empty; +CREATE TABLE thrice LIKE empty; INSERT INTO once SET id=1,c2=1; INSERT INTO twice SET id=1,c2=1; +INSERT INTO thrice SET id=1,c2=1; ALTER TABLE empty ADD COLUMN (d1 INT DEFAULT 15); ALTER TABLE once ADD COLUMN (d1 INT DEFAULT 20); ALTER TABLE twice ADD COLUMN (d1 INT DEFAULT 20); +ALTER TABLE thrice ADD COLUMN (d1 INT DEFAULT 20); ALTER TABLE twice ADD COLUMN (d2 INT NOT NULL DEFAULT 10, d3 VARCHAR(15) NOT NULL DEFAULT 'var och en char'); +ALTER TABLE thrice ADD COLUMN +(d2 INT NOT NULL DEFAULT 10, +d3 TEXT NOT NULL DEFAULT 'con'); +ALTER TABLE thrice DROP c2, DROP d3, CHANGE d2 d3 INT NOT NULL FIRST; +SELECT variable_value-@old_instant instants +FROM information_schema.global_status +WHERE variable_name = 'innodb_instant_alter_column'; +instants +7 BEGIN; INSERT INTO empty set id=0,c2=42; UPDATE once set c2=c2+1; UPDATE twice set c2=c2+1; +UPDATE thrice set d3=d3+1; INSERT INTO twice SET id=2,c2=0,d3=''; +INSERT INTO thrice SET id=2,d3=0; +DELETE FROM empty; +DELETE FROM once; +DELETE FROM twice; +DELETE FROM thrice; connection default; SET GLOBAL innodb_flush_log_at_trx_commit=1; CREATE TABLE foo(a INT PRIMARY KEY) ENGINE=InnoDB; @@ -37,4 +59,7 @@ id c2 d1 SELECT * FROM twice; id c2 d1 d2 d3 1 1 20 10 var och en char -DROP TABLE empty, once, twice; +SELECT * FROM thrice; +d3 id d1 +10 1 20 +DROP TABLE empty, once, twice, thrice; diff --git a/mysql-test/suite/innodb/r/instant_drop.result b/mysql-test/suite/innodb/r/instant_drop.result new file mode 100644 index 00000000000..3f029c3f7d0 --- /dev/null +++ b/mysql-test/suite/innodb/r/instant_drop.result @@ -0,0 +1,190 @@ +create table t1(f1 int not null, f2 int not null, f3 int not null)engine=innodb; +insert into t1 values(1, 2, 3),(4, 5, 6); +alter table t1 drop column f2, algorithm=instant; +select * from t1; +f1 f3 +1 3 +4 6 +insert into t1 values(1,2); +select * from t1; +f1 f3 +1 3 +4 6 +1 2 +alter table t1 add column f4 int not null default 5, algorithm=instant; +select * from t1; +f1 f3 f4 +1 3 5 +4 6 5 +1 2 5 +alter table t1 drop column f1, algorithm=instant; +select * from t1; +f3 f4 +3 5 +6 5 +2 5 +insert into t1 values(7, 9); +select * from t1; +f3 f4 +3 5 +6 5 +2 5 +7 9 +alter table t1 add column f5 blob default repeat('aaa', 950), drop column f4, algorithm=instant; +select * from t1; +f3 f5 +3 aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa +6 aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa +2 aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa +7 aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa +select f3 from t1; +f3 +3 +6 +2 +7 +update t1 set f3 = 10 where f3 > 2; +select * from t1; +f3 f5 +10 aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa +10 aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa +2 aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa +10 aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa +delete from t1 where f3 = 10; +show create table t1; +Table Create Table +t1 CREATE TABLE `t1` ( + `f3` int(11) NOT NULL, + `f5` blob DEFAULT repeat('aaa',950) +) ENGINE=InnoDB DEFAULT CHARSET=latin1 +select f3 from t1; +f3 +2 +update t1 set f5 = 'world'; +select * from t1; +f3 f5 +2 world +drop table t1; +create table t1(f1 int, f2 int not null, index idx(f2))engine=innodb; +insert into t1 values(1, 2); +alter table t1 drop column f1, add column f3 varchar(100) default 'thiru', algorithm=instant; +select * from t1 force index (idx); +f2 f3 +2 thiru +alter table t1 drop column f3, algorithm=instant; +select * from t1; +f2 +2 +begin; +insert into t1 values(10); +select * from t1; +f2 +2 +10 +update t1 set f2 = 100; +select * from t1; +f2 +100 +100 +delete from t1 where f2 = 100; +select * from t1; +f2 +rollback; +select * from t1; +f2 +2 +show create table t1; +Table Create Table +t1 CREATE TABLE `t1` ( + `f2` int(11) NOT NULL, + KEY `idx` (`f2`) +) ENGINE=InnoDB DEFAULT CHARSET=latin1 +drop table t1; +create table t1(f1 int, f2 int not null)engine=innodb; +insert into t1 values(1, 2); +alter table t1 drop column f2, algorithm=instant; +insert into t1 values(NULL); +select * from t1; +f1 +1 +NULL +drop table t1; +create table t1(f1 int not null, f2 int not null)engine=innodb; +insert into t1 values(1, 2); +alter table t1 add column f5 int default 10, algorithm=instant; +alter table t1 add column f3 int not null default 100, algorithm=instant; +alter table t1 add column f4 int default 100, drop column f3, algorithm=instant; +insert into t1 values(2, 3, 20, 100); +select * from t1; +f1 f2 f5 f4 +1 2 10 100 +2 3 20 100 +drop table t1; +create table t1(f1 int not null, f2 int not null) engine=innodb; +insert into t1 values(1, 1); +alter table t1 drop column f2, add column f3 int default 3, algorithm=instant; +select * from t1; +f1 f3 +1 3 +update t1 set f3 = 19; +select * from t1; +f1 f3 +1 19 +alter table t1 drop column f1, add column f5 int default 10, algorithm=instant; +insert into t1 values(4, 10); +select * from t1; +f3 f5 +19 10 +4 10 +create table t2(f1 int, f2 int not null) engine=innodb; +insert into t2(f1, f2) values(1, 2); +alter table t2 drop column f2, add column f4 varchar(100) default repeat('a', 20), add column f5 int default 10, algorithm=instant; +select * from t2; +f1 f4 f5 +1 aaaaaaaaaaaaaaaaaaaa 10 +show create table t2; +Table Create Table +t2 CREATE TABLE `t2` ( + `f1` int(11) DEFAULT NULL, + `f4` varchar(100) DEFAULT repeat('a',20), + `f5` int(11) DEFAULT 10 +) ENGINE=InnoDB DEFAULT CHARSET=latin1 +alter table t2 add column f6 char(100) default repeat('a', 99), algorithm=instant; +create table t3(f1 int, f2 int not null)engine=innodb; +insert into t3 values(1, 2); +alter table t3 drop column f2, add column f3 int default 1, add column f4 int default 4, algorithm=instant; +select * from t1; +f3 f5 +19 10 +4 10 +alter table t1 add column f6 int default 9,drop column f5, algorithm = instant; +insert into t1 values(4, 9); +alter table t1 force, algorithm=inplace; +select * from t1; +f3 f6 +19 9 +4 9 +4 9 +select * from t2; +f1 f4 f5 f6 +1 aaaaaaaaaaaaaaaaaaaa 10 aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa +alter table t2 force, algorithm=inplace; +select * from t2; +f1 f4 f5 f6 +1 aaaaaaaaaaaaaaaaaaaa 10 aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa +show create table t2; +Table Create Table +t2 CREATE TABLE `t2` ( + `f1` int(11) DEFAULT NULL, + `f4` varchar(100) DEFAULT repeat('a',20), + `f5` int(11) DEFAULT 10, + `f6` char(100) DEFAULT repeat('a',99) +) ENGINE=InnoDB DEFAULT CHARSET=latin1 +select * from t3; +f1 f3 f4 +1 1 4 +alter table t3 add column f5 char(100) default repeat('a', 99), algorithm=instant; +select * from t3; +f1 f3 f4 f5 +1 1 4 aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa +drop table t1,t2,t3; diff --git a/mysql-test/suite/innodb/t/innodb-alter.test b/mysql-test/suite/innodb/t/innodb-alter.test index e8266c44a90..4f2ea5f3540 100644 --- a/mysql-test/suite/innodb/t/innodb-alter.test +++ b/mysql-test/suite/innodb/t/innodb-alter.test @@ -206,7 +206,7 @@ CREATE TABLE tu ( ) ENGINE=InnoDB; --error ER_ALTER_OPERATION_NOT_SUPPORTED_REASON ALTER TABLE tu ADD COLUMN c CHAR(1) NOT NULL FIRST, LOCK=NONE; -# Instant ADD COLUMN (adding after the visible FTS_DOC_ID) +--error ER_ALTER_OPERATION_NOT_SUPPORTED_REASON ALTER TABLE tu ADD COLUMN c CHAR(1) NOT NULL, LOCK=NONE; DROP TABLE tu; @@ -217,7 +217,7 @@ CREATE TABLE tv ( ) ENGINE=InnoDB; --error ER_ALTER_OPERATION_NOT_SUPPORTED_REASON ALTER TABLE tv ADD COLUMN c CHAR(1) NOT NULL FIRST, LOCK=NONE; -# Instant ADD COLUMN (adding after the visible FTS_DOC_ID) +--error ER_ALTER_OPERATION_NOT_SUPPORTED_REASON ALTER TABLE tv ADD COLUMN c CHAR(1) NOT NULL, LOCK=NONE; DROP TABLE tv; diff --git a/mysql-test/suite/innodb/t/instant_alter.test b/mysql-test/suite/innodb/t/instant_alter.test index 3e62038e591..fd7fbe94450 100644 --- a/mysql-test/suite/innodb/t/instant_alter.test +++ b/mysql-test/suite/innodb/t/instant_alter.test @@ -361,6 +361,42 @@ COMMIT; --source include/wait_all_purged.inc DROP TABLE t1; +# MDEV-15562 Instant DROP/ADD/reorder columns + +eval CREATE TABLE t1 (a INT, b INT UNIQUE) $engine; +INSERT INTO t1 (a) VALUES (NULL), (NULL); +ALTER TABLE t1 DROP a, ADD COLUMN a INT; +DELETE FROM t1; +BEGIN;INSERT INTO t1 SET a=NULL;ROLLBACK; +DELETE FROM t1; +DROP TABLE t1; + +eval CREATE TABLE t1 (a INT PRIMARY KEY, t VARCHAR(33101) NOT NULL) $engine; +INSERT INTO t1 VALUES(347,''); +ALTER TABLE t1 DROP COLUMN t, ALGORITHM=INSTANT; +SELECT * FROM t1; +DROP TABLE t1; + +eval CREATE TABLE t1 (a INT) $engine; +INSERT INTO t1() VALUES(); +ALTER TABLE t1 ADD COLUMN b INT FIRST, ADD COLUMN c INT AFTER b; +SELECT * FROM t1; +DROP TABLE t1; + +eval CREATE TABLE t1 (t TEXT) $engine; +SET @t=REPEAT('x',@@innodb_page_size / 2); +INSERT INTO t1 VALUES (@t),(@t),(@t),(@t),(@t),(@t),(NULL),(@t),(@t),(@t),(@t); +ALTER TABLE t1 ADD COLUMN a INT FIRST; +UPDATE t1 SET a = 0; +DROP TABLE t1; + +eval CREATE TABLE t1 (t TEXT) $engine; +INSERT INTO t1 SET t = @x; +ALTER TABLE t1 DROP COLUMN t, ADD COLUMN i INT NOT NULL DEFAULT 1; +ALTER TABLE t1 ADD COLUMN t TEXT; +SELECT * FROM t1; +DROP TABLE t1; + dec $format; } disconnect analyze; diff --git a/mysql-test/suite/innodb/t/instant_alter_crash.test b/mysql-test/suite/innodb/t/instant_alter_crash.test index fe7301b4f78..b1615041393 100644 --- a/mysql-test/suite/innodb/t/instant_alter_crash.test +++ b/mysql-test/suite/innodb/t/instant_alter_crash.test @@ -43,9 +43,38 @@ DELETE FROM t1; ROLLBACK; --source include/wait_all_purged.inc +INSERT INTO t2 VALUES +(16,1551,'Omnium enim rerum'),(128,1571,' principia parva sunt'); + +connect ddl, localhost, root; +SET DEBUG_SYNC='innodb_alter_inplace_before_commit SIGNAL ddl WAIT_FOR ever'; +--send +ALTER TABLE t2 DROP COLUMN c3, ADD COLUMN c5 TEXT DEFAULT 'naturam abhorrere'; + +connection default; +SET DEBUG_SYNC='now WAIT_FOR ddl'; +SET GLOBAL innodb_flush_log_at_trx_commit=1; +DELETE FROM t1; + +--source include/kill_mysqld.inc +disconnect ddl; +--source include/start_mysqld.inc + +SET @saved_frequency= @@GLOBAL.innodb_purge_rseg_truncate_frequency; +SET GLOBAL innodb_purge_rseg_truncate_frequency=1; + +SELECT * FROM t1; +SELECT * FROM t2; +BEGIN; +INSERT INTO t1 SET id=1; +DELETE FROM t2; +ROLLBACK; +--source include/wait_all_purged.inc + INSERT INTO t2 VALUES (64,42,'De finibus bonorum'), (347,33101,' et malorum'); connect ddl, localhost, root; +ALTER TABLE t2 DROP COLUMN c3; SET DEBUG_SYNC='innodb_alter_inplace_before_commit SIGNAL ddl WAIT_FOR ever'; --send ALTER TABLE t2 ADD COLUMN (c4 TEXT NOT NULL DEFAULT ' et malorum'); @@ -96,16 +125,39 @@ for (my $offset= 0x65; $offset; my $n_fields= unpack("n", substr($page,$offset-4,2)) >> 1 & 0x3ff; my $start= 0; my $name; - for (my $i= 0; $i < $n_fields; $i++) { - my $end= unpack("C", substr($page, $offset-7-$i, 1)); - print ",\n " if $i; - print "$fields[$i]="; - if ($end & 0x80) { - print "NULL(", ($end & 0x7f) - $start, " bytes)" - } else { - print "0x", unpack("H*", substr($page,$offset+$start,$end-$start)) + if (unpack("C", substr($page,$offset-3,1)) & 1) { + for (my $i= 0; $i < $n_fields; $i++) { + my $end= unpack("C", substr($page, $offset-7-$i, 1)); + print ",\n " if $i; + print "$fields[$i]="; + if ($end & 0x80) { + print "NULL(", ($end & 0x7f) - $start, " bytes)" + } else { + print "0x", unpack("H*", substr($page,$offset+$start,$end-$start)) + } + $start= $end & 0x7f; + } + } else { + for (my $i= 0; $i < $n_fields; $i++) { + my $end= unpack("n", substr($page, $offset-8-2*$i, 2)); + print ",\n " if $i; + if ($i > 2 && !(~unpack("C",substr($page,$offset-6,1)) & 0x30)) { + if ($i == 3) { + print "BLOB="; + $start += 8; # skip the space_id,page_number + } else { + print "$fields[$i - 1]="; + } + } else { + print "$fields[$i]="; + } + if ($end & 0x8000) { + print "NULL(", ($end & 0x7fff) - $start, " bytes)" + } else { + print "0x", unpack("H*", substr($page,$offset+$start,($end-$start) & 0x3fff)) + } + $start= $end & 0x3fff; } - $start= $end & 0x7f; } print ")\n"; } diff --git a/mysql-test/suite/innodb/t/instant_alter_debug.test b/mysql-test/suite/innodb/t/instant_alter_debug.test index cab9decd3d0..05812322317 100644 --- a/mysql-test/suite/innodb/t/instant_alter_debug.test +++ b/mysql-test/suite/innodb/t/instant_alter_debug.test @@ -222,7 +222,8 @@ connection ddl; SET DEBUG_SYNC='row_log_table_apply1_before SIGNAL copied WAIT_FOR logged'; send ALTER TABLE t1 FORCE; -disconnect stop_purge; +connection stop_purge; +COMMIT; connection default; SET DEBUG_SYNC = 'now WAIT_FOR copied'; @@ -235,6 +236,34 @@ SET DEBUG_SYNC = 'now SIGNAL logged'; connection ddl; reap; +connection default; +SET DEBUG_SYNC = RESET; +SELECT * FROM t1; +ALTER TABLE t1 DROP b, ALGORITHM=INSTANT; +connection stop_purge; +START TRANSACTION WITH CONSISTENT SNAPSHOT; + +connection default; +DELETE FROM t1; + +connection ddl; +SET DEBUG_SYNC='row_log_table_apply1_before SIGNAL copied WAIT_FOR logged'; +send ALTER TABLE t1 ADD COLUMN b INT NOT NULL DEFAULT 2 AFTER a, FORCE; + +disconnect stop_purge; + +connection default; +SET DEBUG_SYNC = 'now WAIT_FOR copied'; +let $wait_all_purged = 1; +--source include/wait_all_purged.inc +INSERT INTO t1 SET a=1; +INSERT INTO t1 SET a=2,c=4; +SET DEBUG_SYNC = 'now SIGNAL logged'; + +connection ddl; +reap; +UPDATE t1 SET b = b + 1 WHERE a = 2; + connection default; SET DEBUG_SYNC = RESET; SELECT * FROM t1; diff --git a/mysql-test/suite/innodb/t/instant_alter_limit.test b/mysql-test/suite/innodb/t/instant_alter_limit.test new file mode 100644 index 00000000000..ded14eee89b --- /dev/null +++ b/mysql-test/suite/innodb/t/instant_alter_limit.test @@ -0,0 +1,42 @@ +--source include/have_innodb.inc + +SET @old_instant= +(SELECT variable_value FROM information_schema.global_status +WHERE variable_name = 'innodb_instant_alter_column'); + +CREATE TABLE t(a INT PRIMARY KEY, b INT, c INT, d INT, e INT) +ENGINE=InnoDB; +INSERT INTO t VALUES(1,2,3,4,5); +--disable_query_log +let $n=253; +while ($n) { +dec $n; +ALTER TABLE t DROP b, DROP c, DROP d, DROP e, +ADD COLUMN b INT FIRST, ADD COLUMN c INT, ADD COLUMN d INT AFTER b, +ADD COLUMN e INT AFTER c, ALGORITHM=INSTANT; +} +--enable_query_log +SELECT * FROM t; +--error ER_ALTER_OPERATION_NOT_SUPPORTED +ALTER TABLE t DROP b, DROP c, DROP d, DROP e, +ADD COLUMN b INT, ALGORITHM=INSTANT; +ALTER TABLE t CHANGE COLUMN b beta INT AFTER a, ALGORITHM=INSTANT; +ALTER TABLE t DROP e, DROP c, DROP d, ALGORITHM=INSTANT; +SELECT * FROM t; +ALTER TABLE t DROP COLUMN beta, ALGORITHM=INSTANT; +--error ER_ALTER_OPERATION_NOT_SUPPORTED +ALTER TABLE t ADD COLUMN b INT NOT NULL, ALGORITHM=INSTANT; + +SELECT variable_value-@old_instant instants +FROM information_schema.global_status +WHERE variable_name = 'innodb_instant_alter_column'; + +ALTER TABLE t ADD COLUMN b INT NOT NULL; + +SELECT variable_value-@old_instant instants +FROM information_schema.global_status +WHERE variable_name = 'innodb_instant_alter_column'; + +SELECT * FROM t; + +DROP TABLE t; diff --git a/mysql-test/suite/innodb/t/instant_alter_rollback.test b/mysql-test/suite/innodb/t/instant_alter_rollback.test index b68a6ad3880..cfece7e0738 100644 --- a/mysql-test/suite/innodb/t/instant_alter_rollback.test +++ b/mysql-test/suite/innodb/t/instant_alter_rollback.test @@ -8,28 +8,49 @@ FLUSH TABLES; --echo # --echo # MDEV-11369: Instant ADD COLUMN for InnoDB +--echo # MDEV-15562: Instant DROP COLUMN or changing the order of columns --echo # connect to_be_killed, localhost, root; +SET @old_instant= +(SELECT variable_value FROM information_schema.global_status +WHERE variable_name = 'innodb_instant_alter_column'); CREATE TABLE empty (id INT PRIMARY KEY, c2 INT UNIQUE) ENGINE=InnoDB; CREATE TABLE once LIKE empty; CREATE TABLE twice LIKE empty; +CREATE TABLE thrice LIKE empty; INSERT INTO once SET id=1,c2=1; INSERT INTO twice SET id=1,c2=1; +INSERT INTO thrice SET id=1,c2=1; ALTER TABLE empty ADD COLUMN (d1 INT DEFAULT 15); ALTER TABLE once ADD COLUMN (d1 INT DEFAULT 20); ALTER TABLE twice ADD COLUMN (d1 INT DEFAULT 20); +ALTER TABLE thrice ADD COLUMN (d1 INT DEFAULT 20); ALTER TABLE twice ADD COLUMN (d2 INT NOT NULL DEFAULT 10, d3 VARCHAR(15) NOT NULL DEFAULT 'var och en char'); +ALTER TABLE thrice ADD COLUMN +(d2 INT NOT NULL DEFAULT 10, + d3 TEXT NOT NULL DEFAULT 'con'); +ALTER TABLE thrice DROP c2, DROP d3, CHANGE d2 d3 INT NOT NULL FIRST; + +SELECT variable_value-@old_instant instants +FROM information_schema.global_status +WHERE variable_name = 'innodb_instant_alter_column'; BEGIN; INSERT INTO empty set id=0,c2=42; UPDATE once set c2=c2+1; UPDATE twice set c2=c2+1; +UPDATE thrice set d3=d3+1; INSERT INTO twice SET id=2,c2=0,d3=''; +INSERT INTO thrice SET id=2,d3=0; +DELETE FROM empty; +DELETE FROM once; +DELETE FROM twice; +DELETE FROM thrice; connection default; SET GLOBAL innodb_flush_log_at_trx_commit=1; @@ -48,4 +69,5 @@ SET GLOBAL innodb_purge_rseg_truncate_frequency=@saved_frequency; SELECT * FROM empty; SELECT * FROM once; SELECT * FROM twice; -DROP TABLE empty, once, twice; +SELECT * FROM thrice; +DROP TABLE empty, once, twice, thrice; diff --git a/mysql-test/suite/innodb/t/instant_drop.test b/mysql-test/suite/innodb/t/instant_drop.test new file mode 100644 index 00000000000..7be4998dccd --- /dev/null +++ b/mysql-test/suite/innodb/t/instant_drop.test @@ -0,0 +1,98 @@ +--source include/have_innodb.inc + +create table t1(f1 int not null, f2 int not null, f3 int not null)engine=innodb; +insert into t1 values(1, 2, 3),(4, 5, 6); +alter table t1 drop column f2, algorithm=instant; +select * from t1; +insert into t1 values(1,2); +select * from t1; +alter table t1 add column f4 int not null default 5, algorithm=instant; +select * from t1; +alter table t1 drop column f1, algorithm=instant; +select * from t1; +insert into t1 values(7, 9); +select * from t1; +alter table t1 add column f5 blob default repeat('aaa', 950), drop column f4, algorithm=instant; +select * from t1; +select f3 from t1; +update t1 set f3 = 10 where f3 > 2; +select * from t1; +delete from t1 where f3 = 10; +show create table t1; +select f3 from t1; +update t1 set f5 = 'world'; +select * from t1; +drop table t1; + +create table t1(f1 int, f2 int not null, index idx(f2))engine=innodb; +insert into t1 values(1, 2); +alter table t1 drop column f1, add column f3 varchar(100) default 'thiru', algorithm=instant; +select * from t1 force index (idx); +alter table t1 drop column f3, algorithm=instant; +select * from t1; +begin; +insert into t1 values(10); +select * from t1; +update t1 set f2 = 100; +select * from t1; +delete from t1 where f2 = 100; +select * from t1; +rollback; +select * from t1; +show create table t1; +drop table t1; + +create table t1(f1 int, f2 int not null)engine=innodb; +insert into t1 values(1, 2); +alter table t1 drop column f2, algorithm=instant; +insert into t1 values(NULL); +select * from t1; +drop table t1; + +create table t1(f1 int not null, f2 int not null)engine=innodb; +insert into t1 values(1, 2); +alter table t1 add column f5 int default 10, algorithm=instant; +alter table t1 add column f3 int not null default 100, algorithm=instant; +alter table t1 add column f4 int default 100, drop column f3, algorithm=instant; +insert into t1 values(2, 3, 20, 100); +select * from t1; +drop table t1; + +create table t1(f1 int not null, f2 int not null) engine=innodb; +insert into t1 values(1, 1); +alter table t1 drop column f2, add column f3 int default 3, algorithm=instant; +select * from t1; +update t1 set f3 = 19; +select * from t1; +alter table t1 drop column f1, add column f5 int default 10, algorithm=instant; +insert into t1 values(4, 10); +select * from t1; + +create table t2(f1 int, f2 int not null) engine=innodb; +insert into t2(f1, f2) values(1, 2); +alter table t2 drop column f2, add column f4 varchar(100) default repeat('a', 20), add column f5 int default 10, algorithm=instant; +select * from t2; +show create table t2; +alter table t2 add column f6 char(100) default repeat('a', 99), algorithm=instant; + +create table t3(f1 int, f2 int not null)engine=innodb; +insert into t3 values(1, 2); +alter table t3 drop column f2, add column f3 int default 1, add column f4 int default 4, algorithm=instant; + +--source include/restart_mysqld.inc +select * from t1; +alter table t1 add column f6 int default 9,drop column f5, algorithm = instant; +insert into t1 values(4, 9); +alter table t1 force, algorithm=inplace; +select * from t1; + +select * from t2; +alter table t2 force, algorithm=inplace; +select * from t2; +show create table t2; + +select * from t3; +alter table t3 add column f5 char(100) default repeat('a', 99), algorithm=instant; +--source include/restart_mysqld.inc +select * from t3; +drop table t1,t2,t3; diff --git a/mysql-test/suite/innodb_gis/r/alter_spatial_index.result b/mysql-test/suite/innodb_gis/r/alter_spatial_index.result index 7caa5f6829c..d9febe4b6b6 100644 --- a/mysql-test/suite/innodb_gis/r/alter_spatial_index.result +++ b/mysql-test/suite/innodb_gis/r/alter_spatial_index.result @@ -521,7 +521,7 @@ HEX(c8) ROLLBACK; ALTER TABLE tab add COLUMN c9 POINT NOT NULL AFTER c5, ALGORITHM = INPLACE, LOCK=NONE; -ERROR 0A000: LOCK=NONE is not supported. Reason: Do not support online operation on table with GIS index. Try LOCK=SHARED +ALTER TABLE tab DROP COLUMN c9, ALGORITHM=INSTANT; SHOW CREATE TABLE tab; Table Create Table tab CREATE TABLE `tab` ( diff --git a/mysql-test/suite/innodb_gis/t/alter_spatial_index.test b/mysql-test/suite/innodb_gis/t/alter_spatial_index.test index 653e250017a..5843c6fc8f6 100644 --- a/mysql-test/suite/innodb_gis/t/alter_spatial_index.test +++ b/mysql-test/suite/innodb_gis/t/alter_spatial_index.test @@ -491,9 +491,8 @@ FROM tab LIMIT 1; SELECT HEX(c8) FROM tab; ROLLBACK; -# not instant, not supported ---error ER_ALTER_OPERATION_NOT_SUPPORTED_REASON ALTER TABLE tab add COLUMN c9 POINT NOT NULL AFTER c5, ALGORITHM = INPLACE, LOCK=NONE; +ALTER TABLE tab DROP COLUMN c9, ALGORITHM=INSTANT; SHOW CREATE TABLE tab; diff --git a/mysql-test/suite/versioning/r/online.result b/mysql-test/suite/versioning/r/online.result index b2a34481d63..4c1fed1ecfd 100644 --- a/mysql-test/suite/versioning/r/online.result +++ b/mysql-test/suite/versioning/r/online.result @@ -25,8 +25,7 @@ add s bigint unsigned as row start, add e bigint unsigned as row end, add period for system_time(s, e), add system versioning; -alter table t drop column b, lock=none; -ERROR 0A000: LOCK=NONE is not supported. Reason: Not implemented for system-versioned tables. Try LOCK=SHARED +alter table t drop column b, algorithm=instant; alter table t add index idx(a), lock=none; alter table t drop column s, drop column e; alter table t drop system versioning, lock=none; diff --git a/mysql-test/suite/versioning/t/online.test b/mysql-test/suite/versioning/t/online.test index 4fbd5d85100..537609f48f3 100644 --- a/mysql-test/suite/versioning/t/online.test +++ b/mysql-test/suite/versioning/t/online.test @@ -32,8 +32,7 @@ alter table t add e bigint unsigned as row end, add period for system_time(s, e), add system versioning; ---error ER_ALTER_OPERATION_NOT_SUPPORTED_REASON -alter table t drop column b, lock=none; +alter table t drop column b, algorithm=instant; alter table t add index idx(a), lock=none; alter table t drop column s, drop column e; --error ER_ALTER_OPERATION_NOT_SUPPORTED_REASON diff --git a/storage/innobase/btr/btr0btr.cc b/storage/innobase/btr/btr0btr.cc index 606ab114b2f..27b63d00c1b 100644 --- a/storage/innobase/btr/btr0btr.cc +++ b/storage/innobase/btr/btr0btr.cc @@ -1660,11 +1660,6 @@ btr_page_reorganize_low( goto func_exit; } - if (!recovery && !dict_table_is_locking_disabled(index->table)) { - /* Update the record lock bitmaps */ - lock_move_reorganize_page(block, temp_block); - } - data_size2 = page_get_data_size(page); max_ins_size2 = page_get_max_insert_size_after_reorganize(page, 1); @@ -1688,21 +1683,41 @@ btr_page_reorganize_low( ut_ad(cursor->rec == page_get_infimum_rec(page)); } -func_exit: #ifdef UNIV_ZIP_DEBUG ut_a(!page_zip || page_zip_validate(page_zip, page, index)); #endif /* UNIV_ZIP_DEBUG */ - if (!recovery && page_is_root(temp_page) - && fil_page_get_type(temp_page) == FIL_PAGE_TYPE_INSTANT) { - /* Preserve the PAGE_INSTANT information. */ - ut_ad(!page_zip); - ut_ad(index->is_instant()); - memcpy(FIL_PAGE_TYPE + page, FIL_PAGE_TYPE + temp_page, 2); - memcpy(PAGE_HEADER + PAGE_INSTANT + page, - PAGE_HEADER + PAGE_INSTANT + temp_page, 2); + if (!recovery) { + if (page_is_root(temp_page) + && fil_page_get_type(temp_page) == FIL_PAGE_TYPE_INSTANT) { + /* Preserve the PAGE_INSTANT information. */ + ut_ad(!page_zip); + ut_ad(index->is_instant()); + memcpy(FIL_PAGE_TYPE + page, + FIL_PAGE_TYPE + temp_page, 2); + memcpy(PAGE_HEADER + PAGE_INSTANT + page, + PAGE_HEADER + PAGE_INSTANT + temp_page, 2); + if (!index->table->instant) { + } else if (page_is_comp(page)) { + memcpy(PAGE_NEW_INFIMUM + page, + PAGE_NEW_INFIMUM + temp_page, 8); + memcpy(PAGE_NEW_SUPREMUM + page, + PAGE_NEW_SUPREMUM + temp_page, 8); + } else { + memcpy(PAGE_OLD_INFIMUM + page, + PAGE_OLD_INFIMUM + temp_page, 8); + memcpy(PAGE_OLD_SUPREMUM + page, + PAGE_OLD_SUPREMUM + temp_page, 8); + } + } + + if (!dict_table_is_locking_disabled(index->table)) { + /* Update the record lock bitmaps */ + lock_move_reorganize_page(block, temp_block); + } } +func_exit: buf_block_free(temp_block); /* Restore logging mode */ @@ -1748,6 +1763,14 @@ func_exit: mach_read_from_2(PAGE_HEADER + PAGE_INSTANT + page), MLOG_2BYTES, mtr); + if (!index->table->instant) { + } else if (page_is_comp(page)) { + mlog_log_string(PAGE_NEW_INFIMUM + page, 8, mtr); + mlog_log_string(PAGE_NEW_SUPREMUM + page, 8, mtr); + } else { + mlog_log_string(PAGE_OLD_INFIMUM + page, 8, mtr); + mlog_log_string(PAGE_OLD_SUPREMUM + page, 8, mtr); + } } return(success); @@ -1895,6 +1918,59 @@ btr_page_empty( } } +/** Write instant ALTER TABLE metadata to a root page. +@param[in,out] root clustered index root page +@param[in] index clustered index with instant ALTER TABLE +@param[in,out] mtr mini-transaction */ +void btr_set_instant(buf_block_t* root, const dict_index_t& index, mtr_t* mtr) +{ + ut_ad(index.n_core_fields > 0); + ut_ad(index.n_core_fields < REC_MAX_N_FIELDS); + ut_ad(index.is_instant()); + ut_ad(page_is_root(root->frame)); + + rec_t* infimum = page_get_infimum_rec(root->frame); + rec_t* supremum = page_get_supremum_rec(root->frame); + byte* page_type = root->frame + FIL_PAGE_TYPE; + uint16_t i = page_header_get_field(root->frame, PAGE_INSTANT); + + switch (mach_read_from_2(page_type)) { + case FIL_PAGE_TYPE_INSTANT: + ut_ad(page_get_instant(root->frame) == index.n_core_fields); + if (memcmp(infimum, "infimum", 8) + || memcmp(supremum, "supremum", 8)) { + ut_ad(index.table->instant); + ut_ad(!memcmp(infimum, field_ref_zero, 8)); + ut_ad(!memcmp(supremum, field_ref_zero, 7)); + ut_ad(supremum[7] == index.n_core_null_bytes); + return; + } + break; + default: + ut_ad(!"wrong page type"); + /* fall through */ + case FIL_PAGE_INDEX: + ut_ad(!page_is_comp(root->frame) + || !page_get_instant(root->frame)); + ut_ad(!memcmp(infimum, "infimum", 8)); + ut_ad(!memcmp(supremum, "supremum", 8)); + mlog_write_ulint(page_type, FIL_PAGE_TYPE_INSTANT, + MLOG_2BYTES, mtr); + ut_ad(i <= PAGE_NO_DIRECTION); + i |= index.n_core_fields << 3; + mlog_write_ulint(PAGE_HEADER + PAGE_INSTANT + root->frame, i, + MLOG_2BYTES, mtr); + break; + } + + if (index.table->instant) { + mlog_memset(root, infimum - root->frame, 8, 0, mtr); + mlog_memset(root, supremum - root->frame, 7, 0, mtr); + mlog_write_ulint(&supremum[7], index.n_core_null_bytes, + MLOG_1BYTE, mtr); + } +} + /*************************************************************//** Makes tree one level higher by splitting the root, and inserts the tuple. It is assumed that mtr contains an x-latch on the tree. @@ -2080,11 +2156,7 @@ btr_root_raise_and_insert( if (index->is_instant()) { ut_ad(!root_page_zip); - byte* page_type = root_block->frame + FIL_PAGE_TYPE; - ut_ad(mach_read_from_2(page_type) == FIL_PAGE_INDEX); - mlog_write_ulint(page_type, FIL_PAGE_TYPE_INSTANT, - MLOG_2BYTES, mtr); - page_set_instant(root_block->frame, index->n_core_fields, mtr); + btr_set_instant(root_block, *index, mtr); } /* Set the next node and previous node fields, although @@ -3569,12 +3641,7 @@ btr_lift_page_up( if (page_level == 0 && index->is_instant()) { ut_ad(!father_page_zip); - byte* page_type = father_block->frame + FIL_PAGE_TYPE; - ut_ad(mach_read_from_2(page_type) == FIL_PAGE_INDEX); - mlog_write_ulint(page_type, FIL_PAGE_TYPE_INSTANT, - MLOG_2BYTES, mtr); - page_set_instant(father_block->frame, - index->n_core_fields, mtr); + btr_set_instant(father_block, *index, mtr); } page_level++; @@ -4246,15 +4313,42 @@ btr_discard_only_page_on_level( } #endif /* UNIV_BTR_DEBUG */ + mem_heap_t* heap = NULL; + const rec_t* rec = NULL; + ulint* offsets = NULL; + if (index->table->instant) { + const rec_t* r = page_rec_get_next(page_get_infimum_rec( + block->frame)); + ut_ad(rec_is_metadata(r, *index) == index->is_instant()); + if (rec_is_alter_metadata(r, *index)) { + heap = mem_heap_create(srv_page_size); + offsets = rec_get_offsets(r, index, NULL, true, + ULINT_UNDEFINED, &heap); + rec = rec_copy(mem_heap_alloc(heap, + rec_offs_size(offsets)), + r, offsets); + rec_offs_make_valid(rec, index, true, offsets); + } + } + btr_page_empty(block, buf_block_get_page_zip(block), index, 0, mtr); ut_ad(page_is_leaf(buf_block_get_frame(block))); /* btr_page_empty() is supposed to zero-initialize the field. */ ut_ad(!page_get_instant(block->frame)); if (index->is_primary()) { - /* Concurrent access is prevented by the root_block->lock - X-latch, so this should be safe. */ - index->remove_instant(); + if (rec) { + DBUG_ASSERT(index->table->instant); + DBUG_ASSERT(rec_is_alter_metadata(rec, *index)); + btr_set_instant(block, *index, mtr); + rec = page_cur_insert_rec_low( + page_get_infimum_rec(block->frame), + index, rec, offsets, mtr); + ut_ad(rec); + mem_heap_free(heap); + } else if (index->is_instant()) { + index->clear_instant_add(); + } } else if (!index->table->is_temporary()) { /* We play it safe and reset the free bits for the root */ ibuf_reset_free_bits(block); @@ -4678,14 +4772,32 @@ btr_index_rec_validate( return(FALSE); } + const bool is_alter_metadata = page_is_leaf(page) + && !page_has_prev(page) + && index->is_primary() && index->table->instant + && rec == page_rec_get_next_const(page_get_infimum_rec(page)); + + if (is_alter_metadata + && !rec_is_alter_metadata(rec, page_is_comp(page))) { + btr_index_rec_validate_report(page, rec, index); + + ib::error() << "First record is not ALTER TABLE metadata"; + return FALSE; + } + if (!page_is_comp(page)) { const ulint n_rec_fields = rec_get_n_fields_old(rec); if (n_rec_fields == DICT_FLD__SYS_INDEXES__MERGE_THRESHOLD && index->id == DICT_INDEXES_ID) { /* A record for older SYS_INDEXES table (missing merge_threshold column) is acceptable. */ + } else if (is_alter_metadata) { + if (n_rec_fields != ulint(index->n_fields) + 1) { + goto n_field_mismatch; + } } else if (n_rec_fields < index->n_core_fields || n_rec_fields > index->n_fields) { +n_field_mismatch: btr_index_rec_validate_report(page, rec, index); ib::error() << "Has " << rec_get_n_fields_old(rec) @@ -4704,15 +4816,28 @@ btr_index_rec_validate( offsets = rec_get_offsets(rec, index, offsets, page_is_leaf(page), ULINT_UNDEFINED, &heap); + const dict_field_t* field = index->fields; + ut_ad(rec_offs_n_fields(offsets) + == ulint(index->n_fields) + is_alter_metadata); - for (unsigned i = 0; i < index->n_fields; i++) { - dict_field_t* field = dict_index_get_nth_field(index, i); - ulint fixed_size = dict_col_get_fixed_size( - dict_field_get_col(field), - page_is_comp(page)); - + for (unsigned i = 0; i < rec_offs_n_fields(offsets); i++) { rec_get_nth_field_offs(offsets, i, &len); + ulint fixed_size; + + if (is_alter_metadata && i == index->first_user_field()) { + fixed_size = FIELD_REF_SIZE; + if (len != FIELD_REF_SIZE + || !rec_offs_nth_extern(offsets, i)) { + goto len_mismatch; + } + + continue; + } else { + fixed_size = dict_col_get_fixed_size( + field->col, page_is_comp(page)); + } + /* Note that if fixed_size != 0, it equals the length of a fixed-size column in the clustered index. We should adjust it here. @@ -4724,8 +4849,8 @@ btr_index_rec_validate( && (field->prefix_len ? len > field->prefix_len : (fixed_size && len != fixed_size))) { +len_mismatch: btr_index_rec_validate_report(page, rec, index); - ib::error error; error << "Field " << i << " len is " << len @@ -4743,6 +4868,8 @@ btr_index_rec_validate( } return(FALSE); } + + field++; } #ifdef VIRTUAL_INDEX_DEBUG diff --git a/storage/innobase/btr/btr0cur.cc b/storage/innobase/btr/btr0cur.cc index 2aefdf1df96..1376fbdbfec 100644 --- a/storage/innobase/btr/btr0cur.cc +++ b/storage/innobase/btr/btr0cur.cc @@ -457,8 +457,8 @@ unreadable: return DB_CORRUPTION; } - if (info_bits != REC_INFO_MIN_REC_FLAG - || (comp && rec_get_status(rec) != REC_STATUS_COLUMNS_ADDED)) { + if ((info_bits & ~REC_INFO_DELETED_FLAG) != REC_INFO_MIN_REC_FLAG + || (comp && rec_get_status(rec) != REC_STATUS_INSTANT)) { incompatible: ib::error() << "Table " << index->table->name << " contains unrecognizable instant ALTER metadata"; @@ -476,6 +476,72 @@ incompatible: concurrent operations on the table, including table eviction from the cache. */ + if (info_bits & REC_INFO_DELETED_FLAG) { + /* This metadata record includes a BLOB that identifies + any dropped or reordered columns. */ + ulint trx_id_offset = index->trx_id_offset; + if (!trx_id_offset) { + /* The PRIMARY KEY contains variable-length columns. + For the metadata record, variable-length columns are + always written with zero length. The DB_TRX_ID will + start right after any fixed-length columns. */ + for (uint i = index->n_uniq; i--; ) { + trx_id_offset += index->fields[0].fixed_len; + } + } + + const byte* ptr = rec + trx_id_offset + + (DATA_TRX_ID_LEN + DATA_ROLL_PTR_LEN); + + if (mach_read_from_4(ptr + BTR_EXTERN_LEN)) { + goto incompatible; + } + + uint len = mach_read_from_4(ptr + BTR_EXTERN_LEN + 4); + if (!len + || mach_read_from_4(ptr + BTR_EXTERN_OFFSET) + != FIL_PAGE_DATA + || mach_read_from_4(ptr + BTR_EXTERN_SPACE_ID) + != space->id) { + goto incompatible; + } + + buf_block_t* block = buf_page_get( + page_id_t(space->id, + mach_read_from_4(ptr + BTR_EXTERN_PAGE_NO)), + univ_page_size, RW_S_LATCH, mtr); + buf_block_dbg_add_level(block, SYNC_EXTERN_STORAGE); + if (fil_page_get_type(block->frame) != FIL_PAGE_TYPE_BLOB + || mach_read_from_4(&block->frame[FIL_PAGE_DATA + + BTR_BLOB_HDR_NEXT_PAGE_NO]) + != FIL_NULL + || mach_read_from_4(&block->frame[FIL_PAGE_DATA + + BTR_BLOB_HDR_PART_LEN]) + != len) { + goto incompatible; + } + + /* The unused part of the BLOB page should be zero-filled. */ + for (const byte* b = block->frame + + (FIL_PAGE_DATA + BTR_BLOB_HDR_SIZE) + len, + * const end = block->frame + srv_page_size + - BTR_EXTERN_LEN; + b < end; ) { + if (*b++) { + goto incompatible; + } + } + + if (index->table->deserialise_columns( + &block->frame[FIL_PAGE_DATA + BTR_BLOB_HDR_SIZE], + len)) { + goto incompatible; + } + + /* Proceed to initialize the default values of + any instantly added columns. */ + } + mem_heap_t* heap = NULL; ulint* offsets = rec_get_offsets(rec, index, NULL, true, ULINT_UNDEFINED, &heap); @@ -489,7 +555,8 @@ inconsistent: record, it is also OK to perform READ UNCOMMITTED and then ignore any extra fields, provided that trx_sys.is_registered(DB_TRX_ID). */ - if (rec_offs_n_fields(offsets) > index->n_fields + if (rec_offs_n_fields(offsets) + > ulint(index->n_fields) + !!index->table->instant && !trx_sys.is_registered(current_trx(), row_get_rec_trx_id(rec, index, offsets))) { @@ -497,10 +564,11 @@ inconsistent: } for (unsigned i = index->n_core_fields; i < index->n_fields; i++) { - ulint len; - const byte* data = rec_get_nth_field(rec, offsets, i, &len); dict_col_t* col = index->fields[i].col; - ut_ad(!col->is_instant()); + const unsigned o = i + !!index->table->instant; + ulint len; + const byte* data = rec_get_nth_field(rec, offsets, o, &len); + ut_ad(!col->is_added()); ut_ad(!col->def_val.data); col->def_val.len = len; switch (len) { @@ -511,7 +579,7 @@ inconsistent: continue; } ut_ad(len != UNIV_SQL_DEFAULT); - if (!rec_offs_nth_extern(offsets, i)) { + if (!rec_offs_nth_extern(offsets, o)) { col->def_val.data = mem_heap_dup( index->table->heap, data, len); } else if (len < BTR_EXTERN_FIELD_REF_SIZE @@ -588,30 +656,49 @@ bool btr_cur_instant_root_init(dict_index_t* index, const page_t* page) const uint16_t n = page_get_instant(page); - if (n < index->n_uniq + DATA_ROLL_PTR || n > index->n_fields) { + if (n < index->n_uniq + DATA_ROLL_PTR) { /* The PRIMARY KEY (or hidden DB_ROW_ID) and DB_TRX_ID,DB_ROLL_PTR columns must always be present - as 'core' fields. All fields, including those for - instantly added columns, must be present in the data - dictionary. */ + as 'core' fields. */ return true; } - if (memcmp(page_get_infimum_rec(page), "infimum", 8) - || memcmp(page_get_supremum_rec(page), "supremum", 8)) { - /* In a later format, these fields in a FIL_PAGE_TYPE_INSTANT - root page could be repurposed for something else. */ + if (n > REC_MAX_N_FIELDS) { return true; } index->n_core_fields = n; - ut_ad(!index->is_dummy); - ut_d(index->is_dummy = true); - index->n_core_null_bytes = n == index->n_fields - ? UT_BITS_IN_BYTES(unsigned(index->n_nullable)) - : UT_BITS_IN_BYTES(index->get_n_nullable(n)); - ut_d(index->is_dummy = false); - return false; + + const rec_t* infimum = page_get_infimum_rec(page); + const rec_t* supremum = page_get_supremum_rec(page); + + if (!memcmp(infimum, "infimum", 8) + && !memcmp(supremum, "supremum", 8)) { + if (n > index->n_fields) { + /* All fields, including those for instantly + added columns, must be present in the + data dictionary. */ + return true; + } + + ut_ad(!index->is_dummy); + ut_d(index->is_dummy = true); + index->n_core_null_bytes = UT_BITS_IN_BYTES( + index->get_n_nullable(n)); + ut_d(index->is_dummy = false); + return false; + } + + if (memcmp(infimum, field_ref_zero, 8) + || memcmp(supremum, field_ref_zero, 7)) { + /* The infimum and supremum records must either contain + the original strings, or they must be filled with zero + bytes, except for the bytes that we have repurposed. */ + return true; + } + + index->n_core_null_bytes = supremum[7]; + return index->n_core_null_bytes > 128; } /** Optimistically latches the leaf page or pages requested. @@ -2292,9 +2379,10 @@ need_opposite_intention: ut_ad(index->is_instant()); /* This may be a search tuple for btr_pcur_restore_position(). */ - ut_ad(tuple->info_bits == REC_INFO_METADATA - || tuple->info_bits == REC_INFO_MIN_REC_FLAG); - } else if (rec_is_metadata(btr_cur_get_rec(cursor), index)) { + ut_ad(tuple->is_metadata() + || (tuple->is_metadata(tuple->info_bits + ^ REC_STATUS_INSTANT))); + } else if (rec_is_metadata(btr_cur_get_rec(cursor), *index)) { /* Only user records belong in the adaptive hash index. */ } else { @@ -3257,12 +3345,17 @@ btr_cur_optimistic_insert( leaf = page_is_leaf(page); + if (UNIV_UNLIKELY(entry->is_alter_metadata())) { + ut_ad(leaf); + goto convert_big_rec; + } + /* Calculate the record size when entry is converted to a record */ rec_size = rec_get_converted_size(index, entry, n_ext); if (page_zip_rec_needs_ext(rec_size, page_is_comp(page), dtuple_get_n_fields(entry), page_size)) { - +convert_big_rec: /* The record is so big that we have to store some fields externally on separate database pages */ big_rec_vec = dtuple_convert_big_rec(index, 0, entry, &n_ext); @@ -3433,7 +3526,7 @@ fail_err: } else if (index->disable_ahi) { # endif } else if (entry->info_bits & REC_INFO_MIN_REC_FLAG) { - ut_ad(entry->info_bits == REC_INFO_METADATA); + ut_ad(entry->is_metadata()); ut_ad(index->is_instant()); ut_ad(flags == BTR_NO_LOCKING_FLAG); } else { @@ -3641,7 +3734,7 @@ btr_cur_pessimistic_insert( if (index->disable_ahi); else # endif if (entry->info_bits & REC_INFO_MIN_REC_FLAG) { - ut_ad(entry->info_bits == REC_INFO_METADATA); + ut_ad(entry->is_metadata()); ut_ad(index->is_instant()); ut_ad((flags & ulint(~BTR_KEEP_IBUF_BITMAP)) == BTR_NO_LOCKING_FLAG); @@ -4140,13 +4233,11 @@ btr_cur_trim( const que_thr_t* thr) { if (!index->is_instant()) { - } else if (UNIV_UNLIKELY(update->info_bits == REC_INFO_METADATA)) { + } else if (UNIV_UNLIKELY(update->is_metadata())) { /* We are either updating a metadata record - (instantly adding columns to a table where instant ADD was + (instant ALTER TABLE on a table where instant ALTER was already executed) or rolling back such an operation. */ ut_ad(!upd_get_nth_field(update, 0)->orig_len); - ut_ad(upd_get_nth_field(update, 0)->field_no - > index->n_core_fields); if (thr->graph->trx->in_rollback) { /* This rollback can occur either as part of @@ -4163,6 +4254,19 @@ btr_cur_trim( first instantly added column logged by innobase_add_instant_try(). */ ut_ad(update->n_fields > 2); + if (update->is_alter_metadata()) { + ut_ad(update->fields[0].field_no + == index->first_user_field()); + ut_ad(update->fields[0].new_val.ext); + ut_ad(update->fields[0].new_val.len + == FIELD_REF_SIZE); + ut_ad(entry->n_fields - 1 == index->n_fields); + ulint n_fields = update->fields[1].field_no; + ut_ad(n_fields <= index->n_fields); + entry->n_fields = n_fields; + return; + } + ulint n_fields = upd_get_nth_field(update, 0) ->field_no; ut_ad(n_fields + 1 >= entry->n_fields); @@ -4248,9 +4352,7 @@ btr_cur_optimistic_update( || trx_is_recv(thr_get_trx(thr))); #endif /* UNIV_DEBUG || UNIV_BLOB_LIGHT_DEBUG */ - const bool is_metadata = update->info_bits == REC_INFO_METADATA; - - if (UNIV_LIKELY(!is_metadata) + if (UNIV_LIKELY(!update->is_metadata()) && !row_upd_changes_field_size_or_external(index, *offsets, update)) { @@ -4276,6 +4378,10 @@ any_extern: return(DB_OVERFLOW); } + if (rec_is_metadata(rec, *index) && index->table->instant) { + goto any_extern; + } + for (i = 0; i < upd_get_n_fields(update); i++) { if (dfield_is_ext(&upd_get_nth_field(update, i)->new_val)) { @@ -4334,10 +4440,10 @@ any_extern: } /* We limit max record size to 16k even for 64k page size. */ - if (new_rec_size >= COMPRESSED_REC_MAX_DATA_SIZE || - (!dict_table_is_comp(index->table) - && new_rec_size >= REDUNDANT_REC_MAX_DATA_SIZE)) { - err = DB_OVERFLOW; + if (new_rec_size >= COMPRESSED_REC_MAX_DATA_SIZE || + (!dict_table_is_comp(index->table) + && new_rec_size >= REDUNDANT_REC_MAX_DATA_SIZE)) { + err = DB_OVERFLOW; goto func_exit; } @@ -4410,8 +4516,8 @@ any_extern: lock_rec_store_on_page_infimum(block, rec); } - if (UNIV_UNLIKELY(is_metadata)) { - ut_ad(new_entry->info_bits == REC_INFO_METADATA); + if (UNIV_UNLIKELY(update->is_metadata())) { + ut_ad(new_entry->is_metadata()); ut_ad(index->is_instant()); /* This can be innobase_add_instant_try() performing a subsequent instant ADD COLUMN, or its rollback by @@ -4437,7 +4543,7 @@ any_extern: cursor, new_entry, offsets, heap, 0/*n_ext*/, mtr); ut_a(rec); /* <- We calculated above the insert would fit */ - if (UNIV_UNLIKELY(is_metadata)) { + if (UNIV_UNLIKELY(update->is_metadata())) { /* We must empty the PAGE_FREE list, because if this was a rollback, the shortened metadata record would have too many fields, and we would be unable to @@ -4631,8 +4737,25 @@ btr_cur_pessimistic_update( rec, index, *offsets, page_is_leaf(page), ULINT_UNDEFINED, offsets_heap); - dtuple_t* new_entry = row_rec_to_index_entry( - rec, index, *offsets, &n_ext, entry_heap); + dtuple_t* new_entry; + + const bool is_metadata = rec_is_metadata(rec, *index); + + if (UNIV_UNLIKELY(is_metadata)) { + ut_ad(update->is_metadata()); + ut_ad(flags & BTR_NO_LOCKING_FLAG); + ut_ad(index->is_instant()); + new_entry = row_metadata_to_tuple( + rec, index, *offsets, + &n_ext, entry_heap, + update->info_bits, !thr_get_trx(thr)->in_rollback); + ut_ad(new_entry->n_fields + == ulint(index->n_fields) + + update->is_alter_metadata()); + } else { + new_entry = row_rec_to_index_entry(rec, index, *offsets, + &n_ext, entry_heap); + } /* The page containing the clustered index record corresponding to new_entry is latched in mtr. If the @@ -4644,9 +4767,6 @@ btr_cur_pessimistic_update( entry_heap); btr_cur_trim(new_entry, index, update, thr); - const bool is_metadata = new_entry->info_bits - & REC_INFO_MIN_REC_FLAG; - /* We have to set appropriate extern storage bits in the new record to be inserted: we have to remember which fields were such */ @@ -4674,11 +4794,14 @@ btr_cur_pessimistic_update( } if (page_zip_rec_needs_ext( - rec_get_converted_size(index, new_entry, n_ext), - page_is_comp(page), - dict_index_get_n_fields(index), - block->page.size)) { - + rec_get_converted_size(index, new_entry, n_ext), + page_is_comp(page), + dict_index_get_n_fields(index), + block->page.size) + || (UNIV_UNLIKELY(update->is_alter_metadata()) + && !dfield_is_ext(dtuple_get_nth_field( + new_entry, + index->first_user_field())))) { big_rec_vec = dtuple_convert_big_rec(index, update, new_entry, &n_ext); if (UNIV_UNLIKELY(big_rec_vec == NULL)) { @@ -4739,10 +4862,10 @@ btr_cur_pessimistic_update( } if (UNIV_UNLIKELY(is_metadata)) { - ut_ad(new_entry->info_bits == REC_INFO_METADATA); + ut_ad(new_entry->is_metadata()); ut_ad(index->is_instant()); /* This can be innobase_add_instant_try() performing a - subsequent instant ADD COLUMN, or its rollback by + subsequent instant ALTER TABLE, or its rollback by row_undo_mod_clust_low(). */ ut_ad(flags & BTR_NO_LOCKING_FLAG); } else { @@ -4791,7 +4914,8 @@ btr_cur_pessimistic_update( btr_cur_get_block(cursor), rec, block); } - if (!rec_get_deleted_flag(rec, rec_offs_comp(*offsets))) { + if (!rec_get_deleted_flag(rec, rec_offs_comp(*offsets)) + || rec_is_alter_metadata(rec, *index)) { /* The new inserted record owns its possible externally stored fields */ btr_cur_unmark_extern_fields( @@ -5434,42 +5558,41 @@ btr_cur_optimistic_delete_func( if (UNIV_UNLIKELY(page_is_root(block->frame) && page_get_n_recs(block->frame) == 1 + (cursor->index->is_instant() - && !rec_is_metadata(rec, cursor->index)))) { + && !rec_is_metadata(rec, *cursor->index)))) { /* The whole index (and table) becomes logically empty. Empty the whole page. That is, if we are deleting the only user record, also delete the metadata record - if one exists (it exists if and only if is_instant()). + if one exists for instant ADD COLUMN (not generic ALTER TABLE). If we are deleting the metadata record and the table becomes empty, clean up the whole page. */ dict_index_t* index = cursor->index; + const rec_t* first_rec = page_rec_get_next_const( + page_get_infimum_rec(block->frame)); ut_ad(!index->is_instant() - || rec_is_metadata( - page_rec_get_next_const( - page_get_infimum_rec(block->frame)), - index)); - if (UNIV_UNLIKELY(rec_get_info_bits(rec, page_rec_is_comp(rec)) - & REC_INFO_MIN_REC_FLAG)) { - /* This should be rolling back instant ADD COLUMN. - If this is a recovered transaction, then - index->is_instant() will hold until the - insert into SYS_COLUMNS is rolled back. */ - ut_ad(index->table->supports_instant()); - ut_ad(index->is_primary()); - } else { - lock_update_delete(block, rec); - } - btr_page_empty(block, buf_block_get_page_zip(block), - index, 0, mtr); - page_cur_set_after_last(block, btr_cur_get_page_cur(cursor)); - - if (index->is_primary()) { - /* Concurrent access is prevented by - root_block->lock X-latch, so this should be - safe. */ - index->remove_instant(); + || rec_is_metadata(first_rec, *index)); + const bool is_metadata = rec_is_metadata(rec, *index); + /* We can remove the metadata when rolling back an + instant ALTER TABLE operation, or when deleting the + last user record on the page such that only metadata for + instant ADD COLUMN (not generic ALTER TABLE) remains. */ + const bool empty_table = is_metadata + || !index->is_instant() + || (first_rec != rec + && rec_is_add_metadata(first_rec, *index)); + if (UNIV_LIKELY(empty_table)) { + if (UNIV_LIKELY(!is_metadata)) { + lock_update_delete(block, rec); + } + btr_page_empty(block, buf_block_get_page_zip(block), + index, 0, mtr); + if (index->is_instant()) { + /* MDEV-17383: free metadata BLOBs! */ + index->clear_instant_alter(); + } + page_cur_set_after_last(block, + btr_cur_get_page_cur(cursor)); + return true; } - - return true; } offsets = rec_get_offsets(rec, cursor->index, offsets, true, @@ -5649,10 +5772,10 @@ btr_cur_pessimistic_delete( } if (page_is_leaf(page)) { - const bool is_metadata = rec_get_info_bits( - rec, page_rec_is_comp(rec)) & REC_INFO_MIN_REC_FLAG; + const bool is_metadata = rec_is_metadata( + rec, page_rec_is_comp(rec)); if (UNIV_UNLIKELY(is_metadata)) { - /* This should be rolling back instant ADD COLUMN. + /* This should be rolling back instant ALTER TABLE. If this is a recovered transaction, then index->is_instant() will hold until the insert into SYS_COLUMNS is rolled back. */ @@ -5668,30 +5791,33 @@ btr_cur_pessimistic_delete( goto discard_page; } } else if (page_get_n_recs(page) == 1 - + (index->is_instant() - && !rec_is_metadata(rec, index))) { + + (index->is_instant() && !is_metadata)) { /* The whole index (and table) becomes logically empty. Empty the whole page. That is, if we are deleting the only user record, also delete the metadata record - if one exists (it exists if and only if is_instant()). + if one exists for instant ADD COLUMN + (not generic ALTER TABLE). If we are deleting the metadata record and the table becomes empty, clean up the whole page. */ + + const rec_t* first_rec = page_rec_get_next_const( + page_get_infimum_rec(page)); ut_ad(!index->is_instant() - || rec_is_metadata( - page_rec_get_next_const( - page_get_infimum_rec(page)), - index)); - btr_page_empty(block, page_zip, index, 0, mtr); - page_cur_set_after_last(block, - btr_cur_get_page_cur(cursor)); - if (index->is_primary()) { - /* Concurrent access is prevented by - index->lock and root_block->lock - X-latch, so this should be safe. */ - index->remove_instant(); + || rec_is_metadata(first_rec, *index)); + if (is_metadata || !index->is_instant() + || (first_rec != rec + && rec_is_add_metadata(first_rec, *index))) { + btr_page_empty(block, page_zip, index, 0, mtr); + if (index->is_instant()) { + /* MDEV-17383: free metadata BLOBs! */ + index->clear_instant_alter(); + } + page_cur_set_after_last( + block, + btr_cur_get_page_cur(cursor)); + ret = TRUE; + goto return_after_reservations; } - ret = TRUE; - goto return_after_reservations; } if (UNIV_LIKELY(!is_metadata)) { diff --git a/storage/innobase/btr/btr0pcur.cc b/storage/innobase/btr/btr0pcur.cc index 41661d226e1..ac28cd1e665 100644 --- a/storage/innobase/btr/btr0pcur.cc +++ b/storage/innobase/btr/btr0pcur.cc @@ -151,13 +151,20 @@ btr_pcur_store_position( rec = page_rec_get_prev(rec); ut_ad(!page_rec_is_infimum(rec)); - ut_ad(!rec_is_metadata(rec, index)); + if (UNIV_UNLIKELY(rec_is_metadata(rec, *index))) { + ut_ad(index->table->instant); + ut_ad(page_get_n_recs(block->frame) == 1); + ut_ad(page_is_leaf(page)); + ut_ad(page_get_page_no(page) == index->page); + cursor->rel_pos = BTR_PCUR_AFTER_LAST_IN_TREE; + return; + } cursor->rel_pos = BTR_PCUR_AFTER; } else if (page_rec_is_infimum_low(offs)) { rec = page_rec_get_next(rec); - if (rec_is_metadata(rec, index)) { + if (rec_is_metadata(rec, *index)) { rec = page_rec_get_next(rec); ut_ad(!page_rec_is_supremum(rec)); } diff --git a/storage/innobase/btr/btr0sea.cc b/storage/innobase/btr/btr0sea.cc index 0cb7611b433..58a6458c74b 100644 --- a/storage/innobase/btr/btr0sea.cc +++ b/storage/innobase/btr/btr0sea.cc @@ -1190,7 +1190,7 @@ retry: rec = page_get_infimum_rec(page); rec = page_rec_get_next_low(rec, page_is_comp(page)); - if (rec_is_metadata(rec, index)) { + if (rec_is_metadata(rec, *index)) { rec = page_rec_get_next_low(rec, page_is_comp(page)); } @@ -1398,7 +1398,7 @@ btr_search_build_page_hash_index( rec = page_rec_get_next_const(page_get_infimum_rec(page)); - if (rec_is_metadata(rec, index)) { + if (rec_is_metadata(rec, *index)) { rec = page_rec_get_next_const(rec); if (!--n_recs) return; } @@ -1862,7 +1862,7 @@ btr_search_update_hash_on_insert(btr_cur_t* cursor, rw_lock_t* ahi_latch) n_bytes, index->id); } - if (!page_rec_is_infimum(rec) && !rec_is_metadata(rec, index)) { + if (!page_rec_is_infimum(rec) && !rec_is_metadata(rec, *index)) { offsets = rec_get_offsets( rec, index, offsets, true, btr_search_get_n_fields(n_fields, n_bytes), &heap); diff --git a/storage/innobase/data/data0data.cc b/storage/innobase/data/data0data.cc index ad97674d252..edc05a34130 100644 --- a/storage/innobase/data/data0data.cc +++ b/storage/innobase/data/data0data.cc @@ -60,7 +60,12 @@ void dtuple_t::trim(const dict_index_t& index) for (; i > index.n_core_fields; i--) { const dfield_t* dfield = dtuple_get_nth_field(this, i - 1); const dict_col_t* col = dict_index_get_nth_col(&index, i - 1); - ut_ad(col->is_instant()); + + if (col->is_dropped()) { + continue; + } + + ut_ad(col->is_added()); ulint len = dfield_get_len(dfield); if (len != col->def_val.len) { break; @@ -632,14 +637,23 @@ dtuple_convert_big_rec( n_fields = 0; ulint longest_i; + const bool mblob = entry->is_alter_metadata(); + ut_ad(entry->n_fields >= index->first_user_field() + mblob); + ut_ad(entry->n_fields - mblob <= index->n_fields); + + if (mblob) { + longest_i = index->first_user_field(); + dfield = dtuple_get_nth_field(entry, longest_i); + local_len = BTR_EXTERN_FIELD_REF_SIZE; + goto ext_write; + } + if (!dict_table_has_atomic_blobs(index->table)) { - /* ROW_FORMAT=REDUNDANT or ROW_FORMAT=COMPACT: - store a 768-byte prefix locally */ + /* up to MySQL 5.1: store a 768-byte prefix locally */ local_len = BTR_EXTERN_FIELD_REF_SIZE + DICT_ANTELOPE_MAX_INDEX_COL_LEN; } else { - /* ROW_FORMAT=DYNAMIC or ROW_FORMAT=COMPRESSED: - do not store any BLOB prefix locally */ + /* new-format table: do not store any BLOB prefix locally */ local_len = BTR_EXTERN_FIELD_REF_SIZE; } @@ -649,11 +663,10 @@ dtuple_convert_big_rec( dict_index_get_n_fields(index), dict_table_page_size(index->table))) { longest_i = 0; - for (ulint i = index->first_user_field(), longest = 0; - i < entry->n_fields; i++) { + i + mblob < entry->n_fields; i++) { ulint savings; - dfield = dtuple_get_nth_field(entry, i); + dfield = dtuple_get_nth_field(entry, i + mblob); const dict_field_t* ifield = dict_index_get_nth_field( index, i); @@ -711,8 +724,8 @@ skip_field: We store the first bytes locally to the record. Then we can calculate all ordering fields in all indexes from locally stored data. */ - dfield = dtuple_get_nth_field(entry, longest_i); +ext_write: local_prefix_len = local_len - BTR_EXTERN_FIELD_REF_SIZE; vector->append( diff --git a/storage/innobase/dict/dict0mem.cc b/storage/innobase/dict/dict0mem.cc index a26354657f7..16633b15267 100644 --- a/storage/innobase/dict/dict0mem.cc +++ b/storage/innobase/dict/dict0mem.cc @@ -190,8 +190,6 @@ dict_mem_table_create( || DICT_TF2_FLAG_IS_SET(table, DICT_TF2_FTS_ADD_DOC_ID)) { table->fts = fts_create(table); table->fts->cache = fts_cache_create(table); - } else { - table->fts = NULL; } new(&table->foreign_set) dict_foreign_set(); @@ -531,6 +529,14 @@ dict_mem_table_col_rename_low( = dict_index_get_nth_field( index, i); + ut_ad(!field->name + == field->col->is_dropped()); + if (!field->name) { + /* dropped columns lack a name */ + ut_ad(index->is_instant()); + continue; + } + /* if is_virtual and that in field->col does not match, continue */ if ((!is_virtual) != @@ -717,6 +723,7 @@ dict_mem_fill_column_struct( column->mbmaxlen = mbmaxlen; column->def_val.data = NULL; column->def_val.len = UNIV_SQL_DEFAULT; + ut_ad(!column->is_dropped()); } /**********************************************************************//** @@ -1193,217 +1200,140 @@ operator<< (std::ostream& out, const dict_foreign_set& fk_set) return(out); } -/** Adjust clustered index metadata for instant ADD COLUMN. -@param[in] clustered index definition after instant ADD COLUMN */ -inline void dict_index_t::instant_add_field(const dict_index_t& instant) +/** Reconstruct the clustered index fields. */ +inline void dict_index_t::reconstruct_fields() { DBUG_ASSERT(is_primary()); - DBUG_ASSERT(instant.is_primary()); - DBUG_ASSERT(!instant.is_instant()); - DBUG_ASSERT(n_def == n_fields); - DBUG_ASSERT(instant.n_def == instant.n_fields); - - DBUG_ASSERT(type == instant.type); - DBUG_ASSERT(trx_id_offset == instant.trx_id_offset); - DBUG_ASSERT(n_user_defined_cols == instant.n_user_defined_cols); - DBUG_ASSERT(n_uniq == instant.n_uniq); - DBUG_ASSERT(instant.n_fields > n_fields); - DBUG_ASSERT(instant.n_def > n_def); - DBUG_ASSERT(instant.n_nullable >= n_nullable); - DBUG_ASSERT(instant.n_core_fields >= n_core_fields); - DBUG_ASSERT(instant.n_core_null_bytes >= n_core_null_bytes); - - n_fields = instant.n_fields; - n_def = instant.n_def; - n_nullable = instant.n_nullable; - fields = static_cast( - mem_heap_dup(heap, instant.fields, n_fields * sizeof *fields)); - - ut_d(unsigned n_null = 0); - - for (unsigned i = 0; i < n_fields; i++) { - DBUG_ASSERT(fields[i].same(instant.fields[i])); - const dict_col_t* icol = instant.fields[i].col; - DBUG_ASSERT(!icol->is_virtual()); - dict_col_t* col = fields[i].col = &table->cols[ - icol - instant.table->cols]; - fields[i].name = col->name(*table); - ut_d(n_null += col->is_nullable()); - } - ut_ad(n_null == n_nullable); -} + n_fields += table->instant->n_dropped; + n_def += table->instant->n_dropped; -/** Adjust metadata for instant ADD COLUMN. -@param[in] table table definition after instant ADD COLUMN */ -void dict_table_t::instant_add_column(const dict_table_t& table) -{ - DBUG_ASSERT(!table.cached); - DBUG_ASSERT(table.n_def == table.n_cols); - DBUG_ASSERT(table.n_t_def == table.n_t_cols); - DBUG_ASSERT(n_def == n_cols); - DBUG_ASSERT(n_t_def == n_t_cols); - DBUG_ASSERT(table.n_cols > n_cols); - ut_ad(mutex_own(&dict_sys->mutex)); - - const char* end = table.col_names; - for (unsigned i = table.n_cols; i--; ) end += strlen(end) + 1; - - col_names = static_cast( - mem_heap_dup(heap, table.col_names, - ulint(end - table.col_names))); - const dict_col_t* const old_cols = cols; - const dict_col_t* const old_cols_end = cols + n_cols; - cols = static_cast(mem_heap_dup(heap, table.cols, - table.n_cols - * sizeof *cols)); - - /* Preserve the default values of previously instantly - added columns. */ - for (unsigned i = unsigned(n_cols) - DATA_N_SYS_COLS; i--; ) { - cols[i].def_val = old_cols[i].def_val; - } + const unsigned n_first = first_user_field(); + + dict_field_t* tfields = static_cast( + mem_heap_zalloc(heap, n_fields * sizeof *fields)); + + memcpy(tfields, fields, n_first * sizeof *fields); - /* Copy the new default values to this->heap. */ - for (unsigned i = n_cols; i < table.n_cols; i++) { - dict_col_t& c = cols[i - DATA_N_SYS_COLS]; - DBUG_ASSERT(c.is_instant()); - if (c.def_val.len == 0) { - c.def_val.data = field_ref_zero; - } else if (const void*& d = c.def_val.data) { - d = mem_heap_dup(heap, d, c.def_val.len); + n_nullable = 0; + ulint n_core_null = 0; + const bool comp = dict_table_is_comp(table); + const unsigned* non_pk_col_map = table->instant->non_pk_col_map; + for (unsigned i = n_first, o = i, j = 0; i < n_fields; ) { + dict_field_t& f = tfields[i++]; + unsigned c = *non_pk_col_map++; + if (c & 1U << 15) { + f.col = &table->instant->dropped[j++]; + ut_ad(f.col->is_dropped()); + f.fixed_len = dict_col_get_fixed_size(f.col, comp); } else { - DBUG_ASSERT(c.def_val.len == UNIV_SQL_NULL); + f = fields[o++]; + f.col = dict_table_get_nth_col(table, c); + f.name = f.col->name(*table); } - } - const unsigned old_n_cols = n_cols; - const unsigned n_add = unsigned(table.n_cols - n_cols); - - n_t_def += n_add; - n_t_cols += n_add; - n_cols = table.n_cols; - n_def = n_cols; - - for (unsigned i = n_v_def; i--; ) { - const dict_v_col_t& v = v_cols[i]; - for (ulint n = v.num_base; n--; ) { - dict_col_t*& base = v.base_col[n]; - if (!base->is_virtual()) { - DBUG_ASSERT(base >= old_cols); - size_t n = size_t(base - old_cols); - DBUG_ASSERT(n + DATA_N_SYS_COLS < old_n_cols); - base = &cols[n]; - } + f.col->clear_instant(); + if (f.col->is_nullable()) { + n_nullable++; + n_core_null += i <= n_core_fields; } } - dict_index_t* index = dict_table_get_first_index(this); - - index->instant_add_field(*dict_table_get_first_index(&table)); - - while ((index = dict_table_get_next_index(index)) != NULL) { - for (unsigned i = 0; i < index->n_fields; i++) { - dict_field_t& field = index->fields[i]; - if (field.col < old_cols - || field.col >= old_cols_end) { - DBUG_ASSERT(field.col->is_virtual()); - } else { - /* Secondary indexes may contain user - columns and DB_ROW_ID (if there is - GEN_CLUST_INDEX instead of PRIMARY KEY), - but not DB_TRX_ID,DB_ROLL_PTR. */ - DBUG_ASSERT(field.col >= old_cols); - size_t n = size_t(field.col - old_cols); - DBUG_ASSERT(n + DATA_N_SYS_COLS <= old_n_cols); - if (n + DATA_N_SYS_COLS >= old_n_cols) { - /* Replace DB_ROW_ID */ - n += n_add; - } - field.col = &cols[n]; - DBUG_ASSERT(!field.col->is_virtual()); - field.name = field.col->name(*this); - } - } + fields = tfields; + n_core_null_bytes = UT_BITS_IN_BYTES(n_core_null); +} + +/** Serialise metadata of dropped or reordered columns. +@param[in,out] heap memory heap for allocation +@param[out] field data field with the metadata */ +void dict_table_t::serialise_columns(mem_heap_t* heap, dfield_t* field) const +{ + DBUG_ASSERT(instant); + const dict_index_t& index = *UT_LIST_GET_FIRST(indexes); + unsigned n_fixed = index.first_user_field(); + unsigned num_non_pk_fields = index.n_fields - n_fixed; + + ulint len = 4 + num_non_pk_fields * 2; + + byte* data = static_cast(mem_heap_alloc(heap, len)); + + dfield_set_data(field, data, len); + + mach_write_to_4(data, num_non_pk_fields); + + data += 4; + + for (ulint i = n_fixed; i < index.n_fields; i++) { + mach_write_to_2(data, instant->non_pk_col_map[i - n_fixed]); + data += 2; } } -/** Roll back instant_add_column(). -@param[in] old_n_cols original n_cols -@param[in] old_cols original cols -@param[in] old_col_names original col_names */ -void -dict_table_t::rollback_instant( - unsigned old_n_cols, - dict_col_t* old_cols, - const char* old_col_names) +/** Reconstruct dropped or reordered columns. +@param[in] metadata data from serialise_columns() +@param[in] len length of the metadata, in bytes +@return whether parsing the metadata failed */ +bool dict_table_t::deserialise_columns(const byte* metadata, ulint len) { - ut_ad(mutex_own(&dict_sys->mutex)); - dict_index_t* index = indexes.start; - /* index->is_instant() does not necessarily hold here, because - the table may have been emptied */ - DBUG_ASSERT(old_n_cols >= DATA_N_SYS_COLS); - DBUG_ASSERT(n_cols >= old_n_cols); - DBUG_ASSERT(n_cols == n_def); - DBUG_ASSERT(index->n_def == index->n_fields); - DBUG_ASSERT(index->n_core_fields <= index->n_fields); - - const unsigned n_remove = n_cols - old_n_cols; - - for (unsigned i = index->n_fields - n_remove; i < index->n_fields; - i++) { - if (index->fields[i].col->is_nullable()) { - index->n_nullable--; - } + DBUG_ASSERT(!instant); + + unsigned num_non_pk_fields = mach_read_from_4(metadata); + metadata += 4; + + if (num_non_pk_fields >= REC_MAX_N_FIELDS - 3) { + return true; } - index->n_fields -= n_remove; - index->n_def = index->n_fields; - if (index->n_core_fields > index->n_fields) { - index->n_core_fields = index->n_fields; - index->n_core_null_bytes - = UT_BITS_IN_BYTES(unsigned(index->n_nullable)); + dict_index_t* index = UT_LIST_GET_FIRST(indexes); + + if (num_non_pk_fields < unsigned(index->n_fields) + - index->first_user_field()) { + return true; } - const dict_col_t* const new_cols = cols; - const dict_col_t* const new_cols_end = cols + n_cols; - - cols = old_cols; - col_names = old_col_names; - n_cols = old_n_cols; - n_def = old_n_cols; - n_t_def -= n_remove; - n_t_cols -= n_remove; - - for (unsigned i = n_v_def; i--; ) { - const dict_v_col_t& v = v_cols[i]; - for (ulint n = v.num_base; n--; ) { - dict_col_t*& base = v.base_col[n]; - if (!base->is_virtual()) { - base = &cols[base - new_cols]; + unsigned* non_pk_col_map = static_cast( + mem_heap_alloc(heap, + num_non_pk_fields * sizeof *non_pk_col_map)); + + unsigned n_dropped_cols = 0; + + for (unsigned i = 0; i < num_non_pk_fields; i++) { + non_pk_col_map[i] = mach_read_from_2(metadata); + metadata += 2; + + if (non_pk_col_map[i] & 1U << 15) { + if ((non_pk_col_map[i] & ~(3U << 14)) + > DICT_MAX_FIXED_COL_LEN + 1) { + return true; } + n_dropped_cols++; + } else if (non_pk_col_map[i] >= n_cols) { + return true; } } - do { - for (unsigned i = 0; i < index->n_fields; i++) { - dict_field_t& field = index->fields[i]; - if (field.col < new_cols - || field.col >= new_cols_end) { - DBUG_ASSERT(field.col->is_virtual()); - } else { - DBUG_ASSERT(field.col >= new_cols); - size_t n = size_t(field.col - new_cols); - DBUG_ASSERT(n <= n_cols); - if (n + DATA_N_SYS_COLS >= n_cols) { - n -= n_remove; - } - field.col = &cols[n]; - DBUG_ASSERT(!field.col->is_virtual()); - field.name = field.col->name(*this); - } + dict_col_t* dropped_cols = static_cast(mem_heap_zalloc( + heap, n_dropped_cols * sizeof(dict_col_t))); + instant = new (mem_heap_alloc(heap, sizeof *instant)) dict_instant_t(); + instant->n_dropped = n_dropped_cols; + instant->dropped = dropped_cols; + instant->non_pk_col_map = non_pk_col_map; + + dict_col_t* col = dropped_cols; + for (unsigned i = 0; i < num_non_pk_fields; i++) { + if (non_pk_col_map[i] & 1U << 15) { + unsigned fixed_len = non_pk_col_map[i] & ~(3U << 14); + DBUG_ASSERT(fixed_len <= DICT_MAX_FIXED_COL_LEN + 1); + (col++)->set_dropped(non_pk_col_map[i] & 1U << 14, + fixed_len == 1, + fixed_len > 1 ? fixed_len - 1 + : 0); } - } while ((index = dict_table_get_next_index(index)) != NULL); + } + DBUG_ASSERT(col == &dropped_cols[n_dropped_cols]); + + UT_LIST_GET_FIRST(indexes)->reconstruct_fields(); + return false; } /** Check if record in clustered index is historical row. diff --git a/storage/innobase/fts/fts0fts.cc b/storage/innobase/fts/fts0fts.cc index a558775a97a..98cdc607abe 100644 --- a/storage/innobase/fts/fts0fts.cc +++ b/storage/innobase/fts/fts0fts.cc @@ -3733,7 +3733,7 @@ fts_get_max_doc_id( goto func_exit; } - ut_ad(!rec_is_metadata(rec, index)); + ut_ad(!rec_is_metadata(rec, *index)); offsets = rec_get_offsets( rec, index, offsets, true, ULINT_UNDEFINED, &heap); diff --git a/storage/innobase/handler/ha_innodb.cc b/storage/innobase/handler/ha_innodb.cc index b2df3983831..30bd9444264 100644 --- a/storage/innobase/handler/ha_innodb.cc +++ b/storage/innobase/handler/ha_innodb.cc @@ -9462,12 +9462,14 @@ ha_innobase::change_active_index( } #endif } else { - dtuple_set_n_fields(m_prebuilt->search_tuple, - m_prebuilt->index->n_fields); + ulint n_fields = dict_index_get_n_unique_in_tree( + m_prebuilt->index); + + dtuple_set_n_fields(m_prebuilt->search_tuple, n_fields); dict_index_copy_types( m_prebuilt->search_tuple, m_prebuilt->index, - m_prebuilt->index->n_fields); + n_fields); /* If it's FTS query and FTS_DOC_ID exists FTS_DOC_ID field is always added to read_set. */ diff --git a/storage/innobase/handler/handler0alter.cc b/storage/innobase/handler/handler0alter.cc index 0b0c7a0e42e..ec8eea636f9 100644 --- a/storage/innobase/handler/handler0alter.cc +++ b/storage/innobase/handler/handler0alter.cc @@ -133,6 +133,573 @@ static const alter_table_operations INNOBASE_ALTER_INSTANT | ALTER_COLUMN_UNVERSIONED | ALTER_DROP_VIRTUAL_COLUMN; +/** Set is_instant() before instant_column(). +@param[in] old previous table definition +@param[in] col_map map from old.cols[] and old.v_cols[] to this +@param[out] first_alter_pos 0, or 1 + first changed column position */ +inline void dict_table_t::prepare_instant(const dict_table_t& old, + const ulint* col_map, + unsigned& first_alter_pos) +{ + DBUG_ASSERT(!is_instant()); + DBUG_ASSERT(n_dropped() == 0); + DBUG_ASSERT(old.n_cols == old.n_def); + DBUG_ASSERT(n_cols == n_def); + + const dict_index_t& oindex = *old.indexes.start; + dict_index_t& index = *indexes.start; + first_alter_pos = 0; + + for (unsigned i = 0; i + DATA_N_SYS_COLS < old.n_cols; + i++) { + if (col_map[i] != i) { + first_alter_pos = 1 + i; + goto add_metadata; + } + } + + if (!old.instant) { + /* Columns were not dropped or reordered. + Therefore columns must have been added at the end. */ + DBUG_ASSERT(index.n_fields > oindex.n_fields); +set_core_fields: + index.n_core_fields = oindex.n_core_fields; + index.n_core_null_bytes = oindex.n_core_null_bytes; + } else { +add_metadata: + const unsigned n_old_drop = old.n_dropped(); + unsigned n_drop = n_old_drop; + for (unsigned i = old.n_cols; i--; ) { + if (col_map[i] == ULINT_UNDEFINED) { + DBUG_ASSERT(i + DATA_N_SYS_COLS + < uint(old.n_cols)); + n_drop++; + } + } + + instant = new (mem_heap_alloc(heap, sizeof(dict_instant_t))) + dict_instant_t(); + instant->n_dropped = n_drop; + if (n_drop) { + instant->dropped + = static_cast( + mem_heap_alloc(heap, n_drop + * sizeof(dict_col_t))); + if (n_old_drop) { + memcpy(instant->dropped, old.instant->dropped, + n_old_drop * sizeof(dict_col_t)); + } + } else { + instant->dropped = NULL; + } + + unsigned d = n_old_drop; + + for (unsigned i = 0; i < old.n_cols; i++) { + if (col_map[i] == ULINT_UNDEFINED) { + (new (&instant->dropped[d++]) + dict_col_t(old.cols[i]))->set_dropped(); + } + } +#ifndef DBUG_OFF + for (unsigned i = 0; i < n_drop; i++) { + DBUG_ASSERT(instant->dropped[i].is_dropped()); + } +#endif + DBUG_ASSERT(d == n_drop); + const uint n_fields = index.n_fields + n_dropped(); + + DBUG_ASSERT(n_fields >= oindex.n_fields); + dict_field_t* fields = static_cast( + mem_heap_zalloc(heap, n_fields * sizeof *fields)); + d = n_old_drop; + uint i = 0, j = 0, n_nullable = 0; + ut_d(uint core_null = 0); + for (; i < oindex.n_fields; i++) { + DBUG_ASSERT(j <= i); + dict_field_t&f = fields[i] = oindex.fields[i]; + if (f.col->is_dropped()) { + /* The column has been instantly + dropped earlier. */ + DBUG_ASSERT(f.col >= old.instant->dropped); + { + size_t d = f.col + - old.instant->dropped; + DBUG_ASSERT(d < n_old_drop); + DBUG_ASSERT(&old.instant->dropped[d] + == f.col); + DBUG_ASSERT(!f.name); + f.col = instant->dropped + d; + } + if (f.col->is_nullable()) { +found_nullable: + n_nullable++; + ut_d(core_null + += i < oindex.n_core_fields); + } + continue; + } + + const ulint col_ind = col_map[f.col->ind]; + if (col_ind != ULINT_UNDEFINED) { + if (index.fields[j].col->ind != col_ind) { + /* The fields for instantly + added columns must be placed + last in the clustered index. + Keep pre-existing fields in + the same position. */ + uint k; + for (k = j + 1; k < index.n_fields; + k++) { + if (index.fields[k].col->ind + == col_ind) { + goto found_j; + } + } + DBUG_ASSERT(!"no such col"); +found_j: + std::swap(index.fields[j], + index.fields[k]); + } + DBUG_ASSERT(index.fields[j].col->ind + == col_ind); + fields[i] = index.fields[j++]; + DBUG_ASSERT(!fields[i].col->is_dropped()); + DBUG_ASSERT(fields[i].name + == fields[i].col->name(*this)); + if (fields[i].col->is_nullable()) { + goto found_nullable; + } + continue; + } + + /* This column is being dropped. */ + DBUG_ASSERT(d < n_drop); + f.col = &instant->dropped[d++]; + f.name = NULL; + if (f.col->is_nullable()) { + goto found_nullable; + } + } + ut_ad(UT_BITS_IN_BYTES(core_null) == oindex.n_core_null_bytes); + DBUG_ASSERT(i >= oindex.n_core_fields); + DBUG_ASSERT(j <= i); + DBUG_ASSERT(n_fields - (i - j) == index.n_fields); + std::sort(index.fields + j, index.fields + index.n_fields, + [](const dict_field_t& a, const dict_field_t& b) + { return a.col->ind < b.col->ind; }); + DBUG_ASSERT(d == n_drop); + for (; i < n_fields; i++) { + fields[i] = index.fields[j++]; + n_nullable += fields[i].col->is_nullable(); + DBUG_ASSERT(!fields[i].col->is_dropped()); + DBUG_ASSERT(fields[i].name + == fields[i].col->name(*this)); + } + DBUG_ASSERT(j == index.n_fields); + index.n_fields = index.n_def = n_fields; + index.fields = fields; + DBUG_ASSERT(n_nullable >= index.n_nullable); + DBUG_ASSERT(n_nullable >= oindex.n_nullable); + index.n_nullable = n_nullable; + goto set_core_fields; + } + + DBUG_ASSERT(n_cols + n_dropped() >= old.n_cols + old.n_dropped()); + DBUG_ASSERT(n_dropped() >= old.n_dropped()); + DBUG_ASSERT(index.n_core_fields == oindex.n_core_fields); + DBUG_ASSERT(index.n_core_null_bytes == oindex.n_core_null_bytes); +} + + +/** Adjust index metadata for instant ADD/DROP/reorder COLUMN. +@param[in] clustered index definition after instant ALTER TABLE */ +inline void dict_index_t::instant_add_field(const dict_index_t& instant) +{ + DBUG_ASSERT(is_primary()); + DBUG_ASSERT(instant.is_primary()); + DBUG_ASSERT(!has_virtual()); + DBUG_ASSERT(!instant.has_virtual()); + DBUG_ASSERT(instant.n_core_fields <= instant.n_fields); + DBUG_ASSERT(n_def == n_fields); + DBUG_ASSERT(instant.n_def == instant.n_fields); + DBUG_ASSERT(type == instant.type); + DBUG_ASSERT(trx_id_offset == instant.trx_id_offset); + DBUG_ASSERT(n_user_defined_cols == instant.n_user_defined_cols); + DBUG_ASSERT(n_uniq == instant.n_uniq); + DBUG_ASSERT(instant.n_fields >= n_fields); + DBUG_ASSERT(instant.n_nullable >= n_nullable); + DBUG_ASSERT(instant.n_core_fields == n_core_fields); + DBUG_ASSERT(instant.n_core_null_bytes == n_core_null_bytes); + + /* instant will have all fields (including ones for columns + that have been or are being instantly dropped) in the same position + as this index. Fields for any added columns are appended at the end. */ +#ifndef DBUG_OFF + for (unsigned i = 0; i < n_fields; i++) { + DBUG_ASSERT(fields[i].same(instant.fields[i])); + DBUG_ASSERT(fields[i].col->is_nullable() + == instant.fields[i].col->is_nullable()); + } +#endif + n_fields = instant.n_fields; + n_def = instant.n_def; + n_nullable = instant.n_nullable; + fields = static_cast( + mem_heap_dup(heap, instant.fields, n_fields * sizeof *fields)); + + ut_d(unsigned n_null = 0); + ut_d(unsigned n_dropped = 0); + + for (unsigned i = 0; i < n_fields; i++) { + const dict_col_t* icol = instant.fields[i].col; + dict_field_t& f = fields[i]; + ut_d(n_null += icol->is_nullable()); + DBUG_ASSERT(!icol->is_virtual()); + if (icol->is_dropped()) { + ut_d(n_dropped++); + f.col->set_dropped(); + f.name = NULL; + } else { + f.col = &table->cols[icol - instant.table->cols]; + f.name = f.col->name(*table); + } + } + + ut_ad(n_null == n_nullable); + ut_ad(n_dropped == instant.table->n_dropped()); +} + +/** Adjust table metadata for instant ADD/DROP/reorder COLUMN. +@param[in] table altered table (with dropped columns) +@param[in] col_map mapping from cols[] and v_cols[] to table */ +inline void dict_table_t::instant_column(const dict_table_t& table, + const ulint* col_map) +{ + DBUG_ASSERT(!table.cached); + DBUG_ASSERT(table.n_def == table.n_cols); + DBUG_ASSERT(table.n_t_def == table.n_t_cols); + DBUG_ASSERT(n_def == n_cols); + DBUG_ASSERT(n_t_def == n_t_cols); + DBUG_ASSERT(n_v_def == n_v_cols); + DBUG_ASSERT(table.n_v_def == table.n_v_cols); + DBUG_ASSERT(table.n_cols + table.n_dropped() >= n_cols + n_dropped()); + ut_ad(mutex_own(&dict_sys->mutex)); + + { + const char* end = table.col_names; + for (unsigned i = table.n_cols; i--; ) end += strlen(end) + 1; + + col_names = static_cast( + mem_heap_dup(heap, table.col_names, + ulint(end - table.col_names))); + } + const dict_col_t* const old_cols = cols; + cols = static_cast(mem_heap_dup(heap, table.cols, + table.n_cols + * sizeof *cols)); + + /* Preserve the default values of previously instantly added + columns, or copy the new default values to this->heap. */ + for (ulint i = 0; i < ulint(table.n_cols); i++) { + dict_col_t& c = cols[i]; + + if (const dict_col_t* o = find(old_cols, col_map, n_cols, i)) { + c.def_val = o->def_val; + continue; + } + + DBUG_ASSERT(c.is_added()); + if (c.def_val.len <= sizeof field_ref_zero + && !memcmp(c.def_val.data, field_ref_zero, + c.def_val.len)) { + c.def_val.data = field_ref_zero; + } else if (const void*& d = c.def_val.data) { + d = mem_heap_dup(heap, d, c.def_val.len); + } else { + DBUG_ASSERT(c.def_val.len == UNIV_SQL_NULL); + } + } + + n_t_def += table.n_cols - n_cols; + n_t_cols += table.n_cols - n_cols; + n_def = table.n_cols; + + const dict_v_col_t* const old_v_cols = v_cols; + + if (const char* end = table.v_col_names) { + for (unsigned i = table.n_v_cols; i--; ) { + end += strlen(end) + 1; + } + + v_col_names = static_cast( + mem_heap_dup(heap, table.v_col_names, + ulint(end - table.v_col_names))); + v_cols = static_cast( + mem_heap_dup(heap, table.v_cols, + table.n_v_cols * sizeof *v_cols)); + } else { + ut_ad(table.n_v_cols == 0); + v_col_names = NULL; + v_cols = NULL; + } + + n_t_def += table.n_v_cols - n_v_cols; + n_t_cols += table.n_v_cols - n_v_cols; + n_v_def = table.n_v_cols; + + for (unsigned i = 0; i < n_v_def; i++) { + dict_v_col_t& v = v_cols[i]; + v.v_indexes = UT_NEW_NOKEY(dict_v_idx_list()); + v.base_col = static_cast( + mem_heap_dup(heap, v.base_col, + v.num_base * sizeof *v.base_col)); + + for (ulint n = v.num_base; n--; ) { + dict_col_t*& base = v.base_col[n]; + if (base->is_virtual()) { + } else if (base >= table.cols + && base < table.cols + table.n_cols) { + /* The base column was instantly added. */ + size_t c = base - table.cols; + DBUG_ASSERT(base == &table.cols[c]); + base = &cols[c]; + } else { + DBUG_ASSERT(base >= old_cols); + size_t c = base - old_cols; + DBUG_ASSERT(c + DATA_N_SYS_COLS < n_cols); + DBUG_ASSERT(base == &old_cols[c]); + DBUG_ASSERT(col_map[c] + DATA_N_SYS_COLS + < n_cols); + base = &cols[col_map[c]]; + } + } + } + + dict_index_t* index = dict_table_get_first_index(this); + + index->instant_add_field(*dict_table_get_first_index(&table)); + + if (instant || table.instant) { + const unsigned u = index->first_user_field(); + unsigned* non_pk_col_map = static_cast( + mem_heap_alloc(heap, (index->n_fields - u) + * sizeof *non_pk_col_map)); + /* FIXME: add instant->heap, and transfer ownership here */ + if (!instant) { + instant = new (mem_heap_zalloc(heap, sizeof *instant)) + dict_instant_t(); + goto dup_dropped; + } else if (n_dropped() < table.n_dropped()) { +dup_dropped: + instant->dropped = static_cast( + mem_heap_dup(heap, table.instant->dropped, + table.instant->n_dropped + * sizeof *instant->dropped)); + instant->n_dropped = table.instant->n_dropped; + } else if (table.instant->n_dropped) { + memcpy(instant->dropped, table.instant->dropped, + table.instant->n_dropped + * sizeof *instant->dropped); + } + + instant->non_pk_col_map = non_pk_col_map; + ut_d(unsigned n_drop = 0); + for (unsigned i = u; i < index->n_fields; i++) { + dict_field_t* field = &index->fields[i]; + DBUG_ASSERT(dict_col_get_fixed_size( + field->col, + flags & DICT_TF_COMPACT) + <= DICT_MAX_FIXED_COL_LEN); + if (!field->col->is_dropped()) { + *non_pk_col_map++ = field->col->ind; + continue; + } + + ulint fixed_len = dict_col_get_fixed_size( + field->col, flags & DICT_TF_COMPACT); + *non_pk_col_map++ = 1U << 15 + | unsigned(!field->col->is_nullable()) << 14 + | (fixed_len + ? unsigned(fixed_len + 1) + : field->col->len > 255); + ut_ad(field->col >= table.instant->dropped); + ut_ad(field->col < table.instant->dropped + + table.instant->n_dropped); + ut_d(n_drop++); + size_t d = field->col - table.instant->dropped; + ut_ad(field->col == &table.instant->dropped[d]); + ut_ad(d <= instant->n_dropped); + field->col = &instant->dropped[d]; + } + ut_ad(n_drop == n_dropped()); + ut_ad(non_pk_col_map + == &instant->non_pk_col_map[index->n_fields - u]); + } + + while ((index = dict_table_get_next_index(index)) != NULL) { + if (index->to_be_dropped) { + continue; + } + for (unsigned i = 0; i < index->n_fields; i++) { + dict_field_t& f = index->fields[i]; + if (f.col >= table.cols + && f.col < table.cols + table.n_cols) { + /* This is an instantly added column + in a newly added index. */ + DBUG_ASSERT(!f.col->is_virtual()); + size_t c = f.col - table.cols; + DBUG_ASSERT(f.col == &table.cols[c]); + f.col = &cols[c]; + } else if (f.col >= &table.v_cols->m_col + && f.col < &table.v_cols[n_v_cols].m_col) { + /* This is an instantly added virtual column + in a newly added index. */ + DBUG_ASSERT(f.col->is_virtual()); + size_t c = reinterpret_cast( + f.col) - table.v_cols; + DBUG_ASSERT(f.col == &table.v_cols[c].m_col); + f.col = &v_cols[c].m_col; + } else if (f.col < old_cols + || f.col >= old_cols + n_cols) { + DBUG_ASSERT(f.col->is_virtual()); + f.col = &v_cols[col_map[ + reinterpret_cast( + f.col) + - old_v_cols + n_cols]].m_col; + } else { + f.col = &cols[col_map[f.col - old_cols]]; + DBUG_ASSERT(!f.col->is_virtual()); + } + f.name = f.col->name(*this); + if (f.col->is_virtual()) { + reinterpret_cast(f.col) + ->v_indexes->push_back( + dict_v_idx_t(index, i)); + } + } + } + + n_cols = table.n_cols; + n_v_cols = table.n_v_cols; +} + +/** Find the old column number for the given new column position. +@param[in] col_map column map from old column to new column +@param[in] pos new column position +@param[in] n number of columns present in the column map +@return old column position for the given new column position. */ +static ulint find_old_col_no(const ulint* col_map, ulint pos, ulint n) +{ + do { + ut_ad(n); + } while (col_map[--n] != pos); + return n; +} + +/** Roll back instant_column(). +@param[in] old_n_cols original n_cols +@param[in] old_cols original cols +@param[in] old_col_names original col_names +@param[in] old_instant original instant structure +@param[in] old_fields original fields +@param[in] old_n_fields original number of fields +@param[in] old_n_v_cols original n_v_cols +@param[in] old_v_cols original v_cols +@param[in] old_v_col_names original v_col_names +@param[in] col_map column map */ +inline void dict_table_t::rollback_instant( + unsigned old_n_cols, + dict_col_t* old_cols, + const char* old_col_names, + dict_instant_t* old_instant, + dict_field_t* old_fields, + unsigned old_n_fields, + unsigned old_n_v_cols, + dict_v_col_t* old_v_cols, + const char* old_v_col_names, + const ulint* col_map) +{ + ut_ad(mutex_own(&dict_sys->mutex)); + dict_index_t* index = indexes.start; + /* index->is_instant() does not necessarily hold here, because + the table may have been emptied */ + DBUG_ASSERT(old_n_cols >= DATA_N_SYS_COLS); + DBUG_ASSERT(n_cols == n_def); + DBUG_ASSERT(index->n_def == index->n_fields); + DBUG_ASSERT(index->n_core_fields <= old_n_fields); + DBUG_ASSERT(index->n_core_fields <= index->n_fields); + DBUG_ASSERT(instant || !old_instant); + + instant = old_instant; + + index->n_nullable = 0; + + for (unsigned i = old_n_fields; i--; ) { + if (old_fields[i].col->is_nullable()) { + index->n_nullable++; + } + } + + for (unsigned i = n_v_cols; i--; ) { + UT_DELETE(v_cols[i].v_indexes); + } + + index->n_def = index->n_fields = old_n_fields; + + const dict_col_t* const new_cols = cols; + const dict_col_t* const new_cols_end = cols + n_cols; + const dict_v_col_t* const new_v_cols = v_cols; + const dict_v_col_t* const new_v_cols_end = v_cols + n_v_cols; + + cols = old_cols; + col_names = old_col_names; + v_cols = old_v_cols; + v_col_names = old_v_col_names; + n_def = n_cols = old_n_cols; + n_v_def = n_v_cols = old_n_v_cols; + n_t_def = n_t_cols = n_cols + n_v_cols; + + index->fields = old_fields; + + while ((index = dict_table_get_next_index(index)) != NULL) { + if (index->to_be_dropped) { + /* instant_column() did not adjust these indexes. */ + continue; + } + + for (unsigned i = 0; i < index->n_fields; i++) { + dict_field_t& f = index->fields[i]; + if (f.col->is_virtual()) { + DBUG_ASSERT(f.col >= &new_v_cols->m_col); + DBUG_ASSERT(f.col < &new_v_cols_end->m_col); + size_t n = size_t( + reinterpret_cast(f.col) + - new_v_cols); + DBUG_ASSERT(n <= n_v_cols); + + ulint old_col_no = find_old_col_no( + col_map + n_cols, n, n_v_cols); + DBUG_ASSERT(old_col_no <= n_v_cols); + f.col = &v_cols[old_col_no].m_col; + DBUG_ASSERT(f.col->is_virtual()); + } else { + DBUG_ASSERT(f.col >= new_cols); + DBUG_ASSERT(f.col < new_cols_end); + size_t n = size_t(f.col - new_cols); + DBUG_ASSERT(n <= n_cols); + + ulint old_col_no = find_old_col_no(col_map, + n, n_cols); + DBUG_ASSERT(old_col_no < n_cols); + f.col = &cols[old_col_no]; + DBUG_ASSERT(!f.col->is_virtual()); + } + f.name = f.col->name(*this); + } + } +} + struct ha_innobase_inplace_ctx : public inplace_alter_handler_ctx { /** Dummy query graph */ @@ -171,7 +738,7 @@ struct ha_innobase_inplace_ctx : public inplace_alter_handler_ctx dict_table_t* old_table; /** table where the indexes are being created or dropped */ dict_table_t* new_table; - /** table definition for instant ADD COLUMN */ + /** table definition for instant ADD/DROP/reorder COLUMN */ dict_table_t* instant_table; /** mapping of old column numbers to new ones, or NULL */ const ulint* col_map; @@ -205,7 +772,20 @@ struct ha_innobase_inplace_ctx : public inplace_alter_handler_ctx dict_col_t* const old_cols; /** original column names of the table */ const char* const old_col_names; - + /** original instantly dropped or reordered columns */ + dict_instant_t* const old_instant; + /** original index fields */ + dict_field_t* const old_fields; + /** size of old_fields */ + const unsigned old_n_fields; + /** original number of virtual columns in the table */ + const unsigned old_n_v_cols; + /** original virtual columns of the table */ + dict_v_col_t* const old_v_cols; + /** original virtual column names of the table */ + const char* const old_v_col_names; + /** 0, or 1 + first column whose position changes in instant ALTER */ + unsigned first_alter_pos; /** Allow non-null conversion. (1) Alter ignore should allow the conversion irrespective of sql mode. @@ -262,6 +842,13 @@ struct ha_innobase_inplace_ctx : public inplace_alter_handler_ctx old_n_cols(prebuilt_arg->table->n_cols), old_cols(prebuilt_arg->table->cols), old_col_names(prebuilt_arg->table->col_names), + old_instant(prebuilt_arg->table->instant), + old_fields(prebuilt_arg->table->indexes.start->fields), + old_n_fields(prebuilt_arg->table->indexes.start->n_fields), + old_n_v_cols(prebuilt_arg->table->n_v_cols), + old_v_cols(prebuilt_arg->table->v_cols), + old_v_col_names(prebuilt_arg->table->v_col_names), + first_alter_pos(0), allow_not_null(allow_not_null_flag), page_compression_level(page_compressed ? (page_compression_level_arg @@ -294,6 +881,9 @@ struct ha_innobase_inplace_ctx : public inplace_alter_handler_ctx rw_lock_free(&index->lock); dict_mem_index_free(index); } + for (unsigned i = old_n_v_cols; i--; ) { + UT_DELETE(old_v_cols[i].v_indexes); + } dict_mem_table_free(instant_table); } mem_heap_free(heap); @@ -318,14 +908,14 @@ struct ha_innobase_inplace_ctx : public inplace_alter_handler_ctx { DBUG_ASSERT(need_rebuild()); DBUG_ASSERT(!is_instant()); - DBUG_ASSERT(old_table->n_cols == old_table->n_def); - DBUG_ASSERT(new_table->n_cols == new_table->n_def); DBUG_ASSERT(old_table->n_cols == old_n_cols); - DBUG_ASSERT(new_table->n_cols > old_table->n_cols); - instant_table = new_table; + instant_table = new_table; new_table = old_table; export_vars.innodb_instant_alter_column++; + + instant_table->prepare_instant(*old_table, col_map, + first_alter_pos); } /** Revert prepare_instant() if the transaction is rolled back. */ @@ -333,7 +923,12 @@ struct ha_innobase_inplace_ctx : public inplace_alter_handler_ctx { if (!is_instant()) return; old_table->rollback_instant(old_n_cols, - old_cols, old_col_names); + old_cols, old_col_names, + old_instant, + old_fields, old_n_fields, + old_n_v_cols, old_v_cols, + old_v_col_names, + col_map); } /** @return whether this is instant ALTER TABLE */ @@ -663,20 +1258,56 @@ check_v_col_in_order( } /** Determine if an instant operation is possible for altering columns. +@param[in] ib_table InnoDB table definition @param[in] ha_alter_info the ALTER TABLE operation @param[in] table table definition before ALTER TABLE */ static bool instant_alter_column_possible( + const dict_table_t& ib_table, const Alter_inplace_info* ha_alter_info, const TABLE* table) { + if (!ib_table.supports_instant()) { + return false; + } +#if 1 // MDEV-17459: adjust fts_fetch_doc_from_rec() and friends; remove this + if (ib_table.fts) { + return false; + } +#endif + const dict_index_t* index = ib_table.indexes.start; + if (ha_alter_info->handler_flags & ALTER_ADD_STORED_BASE_COLUMN) { + List_iterator_fast cf_it( + ha_alter_info->alter_info->create_list); + uint n_add = 0; + while (const Create_field* cf = cf_it++) { + n_add += !cf->field; + } + if (index->n_fields >= REC_MAX_N_USER_FIELDS + DATA_N_SYS_COLS + - n_add) { + return false; + } + } +#if 1 // MDEV-17468: fix bugs with indexed virtual columns & remove this + ut_ad(index->is_primary()); + ut_ad(!index->has_virtual()); + while ((index = index->indexes.next) != NULL) { + if (index->has_virtual()) { + ut_ad(ib_table.n_v_cols); + return false; + } + } +#endif // Making table system-versioned instantly is not implemented yet. if (ha_alter_info->handler_flags & ALTER_ADD_SYSTEM_VERSIONING) { return false; } - if (~ha_alter_info->handler_flags & ALTER_ADD_STORED_BASE_COLUMN) { + if (!(ha_alter_info->handler_flags + & (ALTER_ADD_STORED_BASE_COLUMN + | ALTER_DROP_STORED_COLUMN + | ALTER_STORED_COLUMN_ORDER))) { return false; } @@ -699,6 +1330,8 @@ instant_alter_column_possible( columns. */ if (ha_alter_info->handler_flags & ((INNOBASE_ALTER_REBUILD | INNOBASE_ONLINE_CREATE) + & ~ALTER_DROP_STORED_COLUMN + & ~ALTER_STORED_COLUMN_ORDER & ~ALTER_ADD_STORED_BASE_COLUMN & ~ALTER_OPTIONS)) { return false; } @@ -1013,6 +1646,8 @@ ha_innobase::check_if_supported_inplace_alter( DBUG_RETURN(HA_ALTER_INPLACE_NOT_SUPPORTED); } + const bool supports_instant = instant_alter_column_possible( + *m_prebuilt->table, ha_alter_info, table); bool add_drop_v_cols = false; /* If there is add or drop virtual columns, we will support operations @@ -1040,7 +1675,13 @@ ha_innobase::check_if_supported_inplace_alter( */ | ALTER_ADD_NON_UNIQUE_NON_PRIM_INDEX | ALTER_DROP_NON_UNIQUE_NON_PRIM_INDEX); - + if (supports_instant) { + flags &= ~(ALTER_DROP_STORED_COLUMN +#if 0 /* MDEV-17468: remove check_v_col_in_order() and fix the code */ + | ALTER_ADD_STORED_BASE_COLUMN +#endif + | ALTER_STORED_COLUMN_ORDER); + } if (flags != 0 || IF_PARTITIONING((altered_table->s->partition_info_str && altered_table->s->partition_info_str_len), 0) @@ -1221,8 +1862,7 @@ ha_innobase::check_if_supported_inplace_alter( constant DEFAULT expression. */ cf_it.rewind(); Field **af = altered_table->field; - bool add_column_not_last = false; - uint n_stored_cols = 0, n_add_cols = 0; + bool fts_need_rebuild = false; while (Create_field* cf = cf_it++) { DBUG_ASSERT(cf->field @@ -1270,44 +1910,36 @@ ha_innobase::check_if_supported_inplace_alter( goto next_column; } - ha_alter_info->unsupported_reason - = my_get_err_msg( - ER_ALTER_OPERATION_NOT_SUPPORTED_REASON_NOT_NULL); - } else if (!is_non_const_value(*af)) { - - n_add_cols++; - - if (af < &altered_table->field[table_share->fields]) { - add_column_not_last = true; - } - - if (set_default_value(*af)) { - goto next_column; + ha_alter_info->unsupported_reason = my_get_err_msg( + ER_ALTER_OPERATION_NOT_SUPPORTED_REASON_NOT_NULL); + } else if (!is_non_const_value(*af) + && set_default_value(*af)) { + if (m_prebuilt->table->fts + && innobase_fulltext_exist(altered_table) + && !my_strcasecmp(system_charset_info, + (*af)->field_name.str, + FTS_DOC_ID_COL_NAME)) { + /* If a hidden FTS_DOC_ID column exists + (because of FULLTEXT INDEX), it cannot + be replaced with a user-created one + except when using ALGORITHM=COPY. */ + goto cannot_create_many_fulltext_index; } + goto next_column; } DBUG_RETURN(HA_ALTER_INPLACE_NOT_SUPPORTED); next_column: - n_stored_cols += (*af++)->stored_in_db(); - } - - if (!add_column_not_last - && uint(m_prebuilt->table->n_cols) - DATA_N_SYS_COLS + n_add_cols - == n_stored_cols - && m_prebuilt->table->supports_instant() - && instant_alter_column_possible(ha_alter_info, table)) { - - DBUG_RETURN(HA_ALTER_INPLACE_INSTANT); + af++; } - if (!(ha_alter_info->handler_flags & ~(INNOBASE_ALTER_INSTANT - | INNOBASE_INPLACE_IGNORE))) { + if (supports_instant + || !(ha_alter_info->handler_flags + & ~(INNOBASE_ALTER_INSTANT | INNOBASE_INPLACE_IGNORE))) { DBUG_RETURN(HA_ALTER_INPLACE_INSTANT); } - bool fts_need_rebuild = false; - if (!online) { /* We already determined that only a non-locking operation is possible. */ @@ -3370,12 +4002,13 @@ innobase_build_col_map( } } - ut_ad(!is_v); - innobase_build_col_map_add( - heap, dtuple_get_nth_field(defaults, i), - altered_table->field[i + num_v], - NULL, - dict_table_is_comp(new_table)); + if (!is_v) { + innobase_build_col_map_add( + heap, dtuple_get_nth_field(defaults, i), + altered_table->field[i + num_v], + NULL, + dict_table_is_comp(new_table)); + } found_col: if (is_v) { num_v++; @@ -3843,13 +4476,12 @@ prepare_inplace_add_virtual( ha_innobase_inplace_ctx* ctx; ulint i = 0; ulint j = 0; - const Create_field* new_field; ctx = static_cast (ha_alter_info->handler_ctx); - ctx->num_to_add_vcol = altered_table->s->fields - + ctx->num_to_drop_vcol - table->s->fields; + ctx->num_to_add_vcol = altered_table->s->virtual_fields + + ctx->num_to_drop_vcol - table->s->virtual_fields; ctx->add_vcol = static_cast( mem_heap_zalloc(ctx->heap, ctx->num_to_add_vcol @@ -3861,43 +4493,21 @@ prepare_inplace_add_virtual( List_iterator_fast cf_it( ha_alter_info->alter_info->create_list); - while ((new_field = (cf_it++)) != NULL) { - const Field* field = new_field->field; - ulint old_i; - - for (old_i = 0; table->field[old_i]; old_i++) { - const Field* n_field = table->field[old_i]; - if (field == n_field) { - break; - } - } - - i++; + while (const Create_field* new_field = cf_it++) { + const Field* field = altered_table->field[i++]; - if (table->field[old_i]) { + if (new_field->field || !innobase_is_v_fld(field)) { continue; } - ut_ad(!field); - - ulint col_len; ulint is_unsigned; - ulint field_type; ulint charset_no; - - field = altered_table->field[i - 1]; - ulint col_type = get_innobase_type_from_mysql_type( &is_unsigned, field); - - if (!innobase_is_v_fld(field)) { - continue; - } - - col_len = field->pack_length(); - field_type = (ulint) field->type(); + ulint col_len = field->pack_length(); + ulint field_type = (ulint) field->type(); if (!field->real_maybe_null()) { field_type |= DATA_NOT_NULL; @@ -3939,7 +4549,6 @@ prepare_inplace_add_virtual( } } - ctx->add_vcol[j].m_col.prtype = dtype_form_prtype( field_type, charset_no); @@ -3958,6 +4567,7 @@ prepare_inplace_add_virtual( /* No need to track the list */ ctx->add_vcol[j].v_indexes = NULL; + /* MDEV-17468: Do this on ctx->instant_table later */ innodb_base_col_setup(ctx->old_table, field, &ctx->add_vcol[j]); j++; } @@ -4084,72 +4694,308 @@ prepare_inplace_drop_virtual( @param[in] pos virtual column column no @param[in] base_pos base column pos @param[in] trx transaction -@return DB_SUCCESS if successful, otherwise error code */ -static -dberr_t -innobase_insert_sys_virtual( +@retval false on success +@retval true on failure (my_error() will have been called) */ +static bool innobase_insert_sys_virtual( const dict_table_t* table, ulint pos, ulint base_pos, trx_t* trx) +{ + pars_info_t* info = pars_info_create(); + pars_info_add_ull_literal(info, "id", table->id); + pars_info_add_int4_literal(info, "pos", pos); + pars_info_add_int4_literal(info, "base_pos", base_pos); + + if (DB_SUCCESS != que_eval_sql( + info, + "PROCEDURE P () IS\n" + "BEGIN\n" + "INSERT INTO SYS_VIRTUAL VALUES (:id, :pos, :base_pos);\n" + "END;\n", + FALSE, trx)) { + my_error(ER_INTERNAL_ERROR, MYF(0), + "InnoDB: ADD COLUMN...VIRTUAL"); + return true; + } + + return false; +} + +/** Insert a record to the SYS_COLUMNS dictionary table. +@param[in] table_id table id +@param[in] pos position of the column +@param[in] field_name field name +@param[in] mtype main type +@param[in] prtype precise type +@param[in] len fixed length in bytes, or 0 +@param[in] n_base number of base columns of virtual columns, or 0 +@retval false on success +@retval true on failure (my_error() will have been called) */ +static bool innodb_insert_sys_columns( + table_id_t table_id, + ulint pos, + const char* field_name, + ulint mtype, + ulint prtype, + ulint len, + ulint n_base, + trx_t* trx) +{ + pars_info_t* info = pars_info_create(); + pars_info_add_ull_literal(info, "id", table_id); + pars_info_add_int4_literal(info, "pos", pos); + pars_info_add_str_literal(info, "name", field_name); + pars_info_add_int4_literal(info, "mtype", mtype); + pars_info_add_int4_literal(info, "prtype", prtype); + pars_info_add_int4_literal(info, "len", len); + pars_info_add_int4_literal(info, "base", n_base); + + if (DB_SUCCESS != que_eval_sql( + info, + "PROCEDURE ADD_COL () IS\n" + "BEGIN\n" + "INSERT INTO SYS_COLUMNS VALUES" + "(:id,:pos,:name,:mtype,:prtype,:len,:base);\n" + "END;\n", FALSE, trx)) { + my_error(ER_INTERNAL_ERROR, MYF(0), + "InnoDB: Insert into SYS_COLUMNS failed"); + return true; + } + + return false; +} + +/** Update INNODB SYS_COLUMNS on new virtual columns +@param[in] table InnoDB table +@param[in] col_name column name +@param[in] vcol virtual column +@param[in] trx transaction +@retval false on success +@retval true on failure (my_error() will have been called) */ +static bool innobase_add_one_virtual( + const dict_table_t* table, + const char* col_name, + dict_v_col_t* vcol, + trx_t* trx) +{ + ulint pos = dict_create_v_col_pos(vcol->v_pos, + vcol->m_col.ind); + + if (innodb_insert_sys_columns(table->id, pos, col_name, + vcol->m_col.mtype, vcol->m_col.prtype, + vcol->m_col.len, vcol->num_base, trx)) { + return true; + } + + for (ulint i = 0; i < vcol->num_base; i++) { + if (innobase_insert_sys_virtual( + table, pos, vcol->base_col[i]->ind, trx)) { + return true; + } + } + + return false; +} + +/** Update SYS_TABLES.N_COLS in the data dictionary. +@param[in] user_table InnoDB table +@param[in] n the new value of SYS_TABLES.N_COLS +@param[in] trx transaction +@return whether the operation failed */ +static bool innodb_update_cols(const dict_table_t* table, ulint n, trx_t* trx) { pars_info_t* info = pars_info_create(); + pars_info_add_int4_literal(info, "n", n); pars_info_add_ull_literal(info, "id", table->id); + if (DB_SUCCESS != que_eval_sql(info, + "PROCEDURE UPDATE_N_COLS () IS\n" + "BEGIN\n" + "UPDATE SYS_TABLES SET N_COLS = :n" + " WHERE ID = :id;\n" + "END;\n", FALSE, trx)) { + my_error(ER_INTERNAL_ERROR, MYF(0), + "InnoDB: Updating SYS_TABLES.N_COLS failed"); + return true; + } + + return false; +} + +/** Update system table for adding virtual column(s) +@param[in] ha_alter_info Data used during in-place alter +@param[in] user_table InnoDB table +@param[in] trx transaction +@retval true Failure +@retval false Success */ +static +bool +innobase_add_virtual_try( + const Alter_inplace_info* ha_alter_info, + const dict_table_t* user_table, + trx_t* trx) +{ + ha_innobase_inplace_ctx* ctx = static_cast( + ha_alter_info->handler_ctx); + + for (ulint i = 0; i < ctx->num_to_add_vcol; i++) { + if (innobase_add_one_virtual( + user_table, ctx->add_vcol_name[i], + &ctx->add_vcol[i], trx)) { + return true; + } + } + + return false; +} + +/** Add the newly added column in the sys_column system table. +@param[in] table_id table id +@param[in] pos position of the column +@param[in] field_name field name +@param[in] type data type +@retval true Failure +@retval false Success. */ +static bool innobase_instant_add_col( + table_id_t table_id, + ulint pos, + const char* field_name, + const dtype_t& type, + trx_t* trx) +{ + return innodb_insert_sys_columns(table_id, pos, field_name, + type.mtype, type.prtype, type.len, 0, + trx); +} + +/** Delete metadata from SYS_COLUMNS and SYS_VIRTUAL. +@param[in] id table id +@param[in] pos first SYS_COLUMNS.POS +@param[in,out] trx data dictionary transaction +@retval true Failure +@retval false Success. */ +static bool innobase_instant_drop_cols(table_id_t id, ulint pos, trx_t* trx) +{ + pars_info_t* info = pars_info_create(); + pars_info_add_ull_literal(info, "id", id); pars_info_add_int4_literal(info, "pos", pos); - pars_info_add_int4_literal(info, "base_pos", base_pos); + dberr_t err = que_eval_sql( + info, + "PROCEDURE DELETE_COL () IS\n" + "BEGIN\n" + "DELETE FROM SYS_COLUMNS WHERE\n" + "TABLE_ID = :id AND POS >= :pos;\n" + "DELETE FROM SYS_VIRTUAL WHERE TABLE_ID = :id;\n" + "END;\n", FALSE, trx); + if (err != DB_SUCCESS) { + my_error(ER_INTERNAL_ERROR, MYF(0), + "InnoDB: DELETE from SYS_COLUMNS/SYS_VIRTUAL failed"); + return true; + } + + return false; +} + +/** Update INNODB SYS_COLUMNS on new virtual column's position +@param[in] table InnoDB table +@param[in] old_pos old position +@param[in] new_pos new position +@param[in] trx transaction +@return DB_SUCCESS if successful, otherwise error code */ +static +dberr_t +innobase_update_v_pos_sys_columns( + const dict_table_t* table, + ulint old_pos, + ulint new_pos, + trx_t* trx) +{ + pars_info_t* info = pars_info_create(); + + pars_info_add_int4_literal(info, "pos", old_pos); + pars_info_add_int4_literal(info, "val", new_pos); + pars_info_add_ull_literal(info, "id", table->id); dberr_t error = que_eval_sql( info, "PROCEDURE P () IS\n" "BEGIN\n" - "INSERT INTO SYS_VIRTUAL VALUES" - "(:id, :pos, :base_pos);\n" + "UPDATE SYS_COLUMNS\n" + "SET POS = :val\n" + "WHERE POS = :pos\n" + "AND TABLE_ID = :id;\n" "END;\n", FALSE, trx); return(error); } -/** Update INNODB SYS_COLUMNS on new virtual columns -@param[in] table InnoDB table -@param[in] col_name column name -@param[in] vcol virtual column -@param[in] trx transaction +/** Update INNODB SYS_VIRTUAL table with new virtual column position +@param[in] table InnoDB table +@param[in] old_pos old position +@param[in] new_pos new position +@param[in] trx transaction @return DB_SUCCESS if successful, otherwise error code */ static dberr_t -innobase_add_one_virtual( +innobase_update_v_pos_sys_virtual( const dict_table_t* table, - const char* col_name, - dict_v_col_t* vcol, + ulint old_pos, + ulint new_pos, trx_t* trx) { - ulint pos = dict_create_v_col_pos(vcol->v_pos, - vcol->m_col.ind); - ulint mtype = vcol->m_col.mtype; - ulint prtype = vcol->m_col.prtype; - ulint len = vcol->m_col.len; pars_info_t* info = pars_info_create(); + pars_info_add_int4_literal(info, "pos", old_pos); + pars_info_add_int4_literal(info, "val", new_pos); pars_info_add_ull_literal(info, "id", table->id); - pars_info_add_int4_literal(info, "pos", pos); + dberr_t error = que_eval_sql( + info, + "PROCEDURE P () IS\n" + "BEGIN\n" + "UPDATE SYS_VIRTUAL\n" + "SET POS = :val\n" + "WHERE POS = :pos\n" + "AND TABLE_ID = :id;\n" + "END;\n", + FALSE, trx); + + return(error); +} + +/** Update InnoDB system tables on dropping a virtual column +@param[in] table InnoDB table +@param[in] col_name column name of the dropping column +@param[in] drop_col col information for the dropping column +@param[in] n_prev_dropped number of previously dropped columns in the + same alter clause +@param[in] trx transaction +@return DB_SUCCESS if successful, otherwise error code */ +static +dberr_t +innobase_drop_one_virtual_sys_columns( + const dict_table_t* table, + const char* col_name, + dict_col_t* drop_col, + ulint n_prev_dropped, + trx_t* trx) +{ + pars_info_t* info = pars_info_create(); + pars_info_add_ull_literal(info, "id", table->id); pars_info_add_str_literal(info, "name", col_name); - pars_info_add_int4_literal(info, "mtype", mtype); - pars_info_add_int4_literal(info, "prtype", prtype); - pars_info_add_int4_literal(info, "len", len); - pars_info_add_int4_literal(info, "prec", vcol->num_base); dberr_t error = que_eval_sql( info, "PROCEDURE P () IS\n" "BEGIN\n" - "INSERT INTO SYS_COLUMNS VALUES" - "(:id, :pos, :name, :mtype, :prtype, :len, :prec);\n" + "DELETE FROM SYS_COLUMNS\n" + "WHERE TABLE_ID = :id\n" + "AND NAME = :name;\n" "END;\n", FALSE, trx); @@ -4157,9 +5003,26 @@ innobase_add_one_virtual( return(error); } - for (ulint i = 0; i < vcol->num_base; i++) { - error = innobase_insert_sys_virtual( - table, pos, vcol->base_col[i]->ind, trx); + dict_v_col_t* v_col = dict_table_get_nth_v_col_mysql( + table, drop_col->ind); + + /* Adjust column positions for all subsequent columns */ + for (ulint i = v_col->v_pos + 1; i < table->n_v_cols; i++) { + dict_v_col_t* t_col = dict_table_get_nth_v_col(table, i); + ulint old_p = dict_create_v_col_pos( + t_col->v_pos - n_prev_dropped, + t_col->m_col.ind - n_prev_dropped); + ulint new_p = dict_create_v_col_pos( + t_col->v_pos - 1 - n_prev_dropped, + ulint(t_col->m_col.ind) - 1 - n_prev_dropped); + + error = innobase_update_v_pos_sys_columns( + table, old_p, new_p, trx); + if (error != DB_SUCCESS) { + return(error); + } + error = innobase_update_v_pos_sys_virtual( + table, old_p, new_p, trx); if (error != DB_SUCCESS) { return(error); } @@ -4168,37 +5031,37 @@ innobase_add_one_virtual( return(error); } -/** Update SYS_TABLES.N_COLS in the data dictionary. -@param[in] user_table InnoDB table -@param[in] n_cols the new value of SYS_TABLES.N_COLS -@param[in] trx transaction -@return whether the operation failed */ +/** Delete virtual column's info from INNODB SYS_VIRTUAL +@param[in] table InnoDB table +@param[in] pos position of the virtual column to be deleted +@param[in] trx transaction +@return DB_SUCCESS if successful, otherwise error code */ static -bool -innodb_update_n_cols(const dict_table_t* table, ulint n_cols, trx_t* trx) +dberr_t +innobase_drop_one_virtual_sys_virtual( + const dict_table_t* table, + ulint pos, + trx_t* trx) { pars_info_t* info = pars_info_create(); - - pars_info_add_int4_literal(info, "n", n_cols); pars_info_add_ull_literal(info, "id", table->id); - dberr_t err = que_eval_sql(info, - "PROCEDURE UPDATE_N_COLS () IS\n" - "BEGIN\n" - "UPDATE SYS_TABLES SET N_COLS = :n" - " WHERE ID = :id;\n" - "END;\n", FALSE, trx); + pars_info_add_int4_literal(info, "pos", pos); - if (err != DB_SUCCESS) { - my_error(ER_INTERNAL_ERROR, MYF(0), - "InnoDB: Updating SYS_TABLES.N_COLS failed"); - return true; - } + dberr_t error = que_eval_sql( + info, + "PROCEDURE P () IS\n" + "BEGIN\n" + "DELETE FROM SYS_VIRTUAL\n" + "WHERE TABLE_ID = :id\n" + "AND POS = :pos;\n" + "END;\n", + FALSE, trx); - return false; + return(error); } -/** Update system table for adding virtual column(s) +/** Update system table for dropping virtual column(s) @param[in] ha_alter_info Data used during in-place alter @param[in] user_table InnoDB table @param[in] trx transaction @@ -4206,87 +5069,193 @@ innodb_update_n_cols(const dict_table_t* table, ulint n_cols, trx_t* trx) @retval false Success */ static bool -innobase_add_virtual_try( - Alter_inplace_info* ha_alter_info, - const dict_table_t* user_table, - trx_t* trx) +innobase_drop_virtual_try( + const Alter_inplace_info* ha_alter_info, + const dict_table_t* user_table, + trx_t* trx) { ha_innobase_inplace_ctx* ctx; dberr_t err = DB_SUCCESS; - ctx = static_cast( - ha_alter_info->handler_ctx); + ctx = static_cast + (ha_alter_info->handler_ctx); - for (ulint i = 0; i < ctx->num_to_add_vcol; i++) { + for (ulint i = 0; i < ctx->num_to_drop_vcol; i++) { - err = innobase_add_one_virtual( - user_table, ctx->add_vcol_name[i], - &ctx->add_vcol[i], trx); + ulint pos = dict_create_v_col_pos( + ctx->drop_vcol[i].v_pos - i, + ctx->drop_vcol[i].m_col.ind - i); + err = innobase_drop_one_virtual_sys_virtual( + user_table, pos, trx); if (err != DB_SUCCESS) { my_error(ER_INTERNAL_ERROR, MYF(0), - "InnoDB: ADD COLUMN...VIRTUAL"); + "InnoDB: DROP COLUMN...VIRTUAL"); + return(true); + } + + err = innobase_drop_one_virtual_sys_columns( + user_table, ctx->drop_vcol_name[i], + &(ctx->drop_vcol[i].m_col), i, trx); + + if (err != DB_SUCCESS) { + my_error(ER_INTERNAL_ERROR, MYF(0), + "InnoDB: DROP COLUMN...VIRTUAL"); return(true); } } + return false; +} + +/** Construct the metadata record for instant ALTER TABLE. +@param[in] row dummy or default values for existing columns +@param[in,out] heap memory heap for allocations +@return metadata record */ +inline +dtuple_t* +dict_index_t::instant_metadata(const dtuple_t& row, mem_heap_t* heap) const +{ + ut_ad(is_primary()); + dtuple_t* entry; + + if (!table->instant) { + entry = row_build_index_entry(&row, NULL, this, heap); + entry->info_bits = REC_INFO_METADATA_ADD; + return entry; + } - ulint n_col = unsigned(user_table->n_cols) - DATA_N_SYS_COLS; - ulint n_v_col = unsigned(user_table->n_v_cols) - + ctx->num_to_add_vcol - ctx->num_to_drop_vcol; - ulint new_n = dict_table_encode_n_col(n_col, n_v_col) - + (unsigned(user_table->flags & DICT_TF_COMPACT) << 31); + entry = dtuple_create(heap, n_fields + 1); + entry->n_fields_cmp = n_uniq; + entry->info_bits = REC_INFO_METADATA_ALTER; - return innodb_update_n_cols(user_table, new_n, trx); + const dict_field_t* field = fields; + + for (uint i = 0; i <= n_fields; i++, field++) { + dfield_t* dfield = dtuple_get_nth_field(entry, i); + + if (i == first_user_field()) { + table->serialise_columns(heap, dfield); + dfield->type.metadata_blob_init(); + field--; + continue; + } + + ut_ad(!field->col->is_virtual()); + + if (field->col->is_dropped()) { + dict_col_copy_type(field->col, &dfield->type); + if (field->col->is_nullable()) { + dfield_set_null(dfield); + } else { + dfield_set_data(dfield, field_ref_zero, + field->fixed_len); + } + continue; + } + + const dfield_t* s = dtuple_get_nth_field(&row, field->col->ind); + ut_ad(dict_col_type_assert_equal(field->col, &s->type)); + *dfield = *s; + + if (dfield_is_null(dfield)) { + continue; + } + + if (dfield_is_ext(dfield)) { + ut_ad(i > first_user_field()); + ut_ad(!field->prefix_len); + ut_ad(dfield->len >= FIELD_REF_SIZE); + dfield_set_len(dfield, dfield->len - FIELD_REF_SIZE); + } + + if (!field->prefix_len) { + continue; + } + + ut_ad(field->col->ord_part); + ut_ad(i < n_uniq); + + ulint len = dtype_get_at_most_n_mbchars( + field->col->prtype, + field->col->mbminlen, field->col->mbmaxlen, + field->prefix_len, dfield->len, + static_cast(dfield_get_data(dfield))); + dfield_set_len(dfield, len); + } + + return entry; } -/** Insert into SYS_COLUMNS and insert/update the hidden metadata record -for instant ADD COLUMN. +/** Insert or update SYS_COLUMNS and the hidden metadata record +for instant ALTER TABLE. +@param[in] ha_alter_info ALTER TABLE context @param[in,out] ctx ALTER TABLE context for the current partition @param[in] altered_table MySQL table that is being altered @param[in] table MySQL table as it is before the ALTER operation @param[in,out] trx dictionary transaction @retval true failure @retval false success */ -static -bool -innobase_add_instant_try( - ha_innobase_inplace_ctx*ctx, - const TABLE* altered_table, - const TABLE* table, - trx_t* trx) +static bool innobase_instant_try( + const Alter_inplace_info* ha_alter_info, + ha_innobase_inplace_ctx* ctx, + const TABLE* altered_table, + const TABLE* table, + trx_t* trx) { DBUG_ASSERT(!ctx->need_rebuild()); if (!ctx->is_instant()) return false; - DBUG_ASSERT(altered_table->s->fields > table->s->fields); - DBUG_ASSERT(ctx->old_table->n_cols == ctx->old_n_cols); - dict_table_t* user_table = ctx->old_table; - user_table->instant_add_column(*ctx->instant_table); + dict_index_t* index = dict_table_get_first_index(user_table); + uint n_old_fields = index->n_fields; + const dict_col_t* old_cols = user_table->cols; + DBUG_ASSERT(user_table->n_cols == ctx->old_n_cols); + + user_table->instant_column(*ctx->instant_table, ctx->col_map); + + DBUG_ASSERT(index->n_fields >= n_old_fields); /* The table may have been emptied and may have lost its - 'instant-add-ness' during this instant ADD COLUMN. */ + 'instantness' during this ALTER TABLE. */ /* Construct a table row of default values for the stored columns. */ dtuple_t* row = dtuple_create(ctx->heap, user_table->n_cols); dict_table_copy_types(row, user_table); Field** af = altered_table->field; Field** const end = altered_table->field + altered_table->s->fields; - + ut_d(List_iterator_fast cf_it( + ha_alter_info->alter_info->create_list)); + if (ctx->first_alter_pos + && innobase_instant_drop_cols(user_table->id, + ctx->first_alter_pos - 1, trx)) { + return true; + } for (uint i = 0; af < end; af++) { if (!(*af)->stored_in_db()) { + ut_d(cf_it++); continue; } - dict_col_t* col = dict_table_get_nth_col(user_table, i); - DBUG_ASSERT(!strcmp((*af)->field_name.str, - dict_table_get_col_name(user_table, i))); + const dict_col_t* old = dict_table_t::find(old_cols, + ctx->col_map, + ctx->old_n_cols, i); + DBUG_ASSERT(!old || i >= ctx->old_n_cols - DATA_N_SYS_COLS + || old->ind == i + || (ctx->first_alter_pos + && old->ind >= ctx->first_alter_pos - 1)); dfield_t* d = dtuple_get_nth_field(row, i); + const dict_col_t* col = dict_table_get_nth_col(user_table, i); + DBUG_ASSERT(!col->is_virtual()); + DBUG_ASSERT(!col->is_dropped()); + DBUG_ASSERT(col->mtype != DATA_SYS); + DBUG_ASSERT(!strcmp((*af)->field_name.str, + dict_table_get_col_name(user_table, i))); + DBUG_ASSERT(old || col->is_added()); - if (col->is_instant()) { + if (col->is_added()) { dfield_set_data(d, col->def_val.data, col->def_val.len); } else if ((*af)->real_maybe_null()) { @@ -4322,45 +5291,65 @@ innobase_add_instant_try( } } - if (i + DATA_N_SYS_COLS < ctx->old_n_cols) { - i++; - continue; - } - - pars_info_t* info = pars_info_create(); - pars_info_add_ull_literal(info, "id", user_table->id); - pars_info_add_int4_literal(info, "pos", i); - pars_info_add_str_literal(info, "name", (*af)->field_name.str); - pars_info_add_int4_literal(info, "mtype", d->type.mtype); - pars_info_add_int4_literal(info, "prtype", d->type.prtype); - pars_info_add_int4_literal(info, "len", d->type.len); + ut_d(const Create_field* new_field = cf_it++); + /* new_field->field would point to an existing column. + If it is NULL, the column was added by this ALTER TABLE. */ + ut_ad(!new_field->field == !old); - dberr_t err = que_eval_sql( - info, - "PROCEDURE ADD_COL () IS\n" - "BEGIN\n" - "INSERT INTO SYS_COLUMNS VALUES" - "(:id,:pos,:name,:mtype,:prtype,:len,0);\n" - "END;\n", FALSE, trx); - if (err != DB_SUCCESS) { - my_error(ER_INTERNAL_ERROR, MYF(0), - "InnoDB: Insert into SYS_COLUMNS failed"); - return(true); + if (old && (!ctx->first_alter_pos + || i < ctx->first_alter_pos - 1)) { + /* The record is already present in SYS_COLUMNS. */ + } else if (innobase_instant_add_col(user_table->id, i, + (*af)->field_name.str, + d->type, trx)) { + return true; } i++; } - if (innodb_update_n_cols(user_table, dict_table_encode_n_col( - unsigned(user_table->n_cols) - - DATA_N_SYS_COLS, - user_table->n_v_cols) - | (user_table->flags & DICT_TF_COMPACT) << 31, - trx)) { + if (innodb_update_cols(user_table, dict_table_encode_n_col( + unsigned(user_table->n_cols) + - DATA_N_SYS_COLS, + user_table->n_v_cols) + | (user_table->flags & DICT_TF_COMPACT) << 31, + trx)) { return true; } + if (ctx->first_alter_pos) { +add_all_virtual: + for (uint i = 0; i < user_table->n_v_cols; i++) { + if (innobase_add_one_virtual( + user_table, + dict_table_get_v_col_name(user_table, i), + &user_table->v_cols[i], trx)) { + return true; + } + } + } else if (ha_alter_info->handler_flags & ALTER_DROP_VIRTUAL_COLUMN) { + if (innobase_instant_drop_cols(user_table->id, 65536, trx)) { + return true; + } + goto add_all_virtual; + } else if ((ha_alter_info->handler_flags & ALTER_ADD_VIRTUAL_COLUMN) + && innobase_add_virtual_try(ha_alter_info, user_table, + trx)) { + return true; + } + unsigned i = unsigned(user_table->n_cols) - DATA_N_SYS_COLS; + DBUG_ASSERT(i >= altered_table->s->stored_fields); + DBUG_ASSERT(i <= altered_table->s->stored_fields + 1); + if (i > altered_table->s->fields) { + const dict_col_t& fts_doc_id = user_table->cols[i - 1]; + DBUG_ASSERT(!strcmp(fts_doc_id.name(*user_table), + FTS_DOC_ID_COL_NAME)); + DBUG_ASSERT(!fts_doc_id.is_nullable()); + DBUG_ASSERT(fts_doc_id.len == 8); + dfield_set_data(dtuple_get_nth_field(row, i - 1), + field_ref_zero, fts_doc_id.len); + } byte trx_id[DATA_TRX_ID_LEN], roll_ptr[DATA_ROLL_PTR_LEN]; dfield_set_data(dtuple_get_nth_field(row, i++), field_ref_zero, DATA_ROW_ID_LEN); @@ -4374,9 +5363,7 @@ innobase_add_instant_try( row_ins_clust_index_entry_low() searches for the insert position. */ memset(roll_ptr, 0, sizeof roll_ptr); - dtuple_t* entry = row_build_index_entry(row, NULL, index, ctx->heap); - entry->info_bits = REC_INFO_METADATA; - + dtuple_t* entry = index->instant_metadata(*row, ctx->heap); mtr_t mtr; mtr.start(); index->set_modified(mtr); @@ -4395,27 +5382,58 @@ innobase_add_instant_try( NULL, trx, ctx->heap, NULL); dberr_t err; - if (rec_is_metadata(rec, index)) { + if (rec_is_metadata(rec, *index)) { ut_ad(page_rec_is_user_rec(rec)); if (!page_has_next(block->frame) && page_rec_is_last(rec, block->frame)) { goto empty_table; } - /* Extend the record with the instantly added columns. */ - const unsigned n = user_table->n_cols - ctx->old_n_cols; + + /* Ensure that the root page is in the correct format. */ + buf_block_t* root = btr_root_block_get(index, RW_X_LATCH, + &mtr); + DBUG_ASSERT(root); + DBUG_ASSERT(!root->page.encrypted); + if (fil_page_get_type(root->frame) != FIL_PAGE_TYPE_INSTANT) { + DBUG_ASSERT(!"wrong page type"); + err = DB_CORRUPTION; + goto func_exit; + } + + btr_set_instant(root, *index, &mtr); + + /* Extend the record with any added columns. */ + uint n = uint(index->n_fields) - n_old_fields; /* Reserve room for DB_TRX_ID,DB_ROLL_PTR and any non-updated off-page columns in case they are moved off page as a result of the update. */ - upd_t* update = upd_create(index->n_fields, ctx->heap); - update->n_fields = n; - update->info_bits = REC_INFO_METADATA; + const unsigned f = user_table->instant != NULL; + upd_t* update = upd_create(index->n_fields + f, ctx->heap); + update->n_fields = n + f; + update->info_bits = f + ? REC_INFO_METADATA_ALTER + : REC_INFO_METADATA_ADD; + if (f) { + upd_field_t* uf = upd_get_nth_field(update, 0); + uf->field_no = index->first_user_field(); + uf->new_val = entry->fields[uf->field_no]; + DBUG_ASSERT(!dfield_is_ext(&uf->new_val)); + DBUG_ASSERT(!dfield_is_null(&uf->new_val)); + } + /* Add the default values for instantly added columns */ - for (unsigned i = 0; i < n; i++) { - upd_field_t* uf = upd_get_nth_field(update, i); - unsigned f = index->n_fields - n + i; - uf->field_no = f; - uf->new_val = entry->fields[f]; + unsigned j = f; + + for (unsigned k = n_old_fields; k < index->n_fields; k++) { + upd_field_t* uf = upd_get_nth_field(update, j++); + uf->field_no = k + f; + uf->new_val = entry->fields[k + f]; + + ut_ad(j <= n + f); } + + ut_ad(j == n + f); + ulint* offsets = NULL; mem_heap_t* offsets_heap = NULL; big_rec_t* big_rec; @@ -4425,6 +5443,10 @@ innobase_add_instant_try( &offsets, &offsets_heap, ctx->heap, &big_rec, update, UPD_NODE_NO_ORD_CHANGE, thr, trx->id, &mtr); + + offsets = rec_get_offsets( + btr_pcur_get_rec(&pcur), index, offsets, + true, ULINT_UNDEFINED, &offsets_heap); if (big_rec) { if (err == DB_SUCCESS) { err = btr_store_big_rec_extern_fields( @@ -4444,27 +5466,24 @@ empty_table: /* The table is empty. */ ut_ad(page_is_root(block->frame)); btr_page_empty(block, NULL, index, 0, &mtr); - index->remove_instant(); + index->clear_instant_alter(); err = DB_SUCCESS; goto func_exit; } - /* Convert the table to the instant ADD COLUMN format. */ + /* Convert the table to the instant ALTER TABLE format. */ ut_ad(user_table->is_instant()); mtr.commit(); mtr.start(); index->set_modified(mtr); - if (page_t* root = btr_root_get(index, &mtr)) { - if (fil_page_get_type(root) != FIL_PAGE_INDEX) { + if (buf_block_t* root = btr_root_block_get(index, RW_SX_LATCH, &mtr)) { + if (root->page.encrypted + || fil_page_get_type(root->frame) != FIL_PAGE_INDEX) { DBUG_ASSERT(!"wrong page type"); goto err_exit; } - DBUG_ASSERT(!page_is_comp(root) || !page_get_instant(root)); - mlog_write_ulint(root + FIL_PAGE_TYPE, - FIL_PAGE_TYPE_INSTANT, MLOG_2BYTES, - &mtr); - page_set_instant(root, index->n_core_fields, &mtr); + btr_set_instant(root, *index, &mtr); mtr.commit(); mtr.start(); index->set_modified(mtr); @@ -4488,222 +5507,6 @@ func_exit: return false; } -/** Update INNODB SYS_COLUMNS on new virtual column's position -@param[in] table InnoDB table -@param[in] old_pos old position -@param[in] new_pos new position -@param[in] trx transaction -@return DB_SUCCESS if successful, otherwise error code */ -static -dberr_t -innobase_update_v_pos_sys_columns( - const dict_table_t* table, - ulint old_pos, - ulint new_pos, - trx_t* trx) -{ - pars_info_t* info = pars_info_create(); - - pars_info_add_int4_literal(info, "pos", old_pos); - pars_info_add_int4_literal(info, "val", new_pos); - pars_info_add_ull_literal(info, "id", table->id); - - dberr_t error = que_eval_sql( - info, - "PROCEDURE P () IS\n" - "BEGIN\n" - "UPDATE SYS_COLUMNS\n" - "SET POS = :val\n" - "WHERE POS = :pos\n" - "AND TABLE_ID = :id;\n" - "END;\n", - FALSE, trx); - - return(error); -} - -/** Update INNODB SYS_VIRTUAL table with new virtual column position -@param[in] table InnoDB table -@param[in] old_pos old position -@param[in] new_pos new position -@param[in] trx transaction -@return DB_SUCCESS if successful, otherwise error code */ -static -dberr_t -innobase_update_v_pos_sys_virtual( - const dict_table_t* table, - ulint old_pos, - ulint new_pos, - trx_t* trx) -{ - pars_info_t* info = pars_info_create(); - - pars_info_add_int4_literal(info, "pos", old_pos); - pars_info_add_int4_literal(info, "val", new_pos); - pars_info_add_ull_literal(info, "id", table->id); - - dberr_t error = que_eval_sql( - info, - "PROCEDURE P () IS\n" - "BEGIN\n" - "UPDATE SYS_VIRTUAL\n" - "SET POS = :val\n" - "WHERE POS = :pos\n" - "AND TABLE_ID = :id;\n" - "END;\n", - FALSE, trx); - - return(error); -} - -/** Update InnoDB system tables on dropping a virtual column -@param[in] table InnoDB table -@param[in] col_name column name of the dropping column -@param[in] drop_col col information for the dropping column -@param[in] n_prev_dropped number of previously dropped columns in the - same alter clause -@param[in] trx transaction -@return DB_SUCCESS if successful, otherwise error code */ -static -dberr_t -innobase_drop_one_virtual_sys_columns( - const dict_table_t* table, - const char* col_name, - dict_col_t* drop_col, - ulint n_prev_dropped, - trx_t* trx) -{ - pars_info_t* info = pars_info_create(); - pars_info_add_ull_literal(info, "id", table->id); - - pars_info_add_str_literal(info, "name", col_name); - - dberr_t error = que_eval_sql( - info, - "PROCEDURE P () IS\n" - "BEGIN\n" - "DELETE FROM SYS_COLUMNS\n" - "WHERE TABLE_ID = :id\n" - "AND NAME = :name;\n" - "END;\n", - FALSE, trx); - - if (error != DB_SUCCESS) { - return(error); - } - - dict_v_col_t* v_col = dict_table_get_nth_v_col_mysql( - table, drop_col->ind); - - /* Adjust column positions for all subsequent columns */ - for (ulint i = v_col->v_pos + 1; i < table->n_v_cols; i++) { - dict_v_col_t* t_col = dict_table_get_nth_v_col(table, i); - ulint old_p = dict_create_v_col_pos( - t_col->v_pos - n_prev_dropped, - t_col->m_col.ind - n_prev_dropped); - ulint new_p = dict_create_v_col_pos( - t_col->v_pos - 1 - n_prev_dropped, - ulint(t_col->m_col.ind) - 1 - n_prev_dropped); - - error = innobase_update_v_pos_sys_columns( - table, old_p, new_p, trx); - if (error != DB_SUCCESS) { - return(error); - } - error = innobase_update_v_pos_sys_virtual( - table, old_p, new_p, trx); - if (error != DB_SUCCESS) { - return(error); - } - } - - return(error); -} - -/** Delete virtual column's info from INNODB SYS_VIRTUAL -@param[in] table InnoDB table -@param[in] pos position of the virtual column to be deleted -@param[in] trx transaction -@return DB_SUCCESS if successful, otherwise error code */ -static -dberr_t -innobase_drop_one_virtual_sys_virtual( - const dict_table_t* table, - ulint pos, - trx_t* trx) -{ - pars_info_t* info = pars_info_create(); - pars_info_add_ull_literal(info, "id", table->id); - - pars_info_add_int4_literal(info, "pos", pos); - - dberr_t error = que_eval_sql( - info, - "PROCEDURE P () IS\n" - "BEGIN\n" - "DELETE FROM SYS_VIRTUAL\n" - "WHERE TABLE_ID = :id\n" - "AND POS = :pos;\n" - "END;\n", - FALSE, trx); - - return(error); -} - -/** Update system table for dropping virtual column(s) -@param[in] ha_alter_info Data used during in-place alter -@param[in] user_table InnoDB table -@param[in] trx transaction -@retval true Failure -@retval false Success */ -static -bool -innobase_drop_virtual_try( - Alter_inplace_info* ha_alter_info, - const dict_table_t* user_table, - trx_t* trx) -{ - ha_innobase_inplace_ctx* ctx; - dberr_t err = DB_SUCCESS; - - ctx = static_cast - (ha_alter_info->handler_ctx); - - for (ulint i = 0; i < ctx->num_to_drop_vcol; i++) { - - ulint pos = dict_create_v_col_pos( - ctx->drop_vcol[i].v_pos - i, - ctx->drop_vcol[i].m_col.ind - i); - err = innobase_drop_one_virtual_sys_virtual( - user_table, pos, trx); - - if (err != DB_SUCCESS) { - my_error(ER_INTERNAL_ERROR, MYF(0), - "InnoDB: DROP COLUMN...VIRTUAL"); - return(true); - } - - err = innobase_drop_one_virtual_sys_columns( - user_table, ctx->drop_vcol_name[i], - &(ctx->drop_vcol[i].m_col), i, trx); - - if (err != DB_SUCCESS) { - my_error(ER_INTERNAL_ERROR, MYF(0), - "InnoDB: DROP COLUMN...VIRTUAL"); - return(true); - } - } - - - ulint n_col = unsigned(user_table->n_cols) - DATA_N_SYS_COLS; - ulint n_v_col = unsigned(user_table->n_v_cols) - - ctx->num_to_drop_vcol; - ulint new_n = dict_table_encode_n_col(n_col, n_v_col) - | ((user_table->flags & DICT_TF_COMPACT) << 31); - - return innodb_update_n_cols(user_table, new_n, trx); -} - /** Adjust the create index column number from "New table" to "old InnoDB table" while we are doing dropping virtual column. Since we do not create separate new table for the dropping/adding virtual columns. @@ -5296,20 +6099,12 @@ new_clustered_failed: == !!new_clustered); } - if (ctx->need_rebuild() && user_table->supports_instant()) { - if (!instant_alter_column_possible(ha_alter_info, old_table)) { - goto not_instant_add_column; - } - - for (uint i = uint(ctx->old_table->n_cols) - DATA_N_SYS_COLS; - i--; ) { - if (ctx->col_map[i] != i) { - goto not_instant_add_column; - } - } - - DBUG_ASSERT(ctx->new_table->n_cols > ctx->old_table->n_cols); - + if (ctx->need_rebuild() && instant_alter_column_possible( + *user_table, ha_alter_info, old_table) +#if 1 // MDEV-17459: adjust fts_fetch_doc_from_rec() and friends; remove this + && !innobase_fulltext_exist(altered_table) +#endif + ) { for (uint a = 0; a < ctx->num_to_add_index; a++) { ctx->add_index[a]->table = ctx->new_table; ctx->add_index[a] = dict_index_add_to_cache( @@ -5317,6 +6112,7 @@ new_clustered_failed: &error, add_v); ut_a(error == DB_SUCCESS); } + DBUG_ASSERT(ha_alter_info->key_count /* hidden GEN_CLUST_INDEX in InnoDB */ + dict_index_is_auto_gen_clust( @@ -5328,6 +6124,7 @@ new_clustered_failed: altered_table->key_info) != FTS_EXIST_DOC_ID_INDEX) == ctx->num_to_add_index); + ctx->num_to_add_index = 0; ctx->add_index = NULL; @@ -5358,24 +6155,11 @@ new_clustered_failed: DBUG_ASSERT(!strcmp((*af)->field_name.str, dict_table_get_col_name(ctx->new_table, i))); - DBUG_ASSERT(!col->is_instant()); + DBUG_ASSERT(!col->is_added()); if (new_field->field) { - ut_d(const dict_col_t* old_col - = dict_table_get_nth_col(user_table, i)); - ut_d(const dict_index_t* index - = user_table->indexes.start); - DBUG_SLOW_ASSERT(col->mtype == old_col->mtype); - DBUG_SLOW_ASSERT(col->prtype == old_col->prtype); - DBUG_SLOW_ASSERT(col->mbminlen - == old_col->mbminlen); - DBUG_SLOW_ASSERT(col->mbmaxlen - == old_col->mbmaxlen); - DBUG_SLOW_ASSERT(col->len >= old_col->len); - DBUG_SLOW_ASSERT(old_col->is_instant() - == (dict_col_get_clust_pos( - old_col, index) - >= index->n_core_fields)); + /* This is a pre-existing column, + possibly at a different position. */ } else if ((*af)->is_real_null()) { /* DEFAULT NULL */ col->def_val.len = UNIV_SQL_NULL; @@ -5447,7 +6231,6 @@ new_clustered_failed: } if (ctx->need_rebuild()) { -not_instant_add_column: DBUG_ASSERT(ctx->need_rebuild()); DBUG_ASSERT(!ctx->is_instant()); DBUG_ASSERT(num_fts_index <= 1); @@ -7710,8 +8493,10 @@ err_exit: index = dict_table_get_next_index(index)) { for (ulint i = 0; i < dict_index_get_n_fields(index); i++) { - if (strcmp(dict_index_get_nth_field(index, i)->name, - from)) { + const dict_field_t& f = index->fields[i]; + DBUG_ASSERT(!f.name == f.col->is_dropped()); + + if (!f.name || strcmp(f.name, from)) { continue; } @@ -9040,23 +9825,36 @@ commit_try_norebuild( } #endif /* MYSQL_RENAME_INDEX */ - if ((ha_alter_info->handler_flags - & ALTER_DROP_VIRTUAL_COLUMN) - && innobase_drop_virtual_try(ha_alter_info, ctx->old_table, trx)) { - DBUG_RETURN(true); - } + if (!ctx->is_instant() && ha_alter_info->handler_flags + & (ALTER_DROP_VIRTUAL_COLUMN | ALTER_ADD_VIRTUAL_COLUMN)) { + if ((ha_alter_info->handler_flags & ALTER_DROP_VIRTUAL_COLUMN) + && innobase_drop_virtual_try(ha_alter_info, ctx->old_table, + trx)) { + DBUG_RETURN(true); + } - if ((ha_alter_info->handler_flags - & ALTER_ADD_VIRTUAL_COLUMN) - && innobase_add_virtual_try(ha_alter_info, ctx->old_table, trx)) { - DBUG_RETURN(true); - } + if ((ha_alter_info->handler_flags & ALTER_ADD_VIRTUAL_COLUMN) + && innobase_add_virtual_try(ha_alter_info, ctx->old_table, + trx)) { + DBUG_RETURN(true); + } - if (innobase_add_instant_try(ctx, altered_table, old_table, trx)) { - DBUG_RETURN(true); + ulint n_col = unsigned(ctx->old_table->n_cols) + - DATA_N_SYS_COLS; + ulint n_v_col = unsigned(ctx->old_table->n_v_cols) + + ctx->num_to_add_vcol - ctx->num_to_drop_vcol; + + if (innodb_update_cols( + ctx->old_table, + dict_table_encode_n_col(n_col, n_v_col) + | unsigned(ctx->old_table->flags & DICT_TF_COMPACT) + << 31, trx)) { + DBUG_RETURN(true); + } } - DBUG_RETURN(false); + DBUG_RETURN(innobase_instant_try(ha_alter_info, ctx, altered_table, + old_table, trx)); } /** Commit the changes to the data dictionary cache @@ -9217,6 +10015,42 @@ commit_cache_norebuild( if (!ctx->is_instant()) { innobase_rename_or_enlarge_columns_cache( ha_alter_info, table, ctx->new_table); + } else { + ut_ad(ctx->col_map); + + if (fts_t* fts = ctx->new_table->fts) { + ut_ad(fts->doc_col != ULINT_UNDEFINED); + ut_ad(ctx->new_table->n_cols > DATA_N_SYS_COLS); + const ulint c = ctx->col_map[fts->doc_col]; + ut_ad(c < ulint(ctx->new_table->n_cols) + - DATA_N_SYS_COLS); + ut_d(const dict_col_t& col = ctx->new_table->cols[c]); + ut_ad(!col.is_nullable()); + ut_ad(!col.is_virtual()); + ut_ad(!col.is_added()); + ut_ad(col.prtype & DATA_UNSIGNED); + ut_ad(col.mtype == DATA_INT); + ut_ad(col.len == 8); + ut_ad(col.ord_part); + fts->doc_col = c; + } + + if (ha_alter_info->handler_flags & ALTER_DROP_STORED_COLUMN) { + dict_index_t* index = dict_table_get_first_index( + ctx->new_table); + for (const dict_field_t* f = index->fields, + * const end = f + index->n_fields; + f != end; f++) { + dict_col_t& c = *f->col; + if (c.is_dropped()) { + c.set_dropped(!c.is_nullable(), + DATA_LARGE_MTYPE(c.mtype) + || (!f->fixed_len + && c.len > 255), + f->fixed_len); + } + } + } } if (ha_alter_info->handler_flags & ALTER_COLUMN_UNVERSIONED) { @@ -9922,6 +10756,9 @@ foreign_fail: } } + /* MDEV-17468: Avoid this at least when ctx->is_instant(). + Currently dict_load_column_low() is the only place where + num_base for virtual columns is assigned to nonzero. */ if (ctx0->num_to_drop_vcol || ctx0->num_to_add_vcol) { DBUG_ASSERT(ctx0->old_table->get_ref_count() == 1); @@ -9939,6 +10776,12 @@ foreign_fail: tb_name[strlen(m_prebuilt->table->name.m_name)] = 0; dict_table_close(m_prebuilt->table, true, false); + if (ctx0->is_instant()) { + for (unsigned i = ctx0->old_n_v_cols; i--; ) { + UT_DELETE(ctx0->old_v_cols[i].v_indexes); + } + const_cast(ctx0->old_n_v_cols) = 0; + } dict_table_remove_from_cache(m_prebuilt->table); m_prebuilt->table = dict_table_open_on_name( tb_name, TRUE, TRUE, DICT_ERR_IGNORE_NONE); diff --git a/storage/innobase/include/btr0btr.h b/storage/innobase/include/btr0btr.h index 1951d6f4eac..fc3f2c3b2fd 100644 --- a/storage/innobase/include/btr0btr.h +++ b/storage/innobase/include/btr0btr.h @@ -421,6 +421,12 @@ void btr_write_autoinc(dict_index_t* index, ib_uint64_t autoinc, bool reset = false) MY_ATTRIBUTE((nonnull)); +/** Write instant ALTER TABLE metadata to a root page. +@param[in,out] root clustered index root page +@param[in] index clustered index with instant ALTER TABLE +@param[in,out] mtr mini-transaction */ +void btr_set_instant(buf_block_t* root, const dict_index_t& index, mtr_t* mtr); + /*************************************************************//** Makes tree one level higher by splitting the root, and inserts the tuple. It is assumed that mtr contains an x-latch on the tree. diff --git a/storage/innobase/include/data0data.h b/storage/innobase/include/data0data.h index 0fa4cbe8777..a120cd358ab 100644 --- a/storage/innobase/include/data0data.h +++ b/storage/innobase/include/data0data.h @@ -1,7 +1,7 @@ /***************************************************************************** Copyright (c) 1994, 2016, Oracle and/or its affiliates. All Rights Reserved. -Copyright (c) 2017, MariaDB Corporation. +Copyright (c) 2017, 2018, MariaDB Corporation. This program is free software; you can redistribute it and/or modify it under the terms of the GNU General Public License as published by the Free Software @@ -645,6 +645,33 @@ struct dtuple_t { inserted or updated. @param[in] index index possibly with instantly added columns */ void trim(const dict_index_t& index); + + /** + @param info_bits the info_bits of a data tuple + @return whether this is a hidden metadata record + for instant ADD COLUMN or ALTER TABLE */ + static bool is_alter_metadata(ulint info_bits) + { + return UNIV_UNLIKELY(info_bits == REC_INFO_METADATA_ALTER); + } + + /** + @param info_bits the info_bits of a data tuple + @return whether this is a hidden metadata record + for instant ADD COLUMN or ALTER TABLE */ + static bool is_metadata(ulint info_bits) + { + return UNIV_UNLIKELY((info_bits & ~REC_INFO_DELETED_FLAG) + == REC_INFO_METADATA_ADD); + } + + /** @return whether this is a hidden metadata record + for instant ALTER TABLE (not only ADD COLUMN) */ + bool is_alter_metadata() const { return is_alter_metadata(info_bits); } + + /** @return whether this is a hidden metadata record + for instant ADD COLUMN or ALTER TABLE */ + bool is_metadata() const { return is_metadata(info_bits); } }; /** A slot for a field in a big rec vector */ diff --git a/storage/innobase/include/data0type.h b/storage/innobase/include/data0type.h index b999106fee0..3b3ac9d1885 100644 --- a/storage/innobase/include/data0type.h +++ b/storage/innobase/include/data0type.h @@ -554,11 +554,55 @@ struct dtype_t{ { return (prtype & DATA_VERSIONED) == DATA_VERS_END; } + + /** Set the type of the BLOB in the hidden metadata record. */ + void metadata_blob_init() + { + prtype = DATA_NOT_NULL; + mtype = DATA_BLOB; + len = 0; + mbminlen = 0; + mbmaxlen = 0; + } }; /** The DB_TRX_ID,DB_ROLL_PTR values for "no history is available" */ extern const byte reset_trx_id[DATA_TRX_ID_LEN + DATA_ROLL_PTR_LEN]; +/** Info bit denoting the predefined minimum record: this bit is set +if and only if the record is the first user record on a non-leaf +B-tree page that is the leftmost page on its level +(PAGE_LEVEL is nonzero and FIL_PAGE_PREV is FIL_NULL). */ +#define REC_INFO_MIN_REC_FLAG 0x10UL +/** The delete-mark flag in info bits */ +#define REC_INFO_DELETED_FLAG 0x20UL + +/** Record status values for ROW_FORMAT=COMPACT,DYNAMIC,COMPRESSED */ +enum rec_comp_status_t { + /** User record (PAGE_LEVEL=0, heap>=PAGE_HEAP_NO_USER_LOW) */ + REC_STATUS_ORDINARY = 0, + /** Node pointer record (PAGE_LEVEL>=0, heap>=PAGE_HEAP_NO_USER_LOW) */ + REC_STATUS_NODE_PTR = 1, + /** The page infimum pseudo-record (heap=PAGE_HEAP_NO_INFIMUM) */ + REC_STATUS_INFIMUM = 2, + /** The page supremum pseudo-record (heap=PAGE_HEAP_NO_SUPREMUM) */ + REC_STATUS_SUPREMUM = 3, + /** Clustered index record that has been inserted or updated + after instant ADD COLUMN (more than dict_index_t::n_core_fields) */ + REC_STATUS_INSTANT = 4 +}; + +/** The dtuple_t::info_bits of the hidden metadata of instant ADD COLUMN. +@see rec_is_metadata() +@see rec_is_alter_metadata() */ +static const byte REC_INFO_METADATA_ADD + = REC_INFO_MIN_REC_FLAG | REC_STATUS_INSTANT; + +/** The dtuple_t::info_bits of the hidden metadata of instant ALTER TABLE. +@see rec_is_metadata() */ +static const byte REC_INFO_METADATA_ALTER + = REC_INFO_METADATA_ADD | REC_INFO_DELETED_FLAG; + #include "data0type.ic" #endif diff --git a/storage/innobase/include/dict0dict.ic b/storage/innobase/include/dict0dict.ic index 3bcd1abfbbf..6dcc40db70a 100644 --- a/storage/innobase/include/dict0dict.ic +++ b/storage/innobase/include/dict0dict.ic @@ -490,7 +490,8 @@ dict_table_get_nth_v_col( ut_ad(table); ut_ad(pos < table->n_v_def); ut_ad(table->magic_n == DICT_TABLE_MAGIC_N); - ut_ad(!table->v_cols[pos].m_col.is_instant()); + ut_ad(!table->v_cols[pos].m_col.is_added()); + ut_ad(!table->v_cols[pos].m_col.is_dropped()); return &table->v_cols[pos]; } diff --git a/storage/innobase/include/dict0mem.h b/storage/innobase/include/dict0mem.h index 7637a387b8f..b4823ee6a94 100644 --- a/storage/innobase/include/dict0mem.h +++ b/storage/innobase/include/dict0mem.h @@ -587,6 +587,10 @@ struct dict_col_t{ this column. Our current max limit is 3072 (REC_VERSION_56_MAX_INDEX_COL_LEN) bytes. */ +private: + /** Special value of ind for a dropped column */ + static const unsigned DROPPED = 1023; +public: /** Detach the column from an index. @param[in] index index to be detached from */ @@ -630,26 +634,55 @@ struct dict_col_t{ } /** @return whether this is an instantly-added column */ - bool is_instant() const + bool is_added() const { DBUG_ASSERT(def_val.len != UNIV_SQL_DEFAULT || !def_val.data); return def_val.len != UNIV_SQL_DEFAULT; } + /** Flag the column instantly dropped */ + void set_dropped() { ind = DROPPED; } + /** Flag the column instantly dropped. + @param[in] not_null whether the column was NOT NULL + @param[in] len2 whether the length exceeds 255 bytes + @param[in] fixed_len the fixed length in bytes, or 0 */ + void set_dropped(bool not_null, bool len2, unsigned fixed) + { + DBUG_ASSERT(!len2 || !fixed); + prtype = not_null + ? DATA_NOT_NULL | DATA_BINARY_TYPE + : DATA_BINARY_TYPE; + if (fixed) { + mtype = DATA_FIXBINARY; + len = fixed; + } else { + mtype = DATA_BINARY; + len = len2 ? 65535 : 255; + } + mbminlen = mbmaxlen = 0; + ind = DROPPED; + ord_part = 0; + max_prefix = 0; + } + /** @return whether the column was instantly dropped */ + bool is_dropped() const { return ind == DROPPED; } + /** @return whether the column was instantly dropped + @param[in] index the clustered index */ + inline bool is_dropped(const dict_index_t& index) const; + /** Get the default value of an instantly-added column. @param[out] len value length (in bytes), or UNIV_SQL_NULL @return default value @retval NULL if the default value is SQL NULL (len=UNIV_SQL_NULL) */ const byte* instant_value(ulint* len) const { - DBUG_ASSERT(is_instant()); + DBUG_ASSERT(is_added()); *len = def_val.len; return static_cast(def_val.data); } /** Remove the 'instant ADD' status of the column */ - void remove_instant() + void clear_instant() { - DBUG_ASSERT(is_instant()); def_val.len = UNIV_SQL_DEFAULT; def_val.data = NULL; } @@ -853,7 +886,7 @@ to start with. */ /** Data structure for an index. Most fields will be initialized to 0, NULL or FALSE in dict_mem_index_create(). */ -struct dict_index_t{ +struct dict_index_t { index_id_t id; /*!< id of the index */ mem_heap_t* heap; /*!< memory heap */ id_name_t name; /*!< index name */ @@ -1042,7 +1075,7 @@ struct dict_index_t{ page cannot be read or decrypted */ inline bool is_readable() const; - /** @return whether instant ADD COLUMN is in effect */ + /** @return whether instant ALTER TABLE is in effect */ inline bool is_instant() const; /** @return whether the index is the primary key index @@ -1107,24 +1140,20 @@ struct dict_index_t{ return fields[n].col->instant_value(len); } - /** Adjust clustered index metadata for instant ADD COLUMN. - @param[in] clustered index definition after instant ADD COLUMN */ - void instant_add_field(const dict_index_t& instant); - - /** Remove the 'instant ADD' status of a clustered index. - Protected by index root page x-latch or table X-lock. */ - void remove_instant() - { - DBUG_ASSERT(is_primary()); - if (!is_instant()) { - return; - } - for (unsigned i = n_core_fields; i < n_fields; i++) { - fields[i].col->remove_instant(); - } - n_core_fields = n_fields; - n_core_null_bytes = UT_BITS_IN_BYTES(unsigned(n_nullable)); - } + /** Adjust index metadata for instant ADD/DROP/reorder COLUMN. + @param[in] clustered index definition after instant ALTER TABLE */ + inline void instant_add_field(const dict_index_t& instant); + /** Remove instant ADD COLUMN metadata. */ + inline void clear_instant_add(); + /** Remove instant ALTER TABLE metadata. */ + inline void clear_instant_alter(); + + /** Construct the metadata record for instant ALTER TABLE. + @param[in] row dummy or default values for existing columns + @param[in,out] heap memory heap for allocations + @return metadata record */ + inline dtuple_t* + instant_metadata(const dtuple_t& row, mem_heap_t* heap) const; /** Check if record in clustered index is historical row. @param[in] rec clustered row @@ -1139,6 +1168,9 @@ struct dict_index_t{ @return true on error */ bool vers_history_row(const rec_t* rec, bool &history_row); + + /** Reconstruct the clustered index fields. */ + inline void reconstruct_fields(); }; /** Detach a column from an index. @@ -1473,6 +1505,17 @@ struct dict_vcol_templ_t { dict_vcol_templ_t() : vtempl(0), mysql_table_query_id(~0ULL) {} }; +/** Instantly dropped or reordered columns */ +struct dict_instant_t +{ + /** Number of dropped columns */ + unsigned n_dropped; + /** Dropped columns */ + dict_col_t* dropped; + /** Mapping the non-pk field to column of the table. */ + unsigned* non_pk_col_map; +}; + /** These are used when MySQL FRM and InnoDB data dictionary are in inconsistent state. */ typedef enum { @@ -1526,30 +1569,89 @@ struct dict_table_t { return(UNIV_LIKELY(!file_unreadable)); } - /** @return whether instant ADD COLUMN is in effect */ + /** @return whether instant ALTER TABLE is in effect */ bool is_instant() const { return(UT_LIST_GET_FIRST(indexes)->is_instant()); } - /** @return whether the table supports instant ADD COLUMN */ + /** @return whether the table supports instant ALTER TABLE */ bool supports_instant() const { return(!(flags & DICT_TF_MASK_ZIP_SSIZE)); } - /** Adjust metadata for instant ADD COLUMN. - @param[in] table table definition after instant ADD COLUMN */ - void instant_add_column(const dict_table_t& table); + /** @return the number of instantly dropped columns */ + unsigned n_dropped() const { return instant ? instant->n_dropped : 0; } + + /** Look up an old column. + @param[in] cols the old columns of the table + @param[in] col_map map from old table columns to altered ones + @param[in] n_cols number of old columns + @param[in] i the number of the new column + @return old column + @retval NULL if column i was added to the table */ + static const dict_col_t* find(const dict_col_t* cols, + const ulint* col_map, ulint n_cols, + ulint i) + { + for (ulint o = n_cols; o--; ) { + if (col_map[o] == i) { + return &cols[o]; + } + } + return NULL; + } - /** Roll back instant_add_column(). - @param[in] old_n_cols original n_cols - @param[in] old_cols original cols - @param[in] old_col_names original col_names */ - void rollback_instant( + /** Serialise metadata of dropped or reordered columns. + @param[in,out] heap memory heap for allocation + @param[out] field data field with the metadata */ + void serialise_columns(mem_heap_t* heap, dfield_t* field) const; + + /** Reconstruct dropped or reordered columns. + @param[in] metadata data from serialise_columns() + @param[in] len length of the metadata, in bytes + @return whether parsing the metadata failed */ + bool deserialise_columns(const byte* metadata, ulint len); + + /** Set is_instant() before instant_column(). + @param[in] old previous table definition + @param[in] col_map map from old.cols[] + and old.v_cols[] to this + @param[out] first_alter_pos 0, or + 1 + first changed column position */ + inline void prepare_instant(const dict_table_t& old, + const ulint* col_map, + unsigned& first_alter_pos); + + /** Adjust table metadata for instant ADD/DROP/reorder COLUMN. + @param[in] table table on which prepare_instant() was invoked + @param[in] col_map mapping from cols[] and v_cols[] to table */ + inline void instant_column(const dict_table_t& table, + const ulint* col_map); + + /** Roll back instant_column(). + @param[in] old_n_cols original n_cols + @param[in] old_cols original cols + @param[in] old_col_names original col_names + @param[in] old_instant original instant structure + @param[in] old_fields original fields + @param[in] old_n_fields original number of fields + @param[in] old_n_v_cols original n_v_cols + @param[in] old_v_cols original v_cols + @param[in] old_v_col_names original v_col_names + @param[in] col_map column map */ + inline void rollback_instant( unsigned old_n_cols, dict_col_t* old_cols, - const char* old_col_names); + const char* old_col_names, + dict_instant_t* old_instant, + dict_field_t* old_fields, + unsigned old_n_fields, + unsigned old_n_v_cols, + dict_v_col_t* old_v_cols, + const char* old_v_col_names, + const ulint* col_map); /** Add the table definition to the data dictionary cache */ void add_to_cache(); @@ -1691,6 +1793,9 @@ struct dict_table_t { reason s_cols is a part of dict_table_t */ dict_s_col_list* s_cols; + /** Instantly dropped or reordered columns, or NULL if none */ + dict_instant_t* instant; + /** Column names packed in a character string "name1\0name2\0...nameN\0". Until the string contains n_cols, it will be allocated from a temporary heap. The final string will be allocated @@ -1972,12 +2077,15 @@ inline bool dict_index_t::is_readable() const { return table->is_readable(); } inline bool dict_index_t::is_instant() const { ut_ad(n_core_fields > 0); - ut_ad(n_core_fields <= n_fields); + ut_ad(n_core_fields <= n_fields || table->n_dropped()); ut_ad(n_core_fields == n_fields || (type & ~(DICT_UNIQUE | DICT_CORRUPT)) == DICT_CLUSTERED); ut_ad(n_core_fields == n_fields || table->supports_instant()); ut_ad(n_core_fields == n_fields || !table->is_temporary()); - return(n_core_fields != n_fields); + ut_ad(!table->instant || !table->is_temporary()); + + return n_core_fields != n_fields + || (is_primary() && table->instant); } inline bool dict_index_t::is_corrupted() const @@ -1987,6 +2095,84 @@ inline bool dict_index_t::is_corrupted() const || (table && table->corrupted)); } +inline void dict_index_t::clear_instant_add() +{ + DBUG_ASSERT(is_primary()); + DBUG_ASSERT(is_instant()); + DBUG_ASSERT(!table->instant); + for (unsigned i = n_core_fields; i < n_fields; i++) { + fields[i].col->clear_instant(); + } + n_core_fields = n_fields; + n_core_null_bytes = UT_BITS_IN_BYTES(unsigned(n_nullable)); +} + +inline void dict_index_t::clear_instant_alter() +{ + DBUG_ASSERT(is_primary()); + DBUG_ASSERT(n_fields == n_def); + + if (!table->instant) { + if (is_instant()) { + clear_instant_add(); + } + return; + } + +#ifndef DBUG_OFF + for (unsigned i = first_user_field(); i--; ) { + DBUG_ASSERT(!fields[i].col->is_dropped()); + DBUG_ASSERT(!fields[i].col->is_nullable()); + } +#endif + dict_field_t* end = &fields[n_fields]; + + for (dict_field_t* d = &fields[first_user_field()]; d < end; d++) { + /* Move fields for dropped columns to the end. */ + while (d->col->is_dropped()) { + if (d->col->is_nullable()) { + n_nullable--; + } + + std::swap(*d, *--end); + + if (d == end) { + goto done; + } + } + + /* Ensure that the surviving fields are sorted by + ascending order of columns. */ + const unsigned c = d->col->ind; + + for (dict_field_t* s = d + 1; s < end; s++) { + if (s->col->ind < c) { + std::swap(*d, *s); + break; + } + } + } + +done: + DBUG_ASSERT(&fields[n_fields - table->n_dropped()] == end); + + n_core_fields = n_fields = n_def = end - fields; + n_core_null_bytes = UT_BITS_IN_BYTES(n_nullable); + table->instant = NULL; +} + +/** @return whether the column was instantly dropped +@param[in] index the clustered index */ +inline bool dict_col_t::is_dropped(const dict_index_t& index) const +{ + DBUG_ASSERT(index.is_primary()); + DBUG_ASSERT(!is_dropped() == !index.table->instant); + DBUG_ASSERT(!is_dropped() || (this >= index.table->instant->dropped + && this < index.table->instant->dropped + + index.table->instant->n_dropped)); + return is_dropped(); +} + /*******************************************************************//** Initialise the table lock list. */ void diff --git a/storage/innobase/include/page0cur.ic b/storage/innobase/include/page0cur.ic index 86e560395f3..1ba85d93fb4 100644 --- a/storage/innobase/include/page0cur.ic +++ b/storage/innobase/include/page0cur.ic @@ -280,6 +280,7 @@ page_cur_tuple_insert( *offsets = rec_get_offsets(rec, index, *offsets, page_is_leaf(cursor->block->frame), ULINT_UNDEFINED, heap); + ut_ad(size == rec_offs_size(*offsets)); if (buf_block_get_page_zip(cursor->block)) { rec = page_cur_insert_rec_zip( diff --git a/storage/innobase/include/page0page.h b/storage/innobase/include/page0page.h index dd7d31ac3bc..8a805a85eda 100644 --- a/storage/innobase/include/page0page.h +++ b/storage/innobase/include/page0page.h @@ -1025,13 +1025,6 @@ page_get_direction(const page_t* page) inline uint16_t page_get_instant(const page_t* page); -/** Assign the PAGE_INSTANT field. -@param[in,out] page clustered index root page -@param[in] n original number of clustered index fields -@param[in,out] mtr mini-transaction */ -inline -void -page_set_instant(page_t* page, unsigned n, mtr_t* mtr); /**********************************************************//** Create an uncompressed B-tree index page. diff --git a/storage/innobase/include/page0page.ic b/storage/innobase/include/page0page.ic index 307803367c0..7567853667c 100644 --- a/storage/innobase/include/page0page.ic +++ b/storage/innobase/include/page0page.ic @@ -1098,24 +1098,6 @@ page_get_instant(const page_t* page) #endif /* UNIV_DEBUG */ return(i >> 3); } - -/** Assign the PAGE_INSTANT field. -@param[in,out] page clustered index root page -@param[in] n original number of clustered index fields -@param[in,out] mtr mini-transaction */ -inline -void -page_set_instant(page_t* page, unsigned n, mtr_t* mtr) -{ - ut_ad(fil_page_get_type(page) == FIL_PAGE_TYPE_INSTANT); - ut_ad(n > 0); - ut_ad(n < REC_MAX_N_FIELDS); - uint16_t i = page_header_get_field(page, PAGE_INSTANT); - ut_ad(i <= PAGE_NO_DIRECTION); - i |= n << 3; - mlog_write_ulint(PAGE_HEADER + PAGE_INSTANT + page, i, - MLOG_2BYTES, mtr); -} #endif /* !UNIV_INNOCHECKSUM */ #ifdef UNIV_MATERIALIZE diff --git a/storage/innobase/include/page0size.h b/storage/innobase/include/page0size.h index 7b8b7efe617..7c5d3189a97 100644 --- a/storage/innobase/include/page0size.h +++ b/storage/innobase/include/page0size.h @@ -35,7 +35,7 @@ Created Nov 14, 2013 Vasil Dimov /** A BLOB field reference full of zero, for use in assertions and tests.Initially, BLOB field references are set to zero, in dtuple_convert_big_rec(). */ -extern const byte field_ref_zero[FIELD_REF_SIZE]; +extern const byte field_ref_zero[UNIV_PAGE_SIZE_MAX]; #define PAGE_SIZE_T_SIZE_BITS 17 diff --git a/storage/innobase/include/rem0rec.h b/storage/innobase/include/rem0rec.h index a3bd2c8cb50..5cdd3ab49a0 100644 --- a/storage/innobase/include/rem0rec.h +++ b/storage/innobase/include/rem0rec.h @@ -39,15 +39,6 @@ Created 5/30/1994 Heikki Tuuri #include #include -/* Info bit denoting the predefined minimum record: this bit is set -if and only if the record is the first user record on a non-leaf -B-tree page that is the leftmost page on its level -(PAGE_LEVEL is nonzero and FIL_PAGE_PREV is FIL_NULL). */ -#define REC_INFO_MIN_REC_FLAG 0x10UL -/* The deleted flag in info bits */ -#define REC_INFO_DELETED_FLAG 0x20UL /* when bit is set to 1, it means the - record has been delete marked */ - /* Number of extra bytes in an old-style record, in addition to the data and the offsets */ #define REC_N_OLD_EXTRA_BYTES 6 @@ -55,26 +46,6 @@ in addition to the data and the offsets */ in addition to the data and the offsets */ #define REC_N_NEW_EXTRA_BYTES 5 -/** Record status values for ROW_FORMAT=COMPACT,DYNAMIC,COMPRESSED */ -enum rec_comp_status_t { - /** User record (PAGE_LEVEL=0, heap>=PAGE_HEAP_NO_USER_LOW) */ - REC_STATUS_ORDINARY = 0, - /** Node pointer record (PAGE_LEVEL>=0, heap>=PAGE_HEAP_NO_USER_LOW) */ - REC_STATUS_NODE_PTR = 1, - /** The page infimum pseudo-record (heap=PAGE_HEAP_NO_INFIMUM) */ - REC_STATUS_INFIMUM = 2, - /** The page supremum pseudo-record (heap=PAGE_HEAP_NO_SUPREMUM) */ - REC_STATUS_SUPREMUM = 3, - /** Clustered index record that has been inserted or updated - after instant ADD COLUMN (more than dict_index_t::n_core_fields) */ - REC_STATUS_COLUMNS_ADDED = 4 -}; - -/** The dtuple_t::info_bits of the metadata pseudo-record. -@see rec_is_metadata() */ -static const byte REC_INFO_METADATA - = REC_INFO_MIN_REC_FLAG | REC_STATUS_COLUMNS_ADDED; - #define REC_NEW_STATUS 3 /* This is single byte bit-field */ #define REC_NEW_STATUS_MASK 0x7UL #define REC_NEW_STATUS_SHIFT 0 @@ -296,7 +267,7 @@ rec_comp_status_t rec_get_status(const rec_t* rec) { byte bits = rec[-REC_NEW_STATUS] & REC_NEW_STATUS_MASK; - ut_ad(bits <= REC_STATUS_COLUMNS_ADDED); + ut_ad(bits <= REC_STATUS_INSTANT); return static_cast(bits); } @@ -307,12 +278,12 @@ inline void rec_set_status(rec_t* rec, byte bits) { - ut_ad(bits <= REC_STATUS_COLUMNS_ADDED); + ut_ad(bits <= REC_STATUS_INSTANT); rec[-REC_NEW_STATUS] = (rec[-REC_NEW_STATUS] & ~REC_NEW_STATUS_MASK) | bits; } -/** Get the length of added field count in a REC_STATUS_COLUMNS_ADDED record. +/** Get the length of added field count in a REC_STATUS_INSTANT record. @param[in] n_add_field number of added fields, minus one @return storage size of the field count, in bytes */ inline unsigned rec_get_n_add_field_len(ulint n_add_field) @@ -321,8 +292,8 @@ inline unsigned rec_get_n_add_field_len(ulint n_add_field) return n_add_field < 0x80 ? 1 : 2; } -/** Set the added field count in a REC_STATUS_COLUMNS_ADDED record. -@param[in,out] header variable header of a REC_STATUS_COLUMNS_ADDED record +/** Set the added field count in a REC_STATUS_INSTANT record. +@param[in,out] header variable header of a REC_STATUS_INSTANT record @param[in] n_add number of added fields, minus 1 @return record header before the number of added fields */ inline void rec_set_n_add_field(byte*& header, ulint n_add) @@ -781,20 +752,89 @@ rec_offs_comp(const ulint* offsets) } /** Determine if the record is the metadata pseudo-record -in the clustered index. +in the clustered index for instant ADD COLUMN or ALTER TABLE. +@param[in] rec leaf page record +@param[in] comp 0 if ROW_FORMAT=REDUNDANT, else nonzero +@return whether the record is the metadata pseudo-record */ +inline bool rec_is_metadata(const rec_t* rec, ulint comp) +{ + bool is = !!(rec_get_info_bits(rec, comp) & REC_INFO_MIN_REC_FLAG); + ut_ad(!is || !comp || rec_get_status(rec) == REC_STATUS_INSTANT); + return is; +} + +/** Determine if the record is the metadata pseudo-record +in the clustered index for instant ADD COLUMN or ALTER TABLE. @param[in] rec leaf page record @param[in] index index of the record @return whether the record is the metadata pseudo-record */ -inline bool rec_is_metadata(const rec_t* rec, const dict_index_t* index) +inline bool rec_is_metadata(const rec_t* rec, const dict_index_t& index) { - bool is = rec_get_info_bits(rec, dict_table_is_comp(index->table)) - & REC_INFO_MIN_REC_FLAG; - ut_ad(!is || index->is_instant()); - ut_ad(!is || !dict_table_is_comp(index->table) - || rec_get_status(rec) == REC_STATUS_COLUMNS_ADDED); + bool is = rec_is_metadata(rec, dict_table_is_comp(index.table)); + ut_ad(!is || index.is_instant()); return is; } +/** Determine if the record is the metadata pseudo-record +in the clustered index for instant ADD COLUMN (not other ALTER TABLE). +@param[in] rec leaf page record +@param[in] comp 0 if ROW_FORMAT=REDUNDANT, else nonzero +@return whether the record is the metadata pseudo-record */ +inline bool rec_is_add_metadata(const rec_t* rec, ulint comp) +{ + bool is = rec_get_info_bits(rec, comp) == REC_INFO_MIN_REC_FLAG; + ut_ad(!is || !comp || rec_get_status(rec) == REC_STATUS_INSTANT); + return is; +} + +/** Determine if the record is the metadata pseudo-record +in the clustered index for instant ADD COLUMN (not other ALTER TABLE). +@param[in] rec leaf page record +@param[in] index index of the record +@return whether the record is the metadata pseudo-record */ +inline bool rec_is_add_metadata(const rec_t* rec, const dict_index_t& index) +{ + bool is = rec_is_add_metadata(rec, dict_table_is_comp(index.table)); + ut_ad(!is || index.is_instant()); + return is; +} + +/** Determine if the record is the metadata pseudo-record +in the clustered index for instant ALTER TABLE (not plain ADD COLUMN). +@param[in] rec leaf page record +@param[in] comp 0 if ROW_FORMAT=REDUNDANT, else nonzero +@return whether the record is the ALTER TABLE metadata pseudo-record */ +inline bool rec_is_alter_metadata(const rec_t* rec, ulint comp) +{ + bool is = !(~rec_get_info_bits(rec, comp) + & (REC_INFO_MIN_REC_FLAG | REC_INFO_DELETED_FLAG)); + ut_ad(!is || rec_is_metadata(rec, comp)); + return is; +} + +/** Determine if the record is the metadata pseudo-record +in the clustered index for instant ALTER TABLE (not plain ADD COLUMN). +@param[in] rec leaf page record +@param[in] index index of the record +@return whether the record is the ALTER TABLE metadata pseudo-record */ +inline bool rec_is_alter_metadata(const rec_t* rec, const dict_index_t& index) +{ + bool is = rec_is_alter_metadata(rec, dict_table_is_comp(index.table)); + ut_ad(!is || index.is_dummy || index.is_instant()); + return is; +} + +/** Determine if a record is delete-marked (not a metadata pseudo-record). +@param[in] rec record +@param[in] comp nonzero if ROW_FORMAT!=REDUNDANT +@return whether the record is a delete-marked user record */ +inline bool rec_is_delete_marked(const rec_t* rec, ulint comp) +{ + return (rec_get_info_bits(rec, comp) + & (REC_INFO_MIN_REC_FLAG | REC_INFO_DELETED_FLAG)) + == REC_INFO_DELETED_FLAG; +} + /** Get the nth field from an index. @param[in] rec index record @param[in] index index @@ -812,6 +852,7 @@ rec_get_nth_cfield( ulint* len) { ut_ad(rec_offs_validate(rec, index, offsets)); + if (!rec_offs_nth_default(offsets, n)) { return rec_get_nth_field(rec, offsets, n, len); } @@ -958,7 +999,7 @@ rec_copy( @param[in] fields data fields @param[in] n_fields number of data fields @param[out] extra record header size -@param[in] status REC_STATUS_ORDINARY or REC_STATUS_COLUMNS_ADDED +@param[in] status REC_STATUS_ORDINARY or REC_STATUS_INSTANT @return total size, in bytes */ ulint rec_get_converted_size_temp( @@ -975,7 +1016,7 @@ rec_get_converted_size_temp( @param[in,out] offsets offsets to the fields; in: rec_offs_n_fields(offsets) @param[in] n_core number of core fields (index->n_core_fields) @param[in] def_val default values for non-core fields -@param[in] status REC_STATUS_ORDINARY or REC_STATUS_COLUMNS_ADDED */ +@param[in] status REC_STATUS_ORDINARY or REC_STATUS_INSTANT */ void rec_init_offsets_temp( const rec_t* rec, @@ -1002,8 +1043,7 @@ rec_init_offsets_temp( @param[in] index clustered or secondary index @param[in] fields data fields @param[in] n_fields number of data fields -@param[in] status REC_STATUS_ORDINARY or REC_STATUS_COLUMNS_ADDED -*/ +@param[in] status REC_STATUS_ORDINARY or REC_STATUS_INSTANT */ void rec_convert_dtuple_to_temp( rec_t* rec, @@ -1066,21 +1106,20 @@ rec_get_converted_size_comp_prefix( ulint n_fields,/*!< in: number of data fields */ ulint* extra) /*!< out: extra size */ MY_ATTRIBUTE((warn_unused_result, nonnull(1,2))); -/**********************************************************//** -Determines the size of a data tuple in ROW_FORMAT=COMPACT. + +/** Determine the size of a record in ROW_FORMAT=COMPACT. +@param[in] index record descriptor. dict_table_is_comp() + is assumed to hold, even if it doesn't +@param[in] tuple logical record +@param[out] extra extra size @return total size */ ulint rec_get_converted_size_comp( -/*========================*/ - const dict_index_t* index, /*!< in: record descriptor; - dict_table_is_comp() is - assumed to hold, even if - it does not */ - rec_comp_status_t status, /*!< in: status bits of the record */ - const dfield_t* fields, /*!< in: array of data fields */ - ulint n_fields,/*!< in: number of data fields */ - ulint* extra) /*!< out: extra size */ - MY_ATTRIBUTE((nonnull(1,3))); + const dict_index_t* index, + const dtuple_t* tuple, + ulint* extra) + MY_ATTRIBUTE((nonnull(1,2))); + /**********************************************************//** The following function returns the size of a data tuple when converted to a physical record. diff --git a/storage/innobase/include/rem0rec.ic b/storage/innobase/include/rem0rec.ic index 41794582f37..58ac3b73be5 100644 --- a/storage/innobase/include/rem0rec.ic +++ b/storage/innobase/include/rem0rec.ic @@ -67,7 +67,7 @@ most significant bytes and bits are written below less significant. 001=REC_STATUS_NODE_PTR 010=REC_STATUS_INFIMUM 011=REC_STATUS_SUPREMUM - 100=REC_STATUS_COLUMNS_ADDED + 100=REC_STATUS_INSTANT 1xx=reserved 5 bits heap number 4 8 bits heap number @@ -453,7 +453,7 @@ rec_get_n_fields( } switch (rec_get_status(rec)) { - case REC_STATUS_COLUMNS_ADDED: + case REC_STATUS_INSTANT: case REC_STATUS_ORDINARY: return(dict_index_get_n_fields(index)); case REC_STATUS_NODE_PTR: @@ -549,19 +549,6 @@ rec_set_n_owned_new( } } -#ifdef UNIV_DEBUG -/** Check if the info bits are valid. -@param[in] bits info bits to check -@return true if valid */ -inline -bool -rec_info_bits_valid( - ulint bits) -{ - return(0 == (bits & ~(REC_INFO_DELETED_FLAG | REC_INFO_MIN_REC_FLAG))); -} -#endif /* UNIV_DEBUG */ - /******************************************************//** The following function is used to retrieve the info bits of a record. @return info bits */ @@ -575,7 +562,6 @@ rec_get_info_bits( const ulint val = rec_get_bit_field_1( rec, comp ? REC_NEW_INFO_BITS : REC_OLD_INFO_BITS, REC_INFO_BITS_MASK, REC_INFO_BITS_SHIFT); - ut_ad(rec_info_bits_valid(val)); return(val); } @@ -588,7 +574,6 @@ rec_set_info_bits_old( rec_t* rec, /*!< in: old-style physical record */ ulint bits) /*!< in: info bits */ { - ut_ad(rec_info_bits_valid(bits)); rec_set_bit_field_1(rec, bits, REC_OLD_INFO_BITS, REC_INFO_BITS_MASK, REC_INFO_BITS_SHIFT); } @@ -601,7 +586,6 @@ rec_set_info_bits_new( rec_t* rec, /*!< in/out: new-style physical record */ ulint bits) /*!< in: info bits */ { - ut_ad(rec_info_bits_valid(bits)); rec_set_bit_field_1(rec, bits, REC_NEW_INFO_BITS, REC_INFO_BITS_MASK, REC_INFO_BITS_SHIFT); } @@ -894,7 +878,6 @@ rec_get_nth_field_offs( if SQL null; UNIV_SQL_DEFAULT is default value */ { ulint offs; - ulint length; ut_ad(n < rec_offs_n_fields(offsets)); ut_ad(len); @@ -904,7 +887,7 @@ rec_get_nth_field_offs( offs = rec_offs_base(offsets)[n] & REC_OFFS_MASK; } - length = rec_offs_base(offsets)[1 + n]; + ulint length = rec_offs_base(offsets)[1 + n]; if (length & REC_OFFS_SQL_NULL) { length = UNIV_SQL_NULL; @@ -1263,8 +1246,9 @@ rec_offs_data_size( ulint size; ut_ad(rec_offs_validate(NULL, NULL, offsets)); - size = rec_offs_base(offsets)[rec_offs_n_fields(offsets)] - & REC_OFFS_MASK; + + ulint n = rec_offs_n_fields(offsets); + size = rec_offs_base(offsets)[n] & REC_OFFS_MASK; ut_ad(size < srv_page_size); return(size); } @@ -1411,18 +1395,13 @@ rec_get_converted_size( == DICT_FLD__SYS_INDEXES__MERGE_THRESHOLD); } else { ut_ad(dtuple->n_fields >= index->n_core_fields); - ut_ad(dtuple->n_fields <= index->n_fields); + ut_ad(dtuple->n_fields <= index->n_fields + || dtuple->is_alter_metadata()); } #endif if (dict_table_is_comp(index->table)) { - return(rec_get_converted_size_comp( - index, - static_cast( - dtuple->info_bits - & REC_NEW_STATUS_MASK), - dtuple->fields, - dtuple->n_fields, NULL)); + return rec_get_converted_size_comp(index, dtuple, NULL); } data_size = dtuple_get_data_size(dtuple, 0); diff --git a/storage/innobase/include/row0row.h b/storage/innobase/include/row0row.h index 3f8d0e9b254..b46ca6eb3bc 100644 --- a/storage/innobase/include/row0row.h +++ b/storage/innobase/include/row0row.h @@ -77,6 +77,7 @@ row_get_rec_roll_ptr( #define ROW_BUILD_FOR_PURGE 1 /*!< build row for purge. */ #define ROW_BUILD_FOR_UNDO 2 /*!< build row for undo. */ #define ROW_BUILD_FOR_INSERT 3 /*!< build row for insert. */ + /*****************************************************************//** When an insert or purge to a table is performed, this function builds the entry to be inserted into or purged from an index on the table. @@ -230,6 +231,26 @@ row_rec_to_index_entry( mem_heap_t* heap) /*!< in: memory heap from which the memory needed is allocated */ MY_ATTRIBUTE((warn_unused_result)); + +/** Convert a metadata record to a data tuple. +@param[in] rec metadata record +@param[in] index clustered index after instant ALTER TABLE +@param[in] offsets rec_get_offsets(rec) +@param[out] n_ext number of externally stored fields +@param[in,out] heap memory heap for allocations +@param[in] info_bits the info_bits after an update +@param[in] pad whether to pad to index->n_fields */ +dtuple_t* +row_metadata_to_tuple( + const rec_t* rec, + const dict_index_t* index, + const ulint* offsets, + ulint* n_ext, + mem_heap_t* heap, + ulint info_bits, + bool pad) + MY_ATTRIBUTE((nonnull,warn_unused_result)); + /*******************************************************************//** Builds from a secondary index record a row reference with which we can search the clustered index record. diff --git a/storage/innobase/include/row0upd.h b/storage/innobase/include/row0upd.h index 5e01e513a50..db8035cd8b8 100644 --- a/storage/innobase/include/row0upd.h +++ b/storage/innobase/include/row0upd.h @@ -490,6 +490,14 @@ struct upd_t{ return false; } + /** @return whether this is for a hidden metadata record + for instant ALTER TABLE */ + bool is_metadata() const { return dtuple_t::is_metadata(info_bits); } + /** @return whether this is for a hidden metadata record + for instant ALTER TABLE (not only ADD COLUMN) */ + bool is_alter_metadata() const + { return dtuple_t::is_alter_metadata(info_bits); } + #ifdef UNIV_DEBUG bool validate() const { @@ -503,7 +511,6 @@ struct upd_t{ return(true); } #endif // UNIV_DEBUG - }; /** Kinds of update operation */ diff --git a/storage/innobase/lock/lock0lock.cc b/storage/innobase/lock/lock0lock.cc index c19797ab9da..5dc437dbcd7 100644 --- a/storage/innobase/lock/lock0lock.cc +++ b/storage/innobase/lock/lock0lock.cc @@ -337,7 +337,7 @@ lock_report_trx_id_insanity( trx_id_t max_trx_id) /*!< in: trx_sys.get_max_trx_id() */ { ut_ad(rec_offs_validate(rec, index, offsets)); - ut_ad(!rec_is_metadata(rec, index)); + ut_ad(!rec_is_metadata(rec, *index)); ib::error() << "Transaction id " << trx_id @@ -360,7 +360,7 @@ lock_check_trx_id_sanity( const ulint* offsets) /*!< in: rec_get_offsets(rec, index) */ { ut_ad(rec_offs_validate(rec, index, offsets)); - ut_ad(!rec_is_metadata(rec, index)); + ut_ad(!rec_is_metadata(rec, *index)); trx_id_t max_trx_id = trx_sys.get_max_trx_id(); ut_ad(max_trx_id || srv_force_recovery >= SRV_FORCE_NO_UNDO_LOG_SCAN); @@ -389,7 +389,7 @@ lock_clust_rec_cons_read_sees( ut_ad(dict_index_is_clust(index)); ut_ad(page_rec_is_user_rec(rec)); ut_ad(rec_offs_validate(rec, index, offsets)); - ut_ad(!rec_is_metadata(rec, index)); + ut_ad(!rec_is_metadata(rec, *index)); /* Temp-tables are not shared across connections and multiple transactions from different connections cannot simultaneously @@ -428,7 +428,7 @@ lock_sec_rec_cons_read_sees( { ut_ad(page_rec_is_user_rec(rec)); ut_ad(!index->is_primary()); - ut_ad(!rec_is_metadata(rec, index)); + ut_ad(!rec_is_metadata(rec, *index)); /* NOTE that we might call this function while holding the search system latch. */ @@ -1222,7 +1222,7 @@ lock_sec_rec_some_has_impl( ut_ad(!dict_index_is_clust(index)); ut_ad(page_rec_is_user_rec(rec)); ut_ad(rec_offs_validate(rec, index, offsets)); - ut_ad(!rec_is_metadata(rec, index)); + ut_ad(!rec_is_metadata(rec, *index)); max_trx_id = page_get_max_trx_id(page); @@ -5312,7 +5312,7 @@ lock_rec_insert_check_and_lock( trx_t* trx = thr_get_trx(thr); const rec_t* next_rec = page_rec_get_next_const(rec); ulint heap_no = page_rec_get_heap_no(next_rec); - ut_ad(!rec_is_metadata(next_rec, index)); + ut_ad(!rec_is_metadata(next_rec, *index)); lock_mutex_enter(); /* Because this code is invoked for a running transaction by @@ -5440,7 +5440,7 @@ lock_rec_convert_impl_to_expl_for_trx( { ut_ad(trx->is_referenced()); ut_ad(page_rec_is_leaf(rec)); - ut_ad(!rec_is_metadata(rec, index)); + ut_ad(!rec_is_metadata(rec, *index)); DEBUG_SYNC_C("before_lock_rec_convert_impl_to_expl_for_trx"); @@ -5564,7 +5564,7 @@ lock_rec_convert_impl_to_expl( ut_ad(rec_offs_validate(rec, index, offsets)); ut_ad(!page_rec_is_comp(rec) == !rec_offs_comp(offsets)); ut_ad(page_rec_is_leaf(rec)); - ut_ad(!rec_is_metadata(rec, index)); + ut_ad(!rec_is_metadata(rec, *index)); if (dict_index_is_clust(index)) { trx_id_t trx_id; @@ -5641,7 +5641,7 @@ lock_clust_rec_modify_check_and_lock( return(DB_SUCCESS); } - ut_ad(!rec_is_metadata(rec, index)); + ut_ad(!rec_is_metadata(rec, *index)); ut_ad(!index->table->is_temporary()); heap_no = rec_offs_comp(offsets) @@ -5697,7 +5697,7 @@ lock_sec_rec_modify_check_and_lock( ut_ad(block->frame == page_align(rec)); ut_ad(mtr->is_named_space(index->table->space)); ut_ad(page_rec_is_leaf(rec)); - ut_ad(!rec_is_metadata(rec, index)); + ut_ad(!rec_is_metadata(rec, *index)); if (flags & BTR_NO_LOCKING_FLAG) { @@ -5791,7 +5791,7 @@ lock_sec_rec_read_check_and_lock( return(DB_SUCCESS); } - ut_ad(!rec_is_metadata(rec, index)); + ut_ad(!rec_is_metadata(rec, *index)); heap_no = page_rec_get_heap_no(rec); /* Some transaction may have an implicit x-lock on the record only @@ -5853,7 +5853,7 @@ lock_clust_rec_read_check_and_lock( || gap_mode == LOCK_REC_NOT_GAP); ut_ad(rec_offs_validate(rec, index, offsets)); ut_ad(page_rec_is_leaf(rec)); - ut_ad(!rec_is_metadata(rec, index)); + ut_ad(!rec_is_metadata(rec, *index)); if ((flags & BTR_NO_LOCKING_FLAG) || srv_read_only_mode diff --git a/storage/innobase/page/page0cur.cc b/storage/innobase/page/page0cur.cc index e9459a42c7b..a797e3688ec 100644 --- a/storage/innobase/page/page0cur.cc +++ b/storage/innobase/page/page0cur.cc @@ -737,7 +737,7 @@ up_slot_match: & REC_INFO_MIN_REC_FLAG)) { ut_ad(!page_has_prev(page_align(mid_rec))); ut_ad(!page_rec_is_leaf(mid_rec) - || rec_is_metadata(mid_rec, index)); + || rec_is_metadata(mid_rec, *index)); cmp = 1; goto low_rec_match; } @@ -1370,7 +1370,7 @@ use_heap: switch (rec_get_status(current_rec)) { case REC_STATUS_ORDINARY: case REC_STATUS_NODE_PTR: - case REC_STATUS_COLUMNS_ADDED: + case REC_STATUS_INSTANT: case REC_STATUS_INFIMUM: break; case REC_STATUS_SUPREMUM: @@ -1379,7 +1379,7 @@ use_heap: switch (rec_get_status(insert_rec)) { case REC_STATUS_ORDINARY: case REC_STATUS_NODE_PTR: - case REC_STATUS_COLUMNS_ADDED: + case REC_STATUS_INSTANT: break; case REC_STATUS_INFIMUM: case REC_STATUS_SUPREMUM: diff --git a/storage/innobase/page/page0page.cc b/storage/innobase/page/page0page.cc index 3f06ee0e1bc..0d0d3c211e5 100644 --- a/storage/innobase/page/page0page.cc +++ b/storage/innobase/page/page0page.cc @@ -1804,6 +1804,7 @@ page_print_list( count = 0; for (;;) { offsets = rec_get_offsets(cur.rec, index, offsets, + page_rec_is_leaf(cur.rec), ULINT_UNDEFINED, &heap); page_rec_print(cur.rec, offsets); @@ -1826,6 +1827,7 @@ page_print_list( if (count + pr_n >= n_recs) { offsets = rec_get_offsets(cur.rec, index, offsets, + page_rec_is_leaf(cur.rec), ULINT_UNDEFINED, &heap); page_rec_print(cur.rec, offsets); } diff --git a/storage/innobase/page/page0zip.cc b/storage/innobase/page/page0zip.cc index d0845a7f640..4b1ab637c13 100644 --- a/storage/innobase/page/page0zip.cc +++ b/storage/innobase/page/page0zip.cc @@ -31,12 +31,7 @@ Created June 2005 by Marko Makela /** A BLOB field reference full of zero, for use in assertions and tests. Initially, BLOB field references are set to zero, in dtuple_convert_big_rec(). */ -const byte field_ref_zero[FIELD_REF_SIZE] = { - 0, 0, 0, 0, 0, - 0, 0, 0, 0, 0, - 0, 0, 0, 0, 0, - 0, 0, 0, 0, 0, -}; +const byte field_ref_zero[UNIV_PAGE_SIZE_MAX] = { 0, }; #ifndef UNIV_INNOCHECKSUM #include "page0page.h" @@ -105,11 +100,11 @@ Compare at most sizeof(field_ref_zero) bytes. @param s in: size of the memory block, in bytes */ #define ASSERT_ZERO(b, s) \ ut_ad(!memcmp(b, field_ref_zero, \ - ut_min(static_cast(s), sizeof field_ref_zero))); + std::min(s, sizeof field_ref_zero))); /** Assert that a BLOB pointer is filled with zero bytes. @param b in: BLOB pointer */ #define ASSERT_ZERO_BLOB(b) \ - ut_ad(!memcmp(b, field_ref_zero, sizeof field_ref_zero)) + ut_ad(!memcmp(b, field_ref_zero, FIELD_REF_SIZE)) /* Enable some extra debugging output. This code can be enabled independently of any UNIV_ debugging conditions. */ @@ -2130,6 +2125,10 @@ page_zip_apply_log( rec_get_offsets_reverse(data, index, hs & REC_STATUS_NODE_PTR, offsets); + /* Silence a debug assertion in rec_offs_make_valid(). + This will be overwritten in page_zip_set_extra_bytes(), + called by page_zip_decompress_low(). */ + ut_d(rec[-REC_NEW_INFO_BITS] = 0); rec_offs_make_valid(rec, index, is_leaf, offsets); /* Copy the extra bytes (backwards). */ diff --git a/storage/innobase/rem/rem0rec.cc b/storage/innobase/rem/rem0rec.cc index 2633133df5a..c473bb8ff64 100644 --- a/storage/innobase/rem/rem0rec.cc +++ b/storage/innobase/rem/rem0rec.cc @@ -35,6 +35,7 @@ Created 5/30/1994 Heikki Tuuri #include "gis0geo.h" #include "trx0sys.h" #include "mach0data.h" +#include "btr0cur.h" /* PHYSICAL RECORD (OLD STYLE) =========================== @@ -175,7 +176,7 @@ rec_get_n_extern_new( ut_ad(!index->table->supports_instant() || index->is_dummy); ut_ad(!index->is_instant()); ut_ad(rec_get_status(rec) == REC_STATUS_ORDINARY - || rec_get_status(rec) == REC_STATUS_COLUMNS_ADDED); + || rec_get_status(rec) == REC_STATUS_INSTANT); ut_ad(n == ULINT_UNDEFINED || n <= dict_index_get_n_fields(index)); if (n == ULINT_UNDEFINED) { @@ -237,8 +238,8 @@ rec_get_n_extern_new( return(n_extern); } -/** Get the added field count in a REC_STATUS_COLUMNS_ADDED record. -@param[in,out] header variable header of a REC_STATUS_COLUMNS_ADDED record +/** Get the added field count in a REC_STATUS_INSTANT record. +@param[in,out] header variable header of a REC_STATUS_INSTANT record @return number of added fields */ static inline unsigned rec_get_n_add_field(const byte*& header) { @@ -259,18 +260,18 @@ static inline unsigned rec_get_n_add_field(const byte*& header) enum rec_leaf_format { /** Temporary file record */ REC_LEAF_TEMP, - /** Temporary file record, with added columns - (REC_STATUS_COLUMNS_ADDED) */ - REC_LEAF_TEMP_COLUMNS_ADDED, + /** Temporary file record, with added columns (REC_STATUS_INSTANT) */ + REC_LEAF_TEMP_INSTANT, /** Normal (REC_STATUS_ORDINARY) */ REC_LEAF_ORDINARY, - /** With added columns (REC_STATUS_COLUMNS_ADDED) */ - REC_LEAF_COLUMNS_ADDED + /** With add or drop columns (REC_STATUS_INSTANT) */ + REC_LEAF_INSTANT }; /** Determine the offset to each field in a leaf-page record in ROW_FORMAT=COMPACT,DYNAMIC,COMPRESSED. This is a special case of rec_init_offsets() and rec_get_offsets_func(). +@tparam mblob whether the record includes a metadata BLOB @param[in] rec leaf-page record @param[in] index the index that the record belongs in @param[in] n_core number of core fields (index->n_core_fields) @@ -278,6 +279,7 @@ This is a special case of rec_init_offsets() and rec_get_offsets_func(). NULL to refer to index->fields[].col->def_val @param[in,out] offsets offsets, with valid rec_offs_n_fields(offsets) @param[in] format record format */ +template static inline void rec_init_offsets_comp_ordinary( @@ -299,12 +301,32 @@ rec_init_offsets_comp_ordinary( ut_ad(n_core > 0); ut_ad(index->n_fields >= n_core); ut_ad(index->n_core_null_bytes <= UT_BITS_IN_BYTES(index->n_nullable)); - ut_ad(format == REC_LEAF_TEMP || format == REC_LEAF_TEMP_COLUMNS_ADDED + ut_ad(format == REC_LEAF_TEMP || format == REC_LEAF_TEMP_INSTANT || dict_table_is_comp(index->table)); - ut_ad(format != REC_LEAF_TEMP_COLUMNS_ADDED + ut_ad(format != REC_LEAF_TEMP_INSTANT || index->n_fields == rec_offs_n_fields(offsets)); ut_d(ulint n_null= 0); + if (mblob) { + ut_ad(index->is_dummy || index->table->instant); + ut_ad(index->is_dummy || index->is_instant()); + ut_ad(rec_offs_n_fields(offsets) + <= ulint(index->n_fields) + 1); + ut_ad(!def_val); + ut_ad(format == REC_LEAF_INSTANT); + nulls -= REC_N_NEW_EXTRA_BYTES; + n_fields = n_core + 1 + rec_get_n_add_field(nulls); + ut_ad(n_fields <= ulint(index->n_fields) + 1); + const ulint n_nullable = index->get_n_nullable(n_fields - 1); + const ulint n_null_bytes = UT_BITS_IN_BYTES(n_nullable); + ut_d(n_null = n_nullable); + ut_ad(n_null <= index->n_nullable); + ut_ad(n_null_bytes >= index->n_core_null_bytes + || n_core < index->n_core_fields); + lens = --nulls - n_null_bytes; + goto start; + } + switch (format) { case REC_LEAF_TEMP: if (dict_table_is_comp(index->table)) { @@ -318,17 +340,15 @@ rec_init_offsets_comp_ordinary( ordinary: lens = --nulls - index->n_core_null_bytes; - ut_d(n_null = std::min(index->n_core_null_bytes * 8U, - index->n_nullable)); + ut_d(n_null = std::min(index->n_core_null_bytes * 8U, + index->n_nullable)); break; - case REC_LEAF_COLUMNS_ADDED: - /* We would have !index->is_instant() when rolling back - an instant ADD COLUMN operation. */ + case REC_LEAF_INSTANT: nulls -= REC_N_NEW_EXTRA_BYTES; ut_ad(index->is_instant()); /* fall through */ - case REC_LEAF_TEMP_COLUMNS_ADDED: - n_fields = n_core + 1 + rec_get_n_add_field(nulls); + case REC_LEAF_TEMP_INSTANT: + n_fields = n_core + rec_get_n_add_field(nulls) + 1; ut_ad(n_fields <= index->n_fields); const ulint n_nullable = index->get_n_nullable(n_fields); const ulint n_null_bytes = UT_BITS_IN_BYTES(n_nullable); @@ -339,26 +359,34 @@ ordinary: lens = --nulls - n_null_bytes; } -#ifdef UNIV_DEBUG +start: /* We cannot invoke rec_offs_make_valid() if format==REC_LEAF_TEMP. Similarly, rec_offs_validate() will fail in that case, because it invokes rec_get_status(). */ - offsets[2] = (ulint) rec; - offsets[3] = (ulint) index; -#endif /* UNIV_DEBUG */ + ut_d(offsets[2] = ulint(rec)); + ut_d(offsets[3] = ulint(index)); /* read the lengths of fields 0..n_fields */ + ulint len; ulint i = 0; - do { - const dict_field_t* field - = dict_index_get_nth_field(index, i); - const dict_col_t* col - = dict_field_get_col(field); - ulint len; + const dict_field_t* field = index->fields; - /* set default value flag */ - if (i < n_fields) { - } else if (def_val) { + do { + if (mblob) { + if (i == index->first_user_field()) { + offs += FIELD_REF_SIZE; + len = offs | REC_OFFS_EXTERNAL; + any |= REC_OFFS_EXTERNAL; + field--; + continue; + } else if (i >= n_fields) { + len = offs | REC_OFFS_DEFAULT; + any |= REC_OFFS_DEFAULT; + continue; + } + } else if (i < n_fields) { + /* The field is present, and will be covered below. */ + } else if (!mblob && def_val) { const dict_col_t::def_t& d = def_val[i - n_core]; if (!d.data) { len = offs | REC_OFFS_SQL_NULL; @@ -368,21 +396,22 @@ ordinary: any |= REC_OFFS_DEFAULT; } - goto resolved; + continue; } else { - ulint dlen; - if (!index->instant_field_value(i, &dlen)) { + if (!index->instant_field_value(i, &len)) { + ut_ad(len == UNIV_SQL_NULL); len = offs | REC_OFFS_SQL_NULL; - ut_ad(dlen == UNIV_SQL_NULL); } else { len = offs | REC_OFFS_DEFAULT; any |= REC_OFFS_DEFAULT; } - goto resolved; + continue; } - if (!(col->prtype & DATA_NOT_NULL)) { + const dict_col_t* col = field->col; + + if (col->is_nullable()) { /* nullable field => read the null flag */ ut_ad(n_null--); @@ -398,7 +427,7 @@ ordinary: the length to zero and enable the SQL NULL flag in offsets[]. */ len = offs | REC_OFFS_SQL_NULL; - goto resolved; + continue; } null_mask <<= 1; } @@ -429,16 +458,15 @@ ordinary: len = offs; } - goto resolved; + continue; } len = offs += len; } else { len = offs += field->fixed_len; } -resolved: - rec_offs_base(offsets)[i + 1] = len; - } while (++i < rec_offs_n_fields(offsets)); + } while (field++, rec_offs_base(offsets)[++i] = len, + i < rec_offs_n_fields(offsets)); *rec_offs_base(offsets) = ulint(rec - (lens + 1)) | REC_OFFS_COMPACT | any; @@ -457,7 +485,10 @@ rec_offs_make_valid( bool leaf, ulint* offsets) { - ut_ad(rec_offs_n_fields(offsets) + const bool is_alter_metadata = leaf + && rec_is_alter_metadata(rec, *index); + ut_ad(is_alter_metadata + || rec_offs_n_fields(offsets) <= (leaf ? dict_index_get_n_fields(index) : dict_index_get_n_unique_in_tree_nonleaf(index) + 1) @@ -475,7 +506,8 @@ rec_offs_make_valid( || n >= rec_offs_n_fields(offsets)); for (; n < rec_offs_n_fields(offsets); n++) { ut_ad(leaf); - ut_ad(rec_offs_base(offsets)[1 + n] & REC_OFFS_DEFAULT); + ut_ad(is_alter_metadata + || rec_offs_base(offsets)[1 + n] & REC_OFFS_DEFAULT); } offsets[2] = ulint(rec); offsets[3] = ulint(index); @@ -515,14 +547,18 @@ rec_offs_validate( } } if (index) { - ulint max_n_fields; ut_ad(ulint(index) == offsets[3]); - max_n_fields = ut_max( + ulint max_n_fields = ut_max( dict_index_get_n_fields(index), dict_index_get_n_unique_in_tree(index) + 1); if (comp && rec) { switch (rec_get_status(rec)) { - case REC_STATUS_COLUMNS_ADDED: + case REC_STATUS_INSTANT: + ut_ad(index->is_instant() || index->is_dummy); + ut_ad(max_n_fields == index->n_fields); + max_n_fields += index->table->instant + || index->is_dummy; + break; case REC_STATUS_ORDINARY: break; case REC_STATUS_NODE_PTR: @@ -536,14 +572,19 @@ rec_offs_validate( default: ut_error; } + } else if (max_n_fields == index->n_fields + && (index->is_dummy + || (index->is_instant() + && index->table->instant))) { + max_n_fields++; } /* index->n_def == 0 for dummy indexes if !comp */ - ut_a(!comp || index->n_def); - ut_a(!index->n_def || i <= max_n_fields); + ut_ad(!comp || index->n_def); + ut_ad(!index->n_def || i <= max_n_fields); } while (i--) { ulint curr = rec_offs_base(offsets)[1 + i] & REC_OFFS_MASK; - ut_a(curr <= last); + ut_ad(curr <= last); last = curr; } return(TRUE); @@ -604,12 +645,12 @@ rec_init_offsets( = dict_index_get_n_unique_in_tree_nonleaf( index); break; - case REC_STATUS_COLUMNS_ADDED: + case REC_STATUS_INSTANT: ut_ad(leaf); rec_init_offsets_comp_ordinary(rec, index, offsets, index->n_core_fields, NULL, - REC_LEAF_COLUMNS_ADDED); + REC_LEAF_INSTANT); return; case REC_STATUS_ORDINARY: ut_ad(leaf); @@ -786,6 +827,7 @@ rec_get_offsets_func( { ulint n; ulint size; + bool alter_metadata = false; ut_ad(rec); ut_ad(index); @@ -794,10 +836,12 @@ rec_get_offsets_func( if (dict_table_is_comp(index->table)) { switch (UNIV_EXPECT(rec_get_status(rec), REC_STATUS_ORDINARY)) { - case REC_STATUS_COLUMNS_ADDED: + case REC_STATUS_INSTANT: + alter_metadata = rec_is_alter_metadata(rec, true); + /* fall through */ case REC_STATUS_ORDINARY: ut_ad(leaf); - n = dict_index_get_n_fields(index); + n = dict_index_get_n_fields(index) + alter_metadata; break; case REC_STATUS_NODE_PTR: /* Node pointer records consist of the @@ -841,7 +885,8 @@ rec_get_offsets_func( || dict_index_is_ibuf(index) || n == n_fields /* btr_pcur_restore_position() */ || (n + (index->id == DICT_INDEXES_ID) - >= index->n_core_fields && n <= index->n_fields)); + >= index->n_core_fields && n <= index->n_fields + + unsigned(rec_is_alter_metadata(rec, false)))); if (is_user_rec && leaf && n < index->n_fields) { ut_ad(!index->is_dummy); @@ -871,8 +916,24 @@ rec_get_offsets_func( } rec_offs_set_n_fields(offsets, n); - rec_init_offsets(rec, index, leaf, offsets); - return(offsets); + + if (UNIV_UNLIKELY(alter_metadata) + && dict_table_is_comp(index->table)) { + ut_d(offsets[2] = ulint(rec)); + ut_d(offsets[3] = ulint(index)); + ut_ad(leaf); + ut_ad(index->is_dummy || index->table->instant); + ut_ad(index->is_dummy || index->is_instant()); + ut_ad(rec_offs_n_fields(offsets) + <= ulint(index->n_fields) + 1); + rec_init_offsets_comp_ordinary(rec, index, offsets, + index->n_core_fields, + NULL, + REC_LEAF_INSTANT); + } else { + rec_init_offsets(rec, index, leaf, offsets); + } + return offsets; } /******************************************************//** @@ -1050,36 +1111,45 @@ rec_get_nth_field_offs_old( return(os); } -/**********************************************************//** -Determines the size of a data tuple prefix in ROW_FORMAT=COMPACT. +/** Determine the size of a data tuple prefix in ROW_FORMAT=COMPACT. +@tparam mblob whether the record includes a metadata BLOB +@param[in] index record descriptor; dict_table_is_comp() + is assumed to hold, even if it doesn't +@param[in] dfield array of data fields +@param[in] n_fields number of data fields +@param[out] extra extra size +@param[in] status status flags +@param[in] temp whether this is a temporary file record @return total size */ -MY_ATTRIBUTE((warn_unused_result, nonnull(1,2))) +template static inline ulint rec_get_converted_size_comp_prefix_low( -/*===================================*/ - const dict_index_t* index, /*!< in: record descriptor; - dict_table_is_comp() is - assumed to hold, even if - it does not */ - const dfield_t* fields, /*!< in: array of data fields */ - ulint n_fields,/*!< in: number of data fields */ - ulint* extra, /*!< out: extra size */ - rec_comp_status_t status, /*!< in: status flags */ - bool temp) /*!< in: whether this is a - temporary file record */ + const dict_index_t* index, + const dfield_t* dfield, + ulint n_fields, + ulint* extra, + rec_comp_status_t status, + bool temp) { ulint extra_size = temp ? 0 : REC_N_NEW_EXTRA_BYTES; - ulint data_size; - ulint i; ut_ad(n_fields > 0); - ut_ad(n_fields <= dict_index_get_n_fields(index)); + ut_ad(n_fields <= dict_index_get_n_fields(index) + mblob); ut_d(ulint n_null = index->n_nullable); ut_ad(status == REC_STATUS_ORDINARY || status == REC_STATUS_NODE_PTR - || status == REC_STATUS_COLUMNS_ADDED); + || status == REC_STATUS_INSTANT); - if (status == REC_STATUS_COLUMNS_ADDED - && (!temp || n_fields > index->n_core_fields)) { + if (mblob) { + ut_ad(!temp); + ut_ad(index->table->instant); + ut_ad(index->is_instant()); + ut_ad(status == REC_STATUS_INSTANT); + ut_ad(n_fields == ulint(index->n_fields) + 1); + extra_size += UT_BITS_IN_BYTES(index->n_nullable) + + rec_get_n_add_field_len(n_fields - 1 + - index->n_core_fields); + } else if (status == REC_STATUS_INSTANT + && (!temp || n_fields > index->n_core_fields)) { ut_ad(index->is_instant()); ut_ad(UT_BITS_IN_BYTES(n_null) >= index->n_core_null_bytes); extra_size += UT_BITS_IN_BYTES(index->get_n_nullable(n_fields)) @@ -1090,7 +1160,7 @@ rec_get_converted_size_comp_prefix_low( extra_size += index->n_core_null_bytes; } - data_size = 0; + ulint data_size = 0; if (temp && dict_table_is_comp(index->table)) { /* No need to do adjust fixed_len=0. We only need to @@ -1098,48 +1168,50 @@ rec_get_converted_size_comp_prefix_low( temp = false; } + const dfield_t* const end = dfield + n_fields; /* read the lengths of fields 0..n */ - for (i = 0; i < n_fields; i++) { - const dict_field_t* field; - ulint len; - ulint fixed_len; - const dict_col_t* col; + for (ulint i = 0; dfield < end; i++, dfield++) { + if (mblob && i == index->first_user_field()) { + data_size += FIELD_REF_SIZE; + ++dfield; + } - field = dict_index_get_nth_field(index, i); - len = dfield_get_len(&fields[i]); - col = dict_field_get_col(field); + ulint len = dfield_get_len(dfield); + const dict_field_t* field = dict_index_get_nth_field(index, i); #ifdef UNIV_DEBUG - dtype_t* type; - - type = dfield_get_type(&fields[i]); if (dict_index_is_spatial(index)) { - if (DATA_GEOMETRY_MTYPE(col->mtype) && i == 0) { - ut_ad(type->prtype & DATA_GIS_MBR); + if (DATA_GEOMETRY_MTYPE(field->col->mtype) && i == 0) { + ut_ad(dfield->type.prtype & DATA_GIS_MBR); } else { - ut_ad(type->mtype == DATA_SYS_CHILD - || dict_col_type_assert_equal(col, type)); + ut_ad(dfield->type.mtype == DATA_SYS_CHILD + || dict_col_type_assert_equal( + field->col, &dfield->type)); } } else { - ut_ad(dict_col_type_assert_equal(col, type)); + ut_ad(field->col->is_dropped() + || dict_col_type_assert_equal(field->col, + &dfield->type)); } #endif /* All NULLable fields must be included in the n_null count. */ - ut_ad((col->prtype & DATA_NOT_NULL) || n_null--); + ut_ad(!field->col->is_nullable() || n_null--); - if (dfield_is_null(&fields[i])) { + if (dfield_is_null(dfield)) { /* No length is stored for NULL fields. */ - ut_ad(!(col->prtype & DATA_NOT_NULL)); + ut_ad(field->col->is_nullable()); continue; } - ut_ad(len <= col->len || DATA_LARGE_MTYPE(col->mtype) - || (col->len == 0 && col->mtype == DATA_VARCHAR)); + ut_ad(len <= field->col->len + || DATA_LARGE_MTYPE(field->col->mtype) + || (field->col->len == 0 + && field->col->mtype == DATA_VARCHAR)); - fixed_len = field->fixed_len; + ulint fixed_len = field->fixed_len; if (temp && fixed_len - && !dict_col_get_fixed_size(col, temp)) { + && !dict_col_get_fixed_size(field->col, temp)) { fixed_len = 0; } /* If the maximum length of a variable-length field @@ -1154,25 +1226,27 @@ rec_get_converted_size_comp_prefix_low( ut_ad(len <= fixed_len); if (dict_index_is_spatial(index)) { - ut_ad(type->mtype == DATA_SYS_CHILD - || !col->mbmaxlen - || len >= col->mbminlen - * fixed_len / col->mbmaxlen); + ut_ad(dfield->type.mtype == DATA_SYS_CHILD + || !field->col->mbmaxlen + || len >= field->col->mbminlen + * fixed_len / field->col->mbmaxlen); } else { - ut_ad(type->mtype != DATA_SYS_CHILD); - ut_ad(!col->mbmaxlen - || len >= col->mbminlen - * fixed_len / col->mbmaxlen); + ut_ad(dfield->type.mtype != DATA_SYS_CHILD); + + ut_ad(field->col->is_dropped() + || !field->col->mbmaxlen + || len >= field->col->mbminlen + * fixed_len / field->col->mbmaxlen); } /* dict_index_add_col() should guarantee this */ ut_ad(!field->prefix_len || fixed_len == field->prefix_len); #endif /* UNIV_DEBUG */ - } else if (dfield_is_ext(&fields[i])) { - ut_ad(DATA_BIG_COL(col)); + } else if (dfield_is_ext(dfield)) { + ut_ad(DATA_BIG_COL(field->col)); extra_size += 2; - } else if (len < 128 || !DATA_BIG_COL(col)) { + } else if (len < 128 || !DATA_BIG_COL(field->col)) { extra_size++; } else { /* For variable-length columns, we look up the @@ -1208,43 +1282,51 @@ rec_get_converted_size_comp_prefix( REC_STATUS_ORDINARY, false)); } -/**********************************************************//** -Determines the size of a data tuple in ROW_FORMAT=COMPACT. +/** Determine the size of a record in ROW_FORMAT=COMPACT. +@param[in] index record descriptor. dict_table_is_comp() + is assumed to hold, even if it doesn't +@param[in] tuple logical record +@param[out] extra extra size @return total size */ ulint rec_get_converted_size_comp( -/*========================*/ - const dict_index_t* index, /*!< in: record descriptor; - dict_table_is_comp() is - assumed to hold, even if - it does not */ - rec_comp_status_t status, /*!< in: status bits of the record */ - const dfield_t* fields, /*!< in: array of data fields */ - ulint n_fields,/*!< in: number of data fields */ - ulint* extra) /*!< out: extra size */ + const dict_index_t* index, + const dtuple_t* tuple, + ulint* extra) { - ut_ad(n_fields > 0); + ut_ad(tuple->n_fields > 0); + + rec_comp_status_t status = rec_comp_status_t(tuple->info_bits + & REC_NEW_STATUS_MASK); switch (UNIV_EXPECT(status, REC_STATUS_ORDINARY)) { case REC_STATUS_ORDINARY: - if (n_fields > index->n_core_fields) { + ut_ad(!tuple->is_metadata()); + if (tuple->n_fields > index->n_core_fields) { ut_ad(index->is_instant()); - status = REC_STATUS_COLUMNS_ADDED; + status = REC_STATUS_INSTANT; } /* fall through */ - case REC_STATUS_COLUMNS_ADDED: - ut_ad(n_fields >= index->n_core_fields); - ut_ad(n_fields <= index->n_fields); + case REC_STATUS_INSTANT: + ut_ad(tuple->n_fields >= index->n_core_fields); + if (tuple->is_alter_metadata()) { + return rec_get_converted_size_comp_prefix_low( + index, tuple->fields, tuple->n_fields, + extra, status, false); + } + ut_ad(tuple->n_fields <= index->n_fields); return rec_get_converted_size_comp_prefix_low( - index, fields, n_fields, extra, status, false); + index, tuple->fields, tuple->n_fields, + extra, status, false); case REC_STATUS_NODE_PTR: - n_fields--; - ut_ad(n_fields == dict_index_get_n_unique_in_tree_nonleaf( - index)); - ut_ad(dfield_get_len(&fields[n_fields]) == REC_NODE_PTR_SIZE); + ut_ad(tuple->n_fields - 1 + == dict_index_get_n_unique_in_tree_nonleaf(index)); + ut_ad(dfield_get_len(&tuple->fields[tuple->n_fields - 1]) + == REC_NODE_PTR_SIZE); return REC_NODE_PTR_SIZE /* child page number */ + rec_get_converted_size_comp_prefix_low( - index, fields, n_fields, extra, status, false); + index, tuple->fields, tuple->n_fields - 1, + extra, status, false); case REC_STATUS_INFIMUM: case REC_STATUS_SUPREMUM: /* not supported */ @@ -1422,33 +1504,30 @@ rec_convert_dtuple_to_rec_old( } /** Convert a data tuple into a ROW_FORMAT=COMPACT record. +@tparam mblob whether the record includes a metadata BLOB @param[out] rec converted record @param[in] index index -@param[in] fields data fields to convert +@param[in] field data fields to convert @param[in] n_fields number of data fields @param[in] status rec_get_status(rec) @param[in] temp whether to use the format for temporary files in index creation */ +template static inline void rec_convert_dtuple_to_rec_comp( rec_t* rec, const dict_index_t* index, - const dfield_t* fields, + const dfield_t* field, ulint n_fields, rec_comp_status_t status, bool temp) { - const dfield_t* field; - const dtype_t* type; byte* end; byte* nulls = temp ? rec - 1 : rec - (REC_N_NEW_EXTRA_BYTES + 1); byte* UNINIT_VAR(lens); - ulint len; - ulint i; ulint UNINIT_VAR(n_node_ptr_field); - ulint fixed_len; ulint null_mask = 1; ut_ad(n_fields > 0); @@ -1457,8 +1536,22 @@ rec_convert_dtuple_to_rec_comp( ut_d(ulint n_null = index->n_nullable); + if (mblob) { + ut_ad(!temp); + ut_ad(index->table->instant); + ut_ad(index->is_instant()); + ut_ad(status == REC_STATUS_INSTANT); + ut_ad(n_fields == ulint(index->n_fields) + 1); + rec_set_n_add_field(nulls, n_fields - 1 + - index->n_core_fields); + rec_set_heap_no_new(rec, PAGE_HEAP_NO_USER_LOW); + rec_set_status(rec, REC_STATUS_INSTANT); + n_node_ptr_field = ULINT_UNDEFINED; + lens = nulls - UT_BITS_IN_BYTES(index->n_nullable); + goto start; + } switch (status) { - case REC_STATUS_COLUMNS_ADDED: + case REC_STATUS_INSTANT: ut_ad(index->is_instant()); ut_ad(n_fields > index->n_core_fields); rec_set_n_add_field(nulls, n_fields - 1 @@ -1468,19 +1561,24 @@ rec_convert_dtuple_to_rec_comp( ut_ad(n_fields <= dict_index_get_n_fields(index)); if (!temp) { rec_set_heap_no_new(rec, PAGE_HEAP_NO_USER_LOW); - rec_set_status(rec, n_fields == index->n_core_fields - ? REC_STATUS_ORDINARY - : REC_STATUS_COLUMNS_ADDED); - } if (dict_table_is_comp(index->table)) { + + rec_set_status( + rec, n_fields == index->n_core_fields + ? REC_STATUS_ORDINARY + : REC_STATUS_INSTANT); + } + + if (dict_table_is_comp(index->table)) { /* No need to do adjust fixed_len=0. We only need to adjust it for ROW_FORMAT=REDUNDANT. */ temp = false; } n_node_ptr_field = ULINT_UNDEFINED; + lens = nulls - (index->is_instant() ? UT_BITS_IN_BYTES(index->get_n_nullable( - n_fields)) + n_fields)) : UT_BITS_IN_BYTES( unsigned(index->n_nullable))); break; @@ -1490,8 +1588,8 @@ rec_convert_dtuple_to_rec_comp( rec_set_status(rec, status); ut_ad(n_fields == dict_index_get_n_unique_in_tree_nonleaf(index) + 1); - ut_d(n_null = std::min(index->n_core_null_bytes * 8U, - index->n_nullable)); + ut_d(n_null = std::min(index->n_core_null_bytes * 8U, + index->n_nullable)); n_node_ptr_field = n_fields - 1; lens = nulls - index->n_core_null_bytes; break; @@ -1501,30 +1599,33 @@ rec_convert_dtuple_to_rec_comp( return; } +start: end = rec; /* clear the SQL-null flags */ memset(lens + 1, 0, ulint(nulls - lens)); + const dfield_t* const fend = field + n_fields; /* Store the data and the offsets */ - - for (i = 0; i < n_fields; i++) { - const dict_field_t* ifield; - dict_col_t* col = NULL; - - field = &fields[i]; - - type = dfield_get_type(field); - len = dfield_get_len(field); - - if (UNIV_UNLIKELY(i == n_node_ptr_field)) { - ut_ad(dtype_get_prtype(type) & DATA_NOT_NULL); + for (ulint i = 0; field < fend; i++, field++) { + ulint len = dfield_get_len(field); + + if (mblob) { + if (i == index->first_user_field()) { + ut_ad(len == FIELD_REF_SIZE); + ut_ad(dfield_is_ext(field)); + memcpy(end, dfield_get_data(field), len); + end += len; + len = dfield_get_len(++field); + } + } else if (UNIV_UNLIKELY(i == n_node_ptr_field)) { + ut_ad(field->type.prtype & DATA_NOT_NULL); ut_ad(len == REC_NODE_PTR_SIZE); memcpy(end, dfield_get_data(field), len); end += REC_NODE_PTR_SIZE; break; } - if (!(dtype_get_prtype(type) & DATA_NOT_NULL)) { + if (!(field->type.prtype & DATA_NOT_NULL)) { /* nullable field */ ut_ad(n_null--); @@ -1547,11 +1648,12 @@ rec_convert_dtuple_to_rec_comp( /* only nullable fields can be null */ ut_ad(!dfield_is_null(field)); - ifield = dict_index_get_nth_field(index, i); - fixed_len = ifield->fixed_len; - col = ifield->col; + const dict_field_t* ifield + = dict_index_get_nth_field(index, i); + ulint fixed_len = ifield->fixed_len; + if (temp && fixed_len - && !dict_col_get_fixed_size(col, temp)) { + && !dict_col_get_fixed_size(ifield->col, temp)) { fixed_len = 0; } @@ -1563,23 +1665,23 @@ rec_convert_dtuple_to_rec_comp( it is 128 or more, or when the field is stored externally. */ if (fixed_len) { ut_ad(len <= fixed_len); - ut_ad(!col->mbmaxlen - || len >= col->mbminlen - * fixed_len / col->mbmaxlen); + ut_ad(!ifield->col->mbmaxlen + || len >= ifield->col->mbminlen + * fixed_len / ifield->col->mbmaxlen); ut_ad(!dfield_is_ext(field)); } else if (dfield_is_ext(field)) { - ut_ad(DATA_BIG_COL(col)); + ut_ad(DATA_BIG_COL(ifield->col)); ut_ad(len <= REC_ANTELOPE_MAX_INDEX_COL_LEN - + BTR_EXTERN_FIELD_REF_SIZE); + + BTR_EXTERN_FIELD_REF_SIZE); *lens-- = (byte) (len >> 8) | 0xc0; *lens-- = (byte) len; } else { - ut_ad(len <= dtype_get_len(type) - || DATA_LARGE_MTYPE(dtype_get_mtype(type)) + ut_ad(len <= field->type.len + || DATA_LARGE_MTYPE(field->type.mtype) || !strcmp(index->name, FTS_INDEX_TABLE_IND_NAME)); if (len < 128 || !DATA_BIG_LEN_MTYPE( - dtype_get_len(type), dtype_get_mtype(type))) { + field->type.len, field->type.mtype)) { *lens-- = (byte) len; } else { @@ -1612,24 +1714,37 @@ rec_convert_dtuple_to_rec_new( ut_ad(!(dtuple->info_bits & ~(REC_NEW_STATUS_MASK | REC_INFO_DELETED_FLAG | REC_INFO_MIN_REC_FLAG))); - rec_comp_status_t status = static_cast( - dtuple->info_bits & REC_NEW_STATUS_MASK); - if (status == REC_STATUS_ORDINARY - && dtuple->n_fields > index->n_core_fields) { - ut_ad(index->is_instant()); - status = REC_STATUS_COLUMNS_ADDED; - } ulint extra_size; - rec_get_converted_size_comp( - index, status, dtuple->fields, dtuple->n_fields, &extra_size); - rec_t* rec = buf + extra_size; + if (UNIV_UNLIKELY(dtuple->is_alter_metadata())) { + ut_ad((dtuple->info_bits & REC_NEW_STATUS_MASK) + == REC_STATUS_INSTANT); + rec_get_converted_size_comp_prefix_low( + index, dtuple->fields, dtuple->n_fields, + &extra_size, REC_STATUS_INSTANT, false); + buf += extra_size; + rec_convert_dtuple_to_rec_comp( + buf, index, dtuple->fields, dtuple->n_fields, + REC_STATUS_INSTANT, false); + } else { + rec_get_converted_size_comp(index, dtuple, &extra_size); + buf += extra_size; + rec_comp_status_t status = rec_comp_status_t( + dtuple->info_bits & REC_NEW_STATUS_MASK); + if (status == REC_STATUS_ORDINARY + && dtuple->n_fields > index->n_core_fields) { + ut_ad(index->is_instant()); + status = REC_STATUS_INSTANT; + } - rec_convert_dtuple_to_rec_comp( - rec, index, dtuple->fields, dtuple->n_fields, status, false); - rec_set_info_bits_new(rec, dtuple->info_bits & ~REC_NEW_STATUS_MASK); - return(rec); + rec_convert_dtuple_to_rec_comp( + buf, index, dtuple->fields, dtuple->n_fields, + status, false); + } + + rec_set_info_bits_new(buf, dtuple->info_bits & ~REC_NEW_STATUS_MASK); + return buf; } /*********************************************************//** @@ -1668,7 +1783,7 @@ rec_convert_dtuple_to_rec( @param[in] fields data fields @param[in] n_fields number of data fields @param[out] extra record header size -@param[in] status REC_STATUS_ORDINARY or REC_STATUS_COLUMNS_ADDED +@param[in] status REC_STATUS_ORDINARY or REC_STATUS_INSTANT @return total size, in bytes */ ulint rec_get_converted_size_temp( @@ -1688,7 +1803,7 @@ rec_get_converted_size_temp( @param[in,out] offsets offsets to the fields; in: rec_offs_n_fields(offsets) @param[in] n_core number of core fields (index->n_core_fields) @param[in] def_val default values for non-core fields -@param[in] status REC_STATUS_ORDINARY or REC_STATUS_COLUMNS_ADDED */ +@param[in] status REC_STATUS_ORDINARY or REC_STATUS_INSTANT */ void rec_init_offsets_temp( const rec_t* rec, @@ -1699,14 +1814,14 @@ rec_init_offsets_temp( rec_comp_status_t status) { ut_ad(status == REC_STATUS_ORDINARY - || status == REC_STATUS_COLUMNS_ADDED); + || status == REC_STATUS_INSTANT); /* The table may have been converted to plain format if it was emptied during an ALTER TABLE operation. */ ut_ad(index->n_core_fields == n_core || !index->is_instant()); ut_ad(index->n_core_fields >= n_core); rec_init_offsets_comp_ordinary(rec, index, offsets, n_core, def_val, - status == REC_STATUS_COLUMNS_ADDED - ? REC_LEAF_TEMP_COLUMNS_ADDED + status == REC_STATUS_INSTANT + ? REC_LEAF_TEMP_INSTANT : REC_LEAF_TEMP); } @@ -1732,7 +1847,7 @@ rec_init_offsets_temp( @param[in] index clustered or secondary index @param[in] fields data fields @param[in] n_fields number of data fields -@param[in] status REC_STATUS_ORDINARY or REC_STATUS_COLUMNS_ADDED +@param[in] status REC_STATUS_ORDINARY or REC_STATUS_INSTANT */ void rec_convert_dtuple_to_temp( @@ -1902,14 +2017,15 @@ rec_copy_prefix_to_buf( ut_ad(n_fields <= dict_index_get_n_unique_in_tree_nonleaf(index)); break; - case REC_STATUS_COLUMNS_ADDED: + case REC_STATUS_INSTANT: /* We would have !index->is_instant() when rolling back an instant ADD COLUMN operation. */ ut_ad(index->is_instant() || page_rec_is_metadata(rec)); ut_ad(n_fields <= index->first_user_field()); nulls++; const ulint n_rec = ulint(index->n_core_fields) + 1 - + rec_get_n_add_field(nulls); + + rec_get_n_add_field(nulls) + - rec_is_alter_metadata(rec, true); instant_omit = ulint(&rec[-REC_N_NEW_EXTRA_BYTES] - nulls); ut_ad(instant_omit == 1 || instant_omit == 2); nullf = nulls; @@ -1998,7 +2114,7 @@ rec_copy_prefix_to_buf( /* copy the fixed-size header and the record prefix */ memcpy(b - REC_N_NEW_EXTRA_BYTES, rec - REC_N_NEW_EXTRA_BYTES, prefix_len + REC_N_NEW_EXTRA_BYTES); - ut_ad(rec_get_status(b) == REC_STATUS_COLUMNS_ADDED); + ut_ad(rec_get_status(b) == REC_STATUS_INSTANT); rec_set_status(b, REC_STATUS_ORDINARY); return b; } else { diff --git a/storage/innobase/row/row0import.cc b/storage/innobase/row/row0import.cc index d402f6ee67e..6597d583c0f 100644 --- a/storage/innobase/row/row0import.cc +++ b/storage/innobase/row/row0import.cc @@ -1462,7 +1462,7 @@ IndexPurge::open() UNIV_NOTHROW btr_pcur_open_at_index_side( true, m_index, BTR_MODIFY_LEAF, &m_pcur, true, 0, &m_mtr); btr_pcur_move_to_next_user_rec(&m_pcur, &m_mtr); - if (rec_is_metadata(btr_pcur_get_rec(&m_pcur), m_index)) { + if (rec_is_metadata(btr_pcur_get_rec(&m_pcur), *m_index)) { ut_ad(btr_pcur_is_on_user_rec(&m_pcur)); /* Skip the metadata pseudo-record. */ } else { @@ -2267,7 +2267,7 @@ row_import_set_sys_max_row_id( if (page_rec_is_infimum(rec)) { /* The table is empty. */ err = DB_SUCCESS; - } else if (rec_is_metadata(rec, index)) { + } else if (rec_is_metadata(rec, *index)) { /* The clustered index contains the metadata record only, that is, the table is empty. */ err = DB_SUCCESS; diff --git a/storage/innobase/row/row0ins.cc b/storage/innobase/row/row0ins.cc index 4620e493982..28a03b245ad 100644 --- a/storage/innobase/row/row0ins.cc +++ b/storage/innobase/row/row0ins.cc @@ -2642,7 +2642,7 @@ row_ins_clust_index_entry_low( #endif /* UNIV_DEBUG */ if (UNIV_UNLIKELY(entry->info_bits != 0)) { - ut_ad(entry->info_bits == REC_INFO_METADATA); + ut_ad(entry->is_metadata()); ut_ad(flags == BTR_NO_LOCKING_FLAG); ut_ad(index->is_instant()); ut_ad(!dict_index_is_online_ddl(index)); @@ -2650,28 +2650,18 @@ row_ins_clust_index_entry_low( const rec_t* rec = btr_cur_get_rec(cursor); - switch (rec_get_info_bits(rec, page_rec_is_comp(rec)) - & (REC_INFO_MIN_REC_FLAG | REC_INFO_DELETED_FLAG)) { - case REC_INFO_MIN_REC_FLAG: + if (rec_get_info_bits(rec, page_rec_is_comp(rec)) + & REC_INFO_MIN_REC_FLAG) { thr_get_trx(thr)->error_info = index; err = DB_DUPLICATE_KEY; goto err_exit; - case REC_INFO_MIN_REC_FLAG | REC_INFO_DELETED_FLAG: - /* The metadata record never carries the delete-mark - in MariaDB Server 10.3. - If a table loses its 'instantness', it happens - by the rollback of this first-time insert, or - by a call to btr_page_empty() on the root page - when the table becomes empty. */ - err = DB_CORRUPTION; - goto err_exit; - default: - ut_ad(!row_ins_must_modify_rec(cursor)); - goto do_insert; } + + ut_ad(!row_ins_must_modify_rec(cursor)); + goto do_insert; } - if (rec_is_metadata(btr_cur_get_rec(cursor), index)) { + if (rec_is_metadata(btr_cur_get_rec(cursor), *index)) { goto do_insert; } @@ -3455,6 +3445,23 @@ row_ins_index_entry_set_vals( ut_ad(dtuple_get_n_fields(row) == dict_table_get_n_cols(index->table)); row_field = dtuple_get_nth_v_field(row, v_col->v_pos); + } else if (col->is_dropped()) { + ut_ad(index->is_primary()); + + if (!(col->prtype & DATA_NOT_NULL)) { + field->data = NULL; + field->len = UNIV_SQL_NULL; + field->type.prtype = DATA_BINARY_TYPE; + } else { + ut_ad(col->len <= sizeof field_ref_zero); + dfield_set_data(field, field_ref_zero, + col->len); + field->type.prtype = DATA_NOT_NULL; + } + + field->type.mtype = col->len + ? DATA_FIXBINARY : DATA_BINARY; + continue; } else { row_field = dtuple_get_nth_field( row, ind_field->col->ind); @@ -3464,7 +3471,7 @@ row_ins_index_entry_set_vals( /* Check column prefix indexes */ if (ind_field != NULL && ind_field->prefix_len > 0 - && dfield_get_len(row_field) != UNIV_SQL_NULL) { + && len != UNIV_SQL_NULL) { const dict_col_t* col = dict_field_get_col(ind_field); @@ -3518,7 +3525,8 @@ row_ins_index_entry_step( ut_ad(dtuple_check_typed(node->row)); - err = row_ins_index_entry_set_vals(node->index, node->entry, node->row); + err = row_ins_index_entry_set_vals(node->index, node->entry, + node->row); if (err != DB_SUCCESS) { DBUG_RETURN(err); diff --git a/storage/innobase/row/row0log.cc b/storage/innobase/row/row0log.cc index 3a8a2a1ad89..07772fc5468 100644 --- a/storage/innobase/row/row0log.cc +++ b/storage/innobase/row/row0log.cc @@ -851,7 +851,7 @@ row_log_table_low_redundant( const bool is_instant = index->online_log->is_instant(index); rec_comp_status_t status = is_instant - ? REC_STATUS_COLUMNS_ADDED : REC_STATUS_ORDINARY; + ? REC_STATUS_INSTANT : REC_STATUS_ORDINARY; size = rec_get_converted_size_temp( index, tuple->fields, tuple->n_fields, &extra_size, status); @@ -905,7 +905,7 @@ row_log_table_low_redundant( *b++ = static_cast(extra_size); } - if (status == REC_STATUS_COLUMNS_ADDED) { + if (status == REC_STATUS_INSTANT) { ut_ad(is_instant); if (n_fields <= index->online_log->n_core_fields) { status = REC_STATUS_ORDINARY; @@ -970,7 +970,7 @@ row_log_table_low( ut_ad(!"wrong page type"); } #endif /* UNIV_DEBUG */ - ut_ad(!rec_is_metadata(rec, index)); + ut_ad(!rec_is_metadata(rec, *index)); ut_ad(page_rec_is_leaf(rec)); ut_ad(!page_is_comp(page_align(rec)) == !rec_offs_comp(offsets)); /* old_pk=row_log_table_get_pk() [not needed in INSERT] is a prefix @@ -993,7 +993,7 @@ row_log_table_low( ut_ad(page_is_comp(page_align(rec))); ut_ad(rec_get_status(rec) == REC_STATUS_ORDINARY - || rec_get_status(rec) == REC_STATUS_COLUMNS_ADDED); + || rec_get_status(rec) == REC_STATUS_INSTANT); const ulint omit_size = REC_N_NEW_EXTRA_BYTES; @@ -1067,7 +1067,7 @@ row_log_table_low( if (is_instant) { *b++ = fake_extra_size - ? REC_STATUS_COLUMNS_ADDED + ? REC_STATUS_INSTANT : rec_get_status(rec); } else { ut_ad(rec_get_status(rec) == REC_STATUS_ORDINARY); @@ -1559,11 +1559,17 @@ row_log_table_apply_convert_mrec( const dict_col_t* col = dict_field_get_col(ind_field); + if (col->is_dropped()) { + /* the column was instantly dropped earlier */ + ut_ad(index->table->instant); + continue; + } + ulint col_no = log->col_map[dict_col_get_no(col)]; if (col_no == ULINT_UNDEFINED) { - /* dropped column */ + /* the column is being dropped now */ continue; } @@ -3201,7 +3207,8 @@ row_log_allocate( log->head.total = 0; log->path = path; log->n_core_fields = index->n_core_fields; - ut_ad(!table || log->is_instant(index) == index->is_instant()); + ut_ad(!table || log->is_instant(index) + == (index->n_core_fields < index->n_fields)); log->allow_not_null = allow_not_null; log->old_table = old_table; log->n_rows = 0; diff --git a/storage/innobase/row/row0merge.cc b/storage/innobase/row/row0merge.cc index ad44742e57b..89142946615 100644 --- a/storage/innobase/row/row0merge.cc +++ b/storage/innobase/row/row0merge.cc @@ -1873,7 +1873,7 @@ row_merge_read_clustered_index( btr_pcur_open_at_index_side( true, clust_index, BTR_SEARCH_LEAF, &pcur, true, 0, &mtr); btr_pcur_move_to_next_user_rec(&pcur, &mtr); - if (rec_is_metadata(btr_pcur_get_rec(&pcur), clust_index)) { + if (rec_is_metadata(btr_pcur_get_rec(&pcur), *clust_index)) { ut_ad(btr_pcur_is_on_user_rec(&pcur)); /* Skip the metadata pseudo-record. */ } else { diff --git a/storage/innobase/row/row0mysql.cc b/storage/innobase/row/row0mysql.cc index e65ca48811d..efbb7ac0b48 100644 --- a/storage/innobase/row/row0mysql.cc +++ b/storage/innobase/row/row0mysql.cc @@ -330,6 +330,7 @@ row_mysql_read_geometry( ulint col_len) /*!< in: MySQL format length */ { byte* data; + ut_ad(col_len > 8); *len = mach_read_from_n_little_endian(ref, col_len - 8); @@ -829,7 +830,8 @@ row_create_prebuilt( clust_index = dict_table_get_first_index(table); /* Make sure that search_tuple is long enough for clustered index */ - ut_a(2 * dict_table_get_n_cols(table) >= clust_index->n_fields); + ut_a(2 * unsigned(table->n_cols) >= unsigned(clust_index->n_fields) + - clust_index->table->n_dropped()); ref_len = dict_index_get_n_unique(clust_index); diff --git a/storage/innobase/row/row0purge.cc b/storage/innobase/row/row0purge.cc index 527cf0336d5..40488f458b2 100644 --- a/storage/innobase/row/row0purge.cc +++ b/storage/innobase/row/row0purge.cc @@ -845,8 +845,9 @@ static void row_purge_reset_trx_id(purge_node_t* node, mtr_t* mtr) became purgeable) */ if (node->roll_ptr == row_get_rec_roll_ptr(rec, index, offsets)) { - ut_ad(!rec_get_deleted_flag(rec, - rec_offs_comp(offsets))); + ut_ad(!rec_get_deleted_flag( + rec, rec_offs_comp(offsets)) + || rec_is_alter_metadata(rec, *index)); DBUG_LOG("purge", "reset DB_TRX_ID=" << ib::hex(row_get_rec_trx_id( rec, index, offsets))); @@ -1147,10 +1148,13 @@ err_exit: /* Read to the partial row the fields that occur in indexes */ if (!(node->cmpl_info & UPD_NODE_NO_ORD_CHANGE)) { + ut_ad(!(node->update->info_bits & REC_INFO_MIN_REC_FLAG)); ptr = trx_undo_rec_get_partial_row( ptr, clust_index, node->update, &node->row, type == TRX_UNDO_UPD_DEL_REC, node->heap); + } else if (node->update->info_bits & REC_INFO_MIN_REC_FLAG) { + node->ref = &trx_undo_metadata; } return(true); diff --git a/storage/innobase/row/row0quiesce.cc b/storage/innobase/row/row0quiesce.cc index 4bfa7e0760f..074e023795c 100644 --- a/storage/innobase/row/row0quiesce.cc +++ b/storage/innobase/row/row0quiesce.cc @@ -73,17 +73,16 @@ row_quiesce_write_index_fields( return(DB_IO_ERROR); } + const char* field_name = field->name ? field->name : ""; /* Include the NUL byte in the length. */ - ib_uint32_t len = static_cast(strlen(field->name) + 1); - ut_a(len > 1); - + ib_uint32_t len = static_cast(strlen(field_name) + 1); mach_write_to_4(row, len); DBUG_EXECUTE_IF("ib_export_io_write_failure_10", close(fileno(file));); if (fwrite(row, 1, sizeof(len), file) != sizeof(len) - || fwrite(field->name, 1, len, file) != len) { + || fwrite(field_name, 1, len, file) != len) { ib_senderrf( thd, IB_LOG_LEVEL_WARN, ER_IO_WRITE_ERROR, diff --git a/storage/innobase/row/row0row.cc b/storage/innobase/row/row0row.cc index 8cd215d8902..ad3bb6ab9be 100644 --- a/storage/innobase/row/row0row.cc +++ b/storage/innobase/row/row0row.cc @@ -199,7 +199,7 @@ row_build_index_entry_low( { dtuple_t* entry; ulint entry_len; - ulint i; + ulint i = 0; ulint num_v = 0; entry_len = dict_index_get_n_fields(index); @@ -219,90 +219,87 @@ row_build_index_entry_low( } else { dtuple_set_n_fields_cmp( entry, dict_index_get_n_unique_in_tree(index)); - } + if (dict_index_is_spatial(index)) { + /* Set the MBR field */ + if (!row_build_spatial_index_key( + index, ext, + dtuple_get_nth_field(entry, 0), + dtuple_get_nth_field( + row, + dict_index_get_nth_field(index, i) + ->col->ind), flag, heap)) { + return NULL; + } - for (i = 0; i < entry_len + num_v; i++) { - const dict_field_t* ind_field = NULL; - const dict_col_t* col; - ulint col_no = 0; - dfield_t* dfield; - dfield_t* dfield2; - ulint len; - - if (i >= entry_len) { - /* This is to insert new rows to cluster index */ - ut_ad(dict_index_is_clust(index) - && flag == ROW_BUILD_FOR_INSERT); - dfield = dtuple_get_nth_v_field(entry, i - entry_len); - col = &dict_table_get_nth_v_col( - index->table, i - entry_len)->m_col; + i = 1; + } + } - } else { - ind_field = dict_index_get_nth_field(index, i); - col = ind_field->col; - col_no = dict_col_get_no(col); - dfield = dtuple_get_nth_field(entry, i); + for (; i < entry_len; i++) { + const dict_field_t& f = index->fields[i]; + dfield_t* dfield = dtuple_get_nth_field(entry, i); + + if (f.col->is_dropped()) { + ut_ad(index->is_primary()); + ut_ad(index->is_instant()); + ut_ad(!f.col->is_virtual()); + dict_col_copy_type(f.col, &dfield->type); + if (f.col->is_nullable()) { + dfield_set_null(dfield); + } else { + dfield_set_data(dfield, field_ref_zero, + f.fixed_len); + } + continue; } - compile_time_assert(DATA_MISSING == 0); + const dfield_t* dfield2; - if (col->is_virtual()) { - const dict_v_col_t* v_col - = reinterpret_cast(col); + if (f.col->is_virtual()) { + const dict_v_col_t* v_col + = reinterpret_cast(f.col); ut_ad(v_col->v_pos < dtuple_get_n_v_fields(row)); dfield2 = dtuple_get_nth_v_field(row, v_col->v_pos); ut_ad(dfield_is_null(dfield2) || dfield_get_len(dfield2) == 0 || dfield2->data); + ut_ad(!dfield_is_ext(dfield2)); + if (UNIV_UNLIKELY(dfield2->type.mtype + == DATA_MISSING)) { + ut_ad(flag == ROW_BUILD_FOR_PURGE); + return(NULL); + } } else { - dfield2 = dtuple_get_nth_field(row, col_no); - ut_ad(dfield_get_type(dfield2)->mtype == DATA_MISSING - || (!(dfield_get_type(dfield2)->prtype - & DATA_VIRTUAL))); - } - - if (UNIV_UNLIKELY(dfield_get_type(dfield2)->mtype - == DATA_MISSING)) { - /* The field has not been initialized in the row. - This should be from trx_undo_rec_get_partial_row(). */ - return(NULL); - } - -#ifdef UNIV_DEBUG - if (dfield_get_type(dfield2)->prtype & DATA_VIRTUAL - && dict_index_is_clust(index)) { - ut_ad(flag == ROW_BUILD_FOR_INSERT); - } -#endif /* UNIV_DEBUG */ - - /* Special handle spatial index, set the first field - which is for store MBR. */ - if (dict_index_is_spatial(index) && i == 0) { - if (!row_build_spatial_index_key( - index, ext, dfield, dfield2, flag, heap)) { - return NULL; + dfield2 = dtuple_get_nth_field(row, f.col->ind); + if (UNIV_UNLIKELY(dfield2->type.mtype + == DATA_MISSING)) { + /* The field has not been initialized in + the row. This should be from + trx_undo_rec_get_partial_row(). */ + return(NULL); } - continue; + ut_ad(!(dfield2->type.prtype & DATA_VIRTUAL)); } - len = dfield_get_len(dfield2); + compile_time_assert(DATA_MISSING == 0); - dfield_copy(dfield, dfield2); + *dfield = *dfield2; if (dfield_is_null(dfield)) { continue; } - if ((!ind_field || ind_field->prefix_len == 0) + ulint len = dfield_get_len(dfield); + + if (f.prefix_len == 0 && (!dfield_is_ext(dfield) || dict_index_is_clust(index))) { /* The dfield_copy() above suffices for columns that are stored in-page, or for clustered index record columns that are not - part of a column prefix in the PRIMARY KEY, - or for virtaul columns in cluster index record. */ + part of a column prefix in the PRIMARY KEY. */ continue; } @@ -313,11 +310,11 @@ row_build_index_entry_low( index record with an off-page column is when it is a column prefix index. If atomic_blobs, also fully indexed long columns may be stored off-page. */ - ut_ad(col->ord_part); + ut_ad(f.col->ord_part); if (ext) { /* See if the column is stored externally. */ - const byte* buf = row_ext_lookup(ext, col_no, + const byte* buf = row_ext_lookup(ext, f.col->ind, &len); if (UNIV_LIKELY_NULL(buf)) { if (UNIV_UNLIKELY(buf == field_ref_zero)) { @@ -326,7 +323,7 @@ row_build_index_entry_low( dfield_set_data(dfield, buf, len); } - if (ind_field->prefix_len == 0) { + if (f.prefix_len == 0) { /* If ROW_FORMAT=DYNAMIC or ROW_FORMAT=COMPRESSED, we can have a secondary index on an entire column @@ -353,16 +350,33 @@ row_build_index_entry_low( } /* If a column prefix index, take only the prefix. */ - if (ind_field->prefix_len) { + if (f.prefix_len) { len = dtype_get_at_most_n_mbchars( - col->prtype, col->mbminlen, col->mbmaxlen, - ind_field->prefix_len, len, + f.col->prtype, + f.col->mbminlen, f.col->mbmaxlen, + f.prefix_len, len, static_cast(dfield_get_data(dfield))); dfield_set_len(dfield, len); } } - return(entry); + for (i = num_v; i--; ) { + ut_ad(index->is_primary()); + ut_ad(flag == ROW_BUILD_FOR_INSERT); + dfield_t* dfield = dtuple_get_nth_v_field(entry, i); + const dict_v_col_t* v_col = dict_table_get_nth_v_col( + index->table, i); + ut_ad(!v_col->m_col.is_dropped()); + ut_ad(v_col->v_pos < dtuple_get_n_v_fields(row)); + const dfield_t* dfield2 = dtuple_get_nth_v_field( + row, v_col->v_pos); + ut_ad(dfield_is_null(dfield2) || + dfield_get_len(dfield2) == 0 || dfield2->data); + ut_ad(dfield2->type.mtype != DATA_MISSING); + *dfield = *dfield2; + } + + return entry; } /** An inverse function to row_build_index_entry. Builds a row from a @@ -499,11 +513,23 @@ row_build_low( j = 0; + const dict_field_t* ind_field = index->fields; + for (ulint i = 0; i < rec_offs_n_fields(offsets); i++) { - const dict_field_t* ind_field - = dict_index_get_nth_field(index, i); + if (i == index->first_user_field() + && rec_is_alter_metadata(rec, *index)) { + ut_ad(rec_offs_nth_extern(offsets, i)); + ut_d(ulint len); + ut_d(rec_get_nth_field_offs(offsets, i, &len)); + ut_ad(len == FIELD_REF_SIZE); + continue; + } + + ut_ad(ind_field < &index->fields[index->n_fields]); - if (ind_field->prefix_len) { + const dict_col_t* col = dict_field_get_col(ind_field); + + if ((ind_field++)->prefix_len) { /* Column prefixes can only occur in key fields, which cannot be stored externally. For a column prefix, there should also be the full @@ -513,10 +539,11 @@ row_build_low( continue; } - const dict_col_t* col - = dict_field_get_col(ind_field); - ulint col_no - = dict_col_get_no(col); + if (col->is_dropped()) { + continue; + } + + ulint col_no = dict_col_get_no(col); if (col_map) { col_no = col_map[col_no]; @@ -528,6 +555,7 @@ row_build_low( } dfield_t* dfield = dtuple_get_nth_field(row, col_no); + const void* field = rec_get_nth_field( copy, offsets, i, &len); if (len == UNIV_SQL_DEFAULT) { @@ -671,15 +699,19 @@ row_build_w_add_vcol( } /** Convert an index record to a data tuple. -@tparam def whether the index->instant_field_value() needs to be accessed -@param[in] rec index record -@param[in] index index -@param[in] offsets rec_get_offsets(rec, index) -@param[out] n_ext number of externally stored columns -@param[in,out] heap memory heap for allocations +@tparam metadata whether the index->instant_field_value() needs to be accessed +@tparam mblob 1 if rec_is_alter_metadata(); +2 if we want converted metadata corresponding to info_bits +@param[in] rec index record +@param[in] index index +@param[in] offsets rec_get_offsets(rec, index) +@param[out] n_ext number of externally stored columns +@param[in,out] heap memory heap for allocations +@param[in] info_bits (only used if mblob=2) +@param[in] pad (only used if mblob=2) @return index entry built; does not set info_bits, and the data fields in the entry will point directly to rec */ -template +template static inline dtuple_t* row_rec_to_index_entry_impl( @@ -687,44 +719,64 @@ row_rec_to_index_entry_impl( const dict_index_t* index, const ulint* offsets, ulint* n_ext, - mem_heap_t* heap) + mem_heap_t* heap, + ulint info_bits = 0, + bool pad = false) { - dtuple_t* entry; - dfield_t* dfield; - ulint i; - const byte* field; - ulint len; - ulint rec_len; - ut_ad(rec != NULL); ut_ad(heap != NULL); ut_ad(index != NULL); - ut_ad(def || !rec_offs_any_default(offsets)); - + ut_ad(!mblob || index->is_primary()); + ut_ad(!mblob || !dict_index_is_spatial(index)); + compile_time_assert(!mblob || metadata); + compile_time_assert(mblob <= 2); /* Because this function may be invoked by row0merge.cc on a record whose header is in different format, the check rec_offs_validate(rec, index, offsets) must be avoided here. */ ut_ad(n_ext); *n_ext = 0; - rec_len = rec_offs_n_fields(offsets); - - entry = dtuple_create(heap, rec_len); + const bool got = mblob == 2 && rec_is_alter_metadata(rec, *index); + ulint rec_len = rec_offs_n_fields(offsets); + if (mblob == 2) { + ut_ad(info_bits == REC_INFO_METADATA_ALTER + || info_bits == REC_INFO_METADATA_ADD); + ut_ad(rec_len <= ulint(index->n_fields + got)); + if (pad) { + rec_len = ulint(index->n_fields) + + (info_bits == REC_INFO_METADATA_ALTER); + } else if (!got && info_bits == REC_INFO_METADATA_ALTER) { + rec_len++; + } + } else { + ut_ad(info_bits == 0); + ut_ad(!pad); + } + dtuple_t* entry = dtuple_create(heap, rec_len); + dfield_t* dfield = entry->fields; dtuple_set_n_fields_cmp(entry, dict_index_get_n_unique_in_tree(index)); - ut_ad(rec_len == dict_index_get_n_fields(index) + ut_ad(mblob == 2 + || rec_len == dict_index_get_n_fields(index) + uint(mblob == 1) /* a record for older SYS_INDEXES table (missing merge_threshold column) is acceptable. */ || (index->table->id == DICT_INDEXES_ID && rec_len == dict_index_get_n_fields(index) - 1)); - dict_index_copy_types(entry, index, rec_len); - - for (i = 0; i < rec_len; i++) { + ulint i; + for (i = 0; i < (mblob ? index->first_user_field() : rec_len); + i++, dfield++) { + dict_col_copy_type(dict_index_get_nth_col(index, i), + &dfield->type); + if (!mblob + && dict_index_is_spatial(index) + && DATA_GEOMETRY_MTYPE(dfield->type.mtype)) { + dfield->type.prtype |= DATA_GIS_MBR; + } - dfield = dtuple_get_nth_field(entry, i); - field = def + ulint len; + const byte* field = metadata ? rec_get_nth_cfield(rec, index, offsets, i, &len) : rec_get_nth_field(rec, offsets, i, &len); @@ -732,12 +784,80 @@ row_rec_to_index_entry_impl( if (rec_offs_nth_extern(offsets, i)) { dfield_set_ext(dfield); - (*n_ext)++; + ++*n_ext; + } + } + + if (mblob) { + ulint len; + const byte* field; + ulint j = i; + + if (mblob == 2) { + const bool want = info_bits == REC_INFO_METADATA_ALTER; + if (got == want) { + if (got) { + goto copy_metadata; + } + } else { + if (want) { + /* Allocate a placeholder for + adding metadata in an update. */ + len = FIELD_REF_SIZE; + field = static_cast( + mem_heap_zalloc(heap, len)); + /* In reality there is one fewer + field present in the record. */ + rec_len--; + goto init_metadata; + } + + /* Skip the undesired metadata blob + (for example, when rolling back an + instant ALTER TABLE). */ + i++; + } + goto copy_user_fields; + } +copy_metadata: + ut_ad(rec_offs_nth_extern(offsets, i)); + field = rec_get_nth_field(rec, offsets, i++, &len); +init_metadata: + dfield->type.metadata_blob_init(); + ut_ad(len == FIELD_REF_SIZE); + dfield_set_data(dfield, field, len); + dfield_set_ext(dfield++); + ++*n_ext; +copy_user_fields: + for (; i < rec_len; i++, dfield++) { + dict_col_copy_type(dict_index_get_nth_col(index, j++), + &dfield->type); + if (mblob == 2 && pad + && i >= rec_offs_n_fields(offsets)) { + field = index->instant_field_value(j - 1, + &len); + dfield_set_data(dfield, field, len); + continue; + } + + field = rec_get_nth_field(rec, offsets, i, &len); + dfield_set_data(dfield, field, len); + + if (rec_offs_nth_extern(offsets, i)) { + dfield_set_ext(dfield); + ++*n_ext; + } } } + if (mblob == 2) { + ulint n_fields = ulint(dfield - entry->fields); + ut_ad(entry->n_fields >= n_fields); + entry->n_fields = n_fields; + } + ut_ad(dfield == entry->fields + entry->n_fields); ut_ad(dtuple_check_typed(entry)); - return(entry); + return entry; } /** Convert an index record to a data tuple. @@ -773,25 +893,26 @@ row_rec_to_index_entry( mem_heap_t* heap) /*!< in: memory heap from which the memory needed is allocated */ { - dtuple_t* entry; - byte* buf; - const rec_t* copy_rec; - ut_ad(rec != NULL); ut_ad(heap != NULL); ut_ad(index != NULL); ut_ad(rec_offs_validate(rec, index, offsets)); /* Take a copy of rec to heap */ - buf = static_cast( - mem_heap_alloc(heap, rec_offs_size(offsets))); - - copy_rec = rec_copy(buf, rec, offsets); + const rec_t* copy_rec = rec_copy( + static_cast(mem_heap_alloc(heap, + rec_offs_size(offsets))), + rec, offsets); rec_offs_make_valid(copy_rec, index, true, const_cast(offsets)); - entry = row_rec_to_index_entry_impl( - copy_rec, index, offsets, n_ext, heap); + + dtuple_t* entry = rec_is_alter_metadata(copy_rec, *index) + ? row_rec_to_index_entry_impl( + copy_rec, index, offsets, n_ext, heap) + : row_rec_to_index_entry_impl( + copy_rec, index, offsets, n_ext, heap); + rec_offs_make_valid(rec, index, true, const_cast(offsets)); @@ -801,6 +922,51 @@ row_rec_to_index_entry( return(entry); } +/** Convert a metadata record to a data tuple. +@param[in] rec metadata record +@param[in] index clustered index after instant ALTER TABLE +@param[in] offsets rec_get_offsets(rec) +@param[out] n_ext number of externally stored fields +@param[in,out] heap memory heap for allocations +@param[in] info_bits the info_bits after an update +@param[in] pad whether to pad to index->n_fields */ +dtuple_t* +row_metadata_to_tuple( + const rec_t* rec, + const dict_index_t* index, + const ulint* offsets, + ulint* n_ext, + mem_heap_t* heap, + ulint info_bits, + bool pad) +{ + ut_ad(info_bits == REC_INFO_METADATA_ALTER + || info_bits == REC_INFO_METADATA_ADD); + ut_ad(rec_is_metadata(rec, *index)); + ut_ad(rec_offs_validate(rec, index, offsets)); + + const rec_t* copy_rec = rec_copy( + static_cast(mem_heap_alloc(heap, + rec_offs_size(offsets))), + rec, offsets); + + rec_offs_make_valid(copy_rec, index, true, + const_cast(offsets)); + + dtuple_t* entry = info_bits == REC_INFO_METADATA_ALTER + || rec_is_alter_metadata(copy_rec, *index) + ? row_rec_to_index_entry_impl( + copy_rec, index, offsets, n_ext, heap, info_bits, pad) + : row_rec_to_index_entry_impl( + copy_rec, index, offsets, n_ext, heap); + + rec_offs_make_valid(rec, index, true, + const_cast(offsets)); + + dtuple_set_info_bits(entry, info_bits); + return entry; +} + /*******************************************************************//** Builds from a secondary index record a row reference with which we can search the clustered index record. @@ -1035,7 +1201,7 @@ row_search_on_row_ref( index = dict_table_get_first_index(table); if (UNIV_UNLIKELY(ref->info_bits != 0)) { - ut_ad(ref->info_bits == REC_INFO_METADATA); + ut_ad(ref->is_metadata()); ut_ad(ref->n_fields <= index->n_uniq); btr_pcur_open_at_index_side(true, index, mode, pcur, true, 0, mtr); diff --git a/storage/innobase/row/row0sel.cc b/storage/innobase/row/row0sel.cc index 1531fb38b7f..a8787574e5a 100644 --- a/storage/innobase/row/row0sel.cc +++ b/storage/innobase/row/row0sel.cc @@ -1489,7 +1489,7 @@ row_sel_try_search_shortcut( const rec_t* rec = btr_pcur_get_rec(&(plan->pcur)); - if (!page_rec_is_user_rec(rec) || rec_is_metadata(rec, index)) { + if (!page_rec_is_user_rec(rec) || rec_is_metadata(rec, *index)) { retry: rw_lock_s_unlock(ahi_latch); return(SEL_RETRY); @@ -1789,7 +1789,7 @@ skip_lock: goto next_rec; } - if (rec_is_metadata(rec, index)) { + if (rec_is_metadata(rec, *index)) { /* Skip the metadata pseudo-record. */ cost_counter++; goto next_rec; @@ -3564,7 +3564,7 @@ sel_restore_position_for_mysql( next: if (btr_pcur_move_to_next(pcur, mtr) && rec_is_metadata(btr_pcur_get_rec(pcur), - pcur->btr_cur.index)) { + *pcur->btr_cur.index)) { btr_pcur_move_to_next(pcur, mtr); } @@ -3580,7 +3580,7 @@ next: prev: if (btr_pcur_is_on_user_rec(pcur) && !moves_up && !rec_is_metadata(btr_pcur_get_rec(pcur), - pcur->btr_cur.index)) { + *pcur->btr_cur.index)) { btr_pcur_move_to_prev(pcur, mtr); } return true; @@ -3857,7 +3857,7 @@ row_sel_try_search_shortcut_for_mysql( BTR_SEARCH_LEAF, pcur, ahi_latch, mtr); rec = btr_pcur_get_rec(pcur); - if (!page_rec_is_user_rec(rec) || rec_is_metadata(rec, index)) { + if (!page_rec_is_user_rec(rec) || rec_is_metadata(rec, *index)) { retry: rw_lock_s_unlock(ahi_latch); return(SEL_RETRY); diff --git a/storage/innobase/row/row0umod.cc b/storage/innobase/row/row0umod.cc index 63cf63b2614..8b68b277719 100644 --- a/storage/innobase/row/row0umod.cc +++ b/storage/innobase/row/row0umod.cc @@ -113,6 +113,9 @@ row_undo_mod_clust_low( ut_ad(rec_get_trx_id(btr_cur_get_rec(btr_cur), btr_cur_get_index(btr_cur)) == thr_get_trx(thr)->id); + ut_ad(node->ref != &trx_undo_metadata + || node->update->info_bits == REC_INFO_METADATA_ADD + || node->update->info_bits == REC_INFO_METADATA_ALTER); if (mode != BTR_MODIFY_LEAF && dict_index_is_online_ddl(btr_cur_get_index(btr_cur))) { @@ -133,6 +136,7 @@ row_undo_mod_clust_low( btr_cur, offsets, offsets_heap, node->update, node->cmpl_info, thr, thr_get_trx(thr)->id, mtr); + ut_ad(err != DB_SUCCESS || node->ref != &trx_undo_metadata); } else { big_rec_t* dummy_big_rec; @@ -145,6 +149,38 @@ row_undo_mod_clust_low( node->cmpl_info, thr, thr_get_trx(thr)->id, mtr); ut_a(!dummy_big_rec); + + static const byte + INFIMUM[8] = {'i','n','f','i','m','u','m',0}, + SUPREMUM[8] = {'s','u','p','r','e','m','u','m'}; + + if (err == DB_SUCCESS + && node->ref == &trx_undo_metadata + && btr_cur_get_index(btr_cur)->table->instant + && node->update->info_bits == REC_INFO_METADATA_ADD) { + if (page_t* root = btr_root_get( + btr_cur_get_index(btr_cur), mtr)) { + byte* infimum; + byte *supremum; + if (page_is_comp(root)) { + infimum = PAGE_NEW_INFIMUM + root; + supremum = PAGE_NEW_SUPREMUM + root; + } else { + infimum = PAGE_OLD_INFIMUM + root; + supremum = PAGE_OLD_SUPREMUM + root; + } + + ut_ad(!memcmp(infimum, INFIMUM, 8) + == !memcmp(supremum, SUPREMUM, 8)); + + if (memcmp(infimum, INFIMUM, 8)) { + mlog_write_string(infimum, INFIMUM, + 8, mtr); + mlog_write_string(supremum, SUPREMUM, + 8, mtr); + } + } + } } if (err == DB_SUCCESS @@ -415,22 +451,36 @@ row_undo_mod_clust( goto mtr_commit_exit; } + ulint trx_id_offset = index->trx_id_offset; ulint trx_id_pos = index->n_uniq ? index->n_uniq : 1; - ut_ad(index->n_uniq <= MAX_REF_PARTS); - /* Reserve enough offsets for the PRIMARY KEY and 2 columns - so that we can access DB_TRX_ID, DB_ROLL_PTR. */ - ulint offsets_[REC_OFFS_HEADER_SIZE + MAX_REF_PARTS + 2]; - rec_offs_init(offsets_); - offsets = rec_get_offsets( - rec, index, offsets_, true, trx_id_pos + 2, &heap); - ulint len; - ulint trx_id_offset = rec_get_nth_field_offs( - offsets, trx_id_pos, &len); - ut_ad(len == DATA_TRX_ID_LEN); + if (trx_id_offset) { + } else if (rec_is_metadata(rec, *index)) { + ut_ad(!buf_block_get_page_zip(btr_pcur_get_block( + &node->pcur))); + for (unsigned i = index->first_user_field(); i--; ) { + trx_id_offset += index->fields[i].fixed_len; + } + } else { + ut_ad(index->n_uniq <= MAX_REF_PARTS); + /* Reserve enough offsets for the PRIMARY KEY and + 2 columns so that we can access + DB_TRX_ID, DB_ROLL_PTR. */ + ulint offsets_[REC_OFFS_HEADER_SIZE + MAX_REF_PARTS + + 2]; + rec_offs_init(offsets_); + offsets = rec_get_offsets( + rec, index, offsets_, true, trx_id_pos + 2, + &heap); + ulint len; + trx_id_offset = rec_get_nth_field_offs( + offsets, trx_id_pos, &len); + ut_ad(len == DATA_TRX_ID_LEN); + } if (trx_read_trx_id(rec + trx_id_offset) == node->new_trx_id) { ut_ad(!rec_get_deleted_flag( - rec, dict_table_is_comp(node->table))); + rec, dict_table_is_comp(node->table)) + || rec_is_alter_metadata(rec, *index)); index->set_modified(mtr); if (page_zip_des_t* page_zip = buf_block_get_page_zip( btr_pcur_get_block(&node->pcur))) { @@ -1224,16 +1274,21 @@ close_table: ut_ad(!node->ref->info_bits); if (node->update->info_bits & REC_INFO_MIN_REC_FLAG) { - /* This must be an undo log record for a subsequent - instant ALTER TABLE, extending the metadata record. */ - ut_ad(clust_index->is_instant()); - if (node->update->info_bits != REC_INFO_MIN_REC_FLAG) { + if ((node->update->info_bits & ~REC_INFO_DELETED_FLAG) + != REC_INFO_MIN_REC_FLAG) { ut_ad(!"wrong info_bits in undo log record"); goto close_table; } - node->update->info_bits = REC_INFO_METADATA; - const_cast(node->ref)->info_bits - = REC_INFO_METADATA; + /* This must be an undo log record for a subsequent + instant ALTER TABLE, extending the metadata record. */ + ut_ad(clust_index->is_instant()); + ut_ad(clust_index->table->instant + || !(node->update->info_bits & REC_INFO_DELETED_FLAG)); + node->ref = &trx_undo_metadata; + node->update->info_bits = (node->update->info_bits + & REC_INFO_DELETED_FLAG) + ? REC_INFO_METADATA_ALTER + : REC_INFO_METADATA_ADD; } if (!row_undo_search_clust_to_pcur(node)) { @@ -1310,7 +1365,7 @@ row_undo_mod( ut_ad(dict_index_is_clust(node->index)); if (node->ref->info_bits) { - ut_ad(node->ref->info_bits == REC_INFO_METADATA); + ut_ad(node->ref->is_metadata()); goto rollback_clust; } diff --git a/storage/innobase/row/row0undo.cc b/storage/innobase/row/row0undo.cc index 2c261c5b9d3..e18f5a24982 100644 --- a/storage/innobase/row/row0undo.cc +++ b/storage/innobase/row/row0undo.cc @@ -229,13 +229,15 @@ row_undo_search_clust_to_pcur( } if (node->rec_type == TRX_UNDO_UPD_EXIST_REC) { - ut_ad(node->row->info_bits == REC_INFO_MIN_REC_FLAG + ut_ad((node->row->info_bits & ~REC_INFO_DELETED_FLAG) + == REC_INFO_MIN_REC_FLAG || node->row->info_bits == 0); node->undo_row = dtuple_copy(node->row, node->heap); row_upd_replace(node->undo_row, &node->undo_ext, clust_index, node->update, node->heap); } else { - ut_ad((node->row->info_bits == REC_INFO_MIN_REC_FLAG) + ut_ad(((node->row->info_bits & ~REC_INFO_DELETED_FLAG) + == REC_INFO_MIN_REC_FLAG) == (node->rec_type == TRX_UNDO_INSERT_METADATA)); node->undo_row = NULL; node->undo_ext = NULL; diff --git a/storage/innobase/row/row0upd.cc b/storage/innobase/row/row0upd.cc index 5279d21b452..f2724435b0d 100644 --- a/storage/innobase/row/row0upd.cc +++ b/storage/innobase/row/row0upd.cc @@ -682,7 +682,7 @@ row_upd_rec_in_place( switch (rec_get_status(rec)) { case REC_STATUS_ORDINARY: break; - case REC_STATUS_COLUMNS_ADDED: + case REC_STATUS_INSTANT: ut_ad(index->is_instant()); break; case REC_STATUS_NODE_PTR: @@ -1256,7 +1256,7 @@ row_upd_index_replace_new_col_val( len = dfield_get_len(dfield); data = static_cast(dfield_get_data(dfield)); - if (field->prefix_len > 0) { + if (field && field->prefix_len > 0) { ibool fetch_ext = dfield_is_ext(dfield) && len < (ulint) field->prefix_len + BTR_EXTERN_FIELD_REF_SIZE; @@ -1322,6 +1322,57 @@ row_upd_index_replace_new_col_val( } } +/** Apply an update vector to an metadata entry. +@param[in,out] entry clustered index metadata record to be updated +@param[in] index index of the entry +@param[in] update update vector built for the entry +@param[in,out] heap memory heap for copying off-page columns */ +static +void +row_upd_index_replace_metadata( + dtuple_t* entry, + const dict_index_t* index, + const upd_t* update, + mem_heap_t* heap) +{ + ut_ad(!index->table->skip_alter_undo); + ut_ad(update->is_alter_metadata()); + ut_ad(entry->info_bits == update->info_bits); + ut_ad(entry->n_fields == ulint(index->n_fields) + 1); + const page_size_t& page_size = dict_table_page_size(index->table); + const ulint first = index->first_user_field(); + ut_d(bool found_mblob = false); + + for (ulint i = upd_get_n_fields(update); i--; ) { + const upd_field_t* uf = upd_get_nth_field(update, i); + ut_ad(!upd_fld_is_virtual_col(uf)); + ut_ad(uf->field_no >= first - 2); + ulint f = uf->field_no; + dfield_t* dfield = dtuple_get_nth_field(entry, f); + + if (f == first) { + ut_d(found_mblob = true); + ut_ad(!dfield_is_null(&uf->new_val)); + ut_ad(dfield_is_ext(dfield)); + ut_ad(dfield_get_len(dfield) == FIELD_REF_SIZE); + ut_ad(!dfield_is_null(dfield)); + dfield_set_data(dfield, uf->new_val.data, + uf->new_val.len); + if (dfield_is_ext(&uf->new_val)) { + dfield_set_ext(dfield); + } + continue; + } + + f -= f > first; + const dict_field_t* field = dict_index_get_nth_field(index, f); + row_upd_index_replace_new_col_val(dfield, field, field->col, + uf, heap, page_size); + } + + ut_ad(found_mblob); +} + /** Apply an update vector to an index entry. @param[in,out] entry index entry to be updated; the clustered index record must be covered by a lock or a page latch to prevent @@ -1337,6 +1388,12 @@ row_upd_index_replace_new_col_vals_index_pos( mem_heap_t* heap) { ut_ad(!index->table->skip_alter_undo); + ut_ad(!entry->is_metadata() || entry->info_bits == update->info_bits); + + if (UNIV_UNLIKELY(entry->is_alter_metadata())) { + row_upd_index_replace_metadata(entry, index, update, heap); + return; + } const page_size_t& page_size = dict_table_page_size(index->table); @@ -2560,10 +2617,10 @@ row_upd_sec_step( } #ifdef UNIV_DEBUG -# define row_upd_clust_rec_by_insert_inherit(rec,offsets,entry,update) \ - row_upd_clust_rec_by_insert_inherit_func(rec,offsets,entry,update) +# define row_upd_clust_rec_by_insert_inherit(rec,index,offsets,entry,update) \ + row_upd_clust_rec_by_insert_inherit_func(rec,index,offsets,entry,update) #else /* UNIV_DEBUG */ -# define row_upd_clust_rec_by_insert_inherit(rec,offsets,entry,update) \ +# define row_upd_clust_rec_by_insert_inherit(rec,index,offsets,entry,update) \ row_upd_clust_rec_by_insert_inherit_func(rec,entry,update) #endif /* UNIV_DEBUG */ /*******************************************************************//** @@ -2578,6 +2635,7 @@ row_upd_clust_rec_by_insert_inherit_func( /*=====================================*/ const rec_t* rec, /*!< in: old record, or NULL */ #ifdef UNIV_DEBUG + dict_index_t* index, /*!< in: index, or NULL */ const ulint* offsets,/*!< in: rec_get_offsets(rec), or NULL */ #endif /* UNIV_DEBUG */ dtuple_t* entry, /*!< in/out: updated entry to be @@ -2588,6 +2646,8 @@ row_upd_clust_rec_by_insert_inherit_func( ulint i; ut_ad(!rec == !offsets); + ut_ad(!rec == !index); + ut_ad(!rec || rec_offs_validate(rec, index, offsets)); ut_ad(!rec || rec_offs_any_extern(offsets)); for (i = 0; i < dtuple_get_n_fields(entry); i++) { @@ -2598,6 +2658,9 @@ row_upd_clust_rec_by_insert_inherit_func( ut_ad(!offsets || !rec_offs_nth_extern(offsets, i) == !dfield_is_ext(dfield) + || (!dict_index_get_nth_field(index, i)->name + && !dfield_is_ext(dfield) + && (dfield_is_null(dfield) || dfield->len == 0)) || upd_get_field_by_field_no(update, i, false)); if (!dfield_is_ext(dfield) || upd_get_field_by_field_no(update, i, false)) { @@ -2705,7 +2768,7 @@ row_upd_clust_rec_by_insert( /* A lock wait occurred in row_ins_clust_index_entry() in the previous invocation of this function. */ row_upd_clust_rec_by_insert_inherit( - NULL, NULL, entry, node->update); + NULL, NULL, NULL, entry, node->update); break; case UPD_NODE_UPDATE_CLUSTERED: /* This is the first invocation of the function where @@ -2746,7 +2809,8 @@ err_exit: if (rec_offs_any_extern(offsets)) { if (row_upd_clust_rec_by_insert_inherit( - rec, offsets, entry, node->update)) { + rec, index, offsets, + entry, node->update)) { /* The blobs are disowned here, expecting the insert down below to inherit them. But if the insert fails, then this disown will be undone diff --git a/storage/innobase/trx/trx0rec.cc b/storage/innobase/trx/trx0rec.cc index 67b5617809f..4520f05e4c3 100644 --- a/storage/innobase/trx/trx0rec.cc +++ b/storage/innobase/trx/trx0rec.cc @@ -40,9 +40,12 @@ Created 3/26/1996 Heikki Tuuri #include "fsp0sysspace.h" #include "row0mysql.h" -/** The search tuple corresponding to TRX_UNDO_INSERT_METADATA */ +/** The search tuple corresponding to TRX_UNDO_INSERT_METADATA. */ const dtuple_t trx_undo_metadata = { - REC_INFO_METADATA, 0, 0, + /* This also works for REC_INFO_METADATA_ALTER, because the + delete-mark (REC_INFO_DELETED_FLAG) is ignored when searching. */ + REC_INFO_METADATA_ADD, + 0, 0, NULL, 0, NULL, UT_LIST_NODE_T(dtuple_t)() #ifdef UNIV_DEBUG @@ -506,7 +509,7 @@ trx_undo_page_report_insert( /* Store then the fields required to uniquely determine the record to be inserted in the clustered index */ if (UNIV_UNLIKELY(clust_entry->info_bits != 0)) { - ut_ad(clust_entry->info_bits == REC_INFO_METADATA); + ut_ad(clust_entry->is_metadata()); ut_ad(index->is_instant()); ut_ad(undo_block->frame[first_free + 2] == TRX_UNDO_INSERT_REC); @@ -920,9 +923,9 @@ trx_undo_page_report_modify( /* Store first some general parameters to the undo log */ if (!update) { - ut_ad(!rec_get_deleted_flag(rec, dict_table_is_comp(table))); + ut_ad(!rec_is_delete_marked(rec, dict_table_is_comp(table))); type_cmpl = TRX_UNDO_DEL_MARK_REC; - } else if (rec_get_deleted_flag(rec, dict_table_is_comp(table))) { + } else if (rec_is_delete_marked(rec, dict_table_is_comp(table))) { /* In delete-marked records, DB_TRX_ID must always refer to an existing update_undo log record. */ ut_ad(row_get_rec_trx_id(rec, index, offsets)); @@ -1036,20 +1039,35 @@ trx_undo_page_report_modify( } } + i = 0; + + if (UNIV_UNLIKELY(update->is_alter_metadata())) { + ut_ad(update->n_fields >= 1); + ut_ad(!upd_fld_is_virtual_col(&update->fields[0])); + ut_ad(update->fields[0].field_no + == index->first_user_field()); + ut_ad(!dfield_is_ext(&update->fields[0].new_val)); + ut_ad(!dfield_is_null(&update->fields[0].new_val)); + /* The instant ADD COLUMN metadata record does not + contain the BLOB. Do not write anything for it. */ + i = !rec_is_alter_metadata(rec, *index); + n_updated -= i; + } + ptr += mach_write_compressed(ptr, n_updated); - for (i = 0; i < upd_get_n_fields(update); i++) { + for (; i < upd_get_n_fields(update); i++) { + if (trx_undo_left(undo_block, ptr) < 5) { + return 0; + } + upd_field_t* fld = upd_get_nth_field(update, i); bool is_virtual = upd_fld_is_virtual_col(fld); ulint max_v_log_len = 0; - ulint pos = fld->field_no; - - /* Write field number to undo log */ - if (trx_undo_left(undo_block, ptr) < 5) { - return(0); - } + ulint pos = fld->field_no; + const dict_col_t* col = NULL; if (is_virtual) { /* Skip the non-indexed column, during @@ -1062,13 +1080,13 @@ trx_undo_page_report_modify( /* add REC_MAX_N_FIELDS to mark this is a virtual col */ - pos += REC_MAX_N_FIELDS; - } + ptr += mach_write_compressed( + ptr, pos + REC_MAX_N_FIELDS); - ptr += mach_write_compressed(ptr, pos); + if (trx_undo_left(undo_block, ptr) < 15) { + return 0; + } - /* Save the old value of field */ - if (is_virtual) { ut_ad(fld->field_no < table->n_v_def); ptr = trx_undo_log_v_idx(undo_block, table, @@ -1093,28 +1111,78 @@ trx_undo_page_report_modify( flen = ut_min( flen, max_v_log_len); } + + goto store_len; + } + + if (UNIV_UNLIKELY(update->is_metadata())) { + ut_ad(pos >= index->first_user_field()); + ut_ad(rec_is_metadata(rec, *index)); + + if (rec_is_alter_metadata(rec, *index)) { + ut_ad(update->is_alter_metadata()); + + field = rec_offs_n_fields(offsets) + > pos + && !rec_offs_nth_default( + offsets, pos) + ? rec_get_nth_field( + rec, offsets, + pos, &flen) + : index->instant_field_value( + pos - 1, &flen); + + if (pos == index->first_user_field()) { + ut_ad(rec_offs_nth_extern( + offsets, pos)); + ut_ad(flen == FIELD_REF_SIZE); + goto write_field; + } + col = dict_index_get_nth_col(index, + pos - 1); + } else if (!update->is_alter_metadata()) { + goto get_field; + } else { + /* We are converting an ADD COLUMN + metadata record to an ALTER TABLE + metadata record, with BLOB. Subtract + the missing metadata BLOB field. */ + ut_ad(pos > index->first_user_field()); + --pos; + goto get_field; + } } else { +get_field: + col = dict_index_get_nth_col(index, pos); field = rec_get_nth_cfield( rec, index, offsets, pos, &flen); } +write_field: + /* Write field number to undo log */ + ptr += mach_write_compressed(ptr, pos); if (trx_undo_left(undo_block, ptr) < 15) { - return(0); + return 0; } - if (!is_virtual && rec_offs_nth_extern(offsets, pos)) { - const dict_col_t* col - = dict_index_get_nth_col(index, pos); - ulint prefix_len - = dict_max_field_len_store_undo( - table, col); + if (rec_offs_n_fields(offsets) > pos + && rec_offs_nth_extern(offsets, pos)) { + ut_ad(col || pos == index->first_user_field()); + ut_ad(col || update->is_alter_metadata()); + ut_ad(col + || rec_is_alter_metadata(rec, *index)); + ulint prefix_len = col + ? dict_max_field_len_store_undo( + table, col) + : 0; ut_ad(prefix_len + BTR_EXTERN_FIELD_REF_SIZE <= sizeof ext_buf); ptr = trx_undo_page_report_modify_ext( ptr, - col->ord_part + col + && col->ord_part && !ignore_prefix && flen < REC_ANTELOPE_MAX_INDEX_COL_LEN ? ext_buf : NULL, prefix_len, @@ -1123,6 +1191,7 @@ trx_undo_page_report_modify( *type_cmpl_ptr |= TRX_UNDO_UPD_EXTERN; } else { +store_len: ptr += mach_write_compressed(ptr, flen); } @@ -1531,14 +1600,12 @@ trx_undo_update_rec_get_update( const byte* field; ulint len; - ulint field_no; ulint orig_len; - bool is_virtual; upd_field = upd_get_nth_field(update, i); - field_no = mach_read_next_compressed(&ptr); + ulint field_no = mach_read_next_compressed(&ptr); - is_virtual = (field_no >= REC_MAX_N_FIELDS); + const bool is_virtual = (field_no >= REC_MAX_N_FIELDS); if (is_virtual) { /* If new version, we need to check index list to figure @@ -1561,15 +1628,33 @@ trx_undo_update_rec_get_update( } upd_field_set_v_field_no(upd_field, field_no, index); - } else if (field_no < index->n_fields) { - upd_field_set_field_no(upd_field, field_no, index); - } else if (update->info_bits == REC_INFO_MIN_REC_FLAG + } else if (UNIV_UNLIKELY((update->info_bits + & ~REC_INFO_DELETED_FLAG) + == REC_INFO_MIN_REC_FLAG) && index->is_instant()) { - /* This must be a rollback of a subsequent - instant ADD COLUMN operation. This will be - detected and handled by btr_cur_trim(). */ + const ulint uf = index->first_user_field(); + ut_ad(field_no >= uf); + + if (update->info_bits != REC_INFO_MIN_REC_FLAG) { + if (field_no == uf) { + upd_field->new_val.type + .metadata_blob_init(); + } else { + ut_ad(field_no > uf); + dict_col_copy_type( + dict_index_get_nth_col( + index, field_no - 1), + &upd_field->new_val.type); + } + } else { + dict_col_copy_type( + dict_index_get_nth_col(index, + field_no), + &upd_field->new_val.type); + } upd_field->field_no = field_no; - upd_field->orig_len = 0; + } else if (field_no < index->n_fields) { + upd_field_set_field_no(upd_field, field_no, index); } else { ib::error() << "Trying to access update undo rec" " field " << field_no @@ -1602,6 +1687,12 @@ trx_undo_update_rec_get_update( dfield_set_ext(&upd_field->new_val); } + ut_ad(update->info_bits != (REC_INFO_DELETED_FLAG + | REC_INFO_MIN_REC_FLAG) + || field_no != index->first_user_field() + || (upd_field->new_val.ext + && upd_field->new_val.len == FIELD_REF_SIZE)); + if (is_virtual) { upd_field->old_v_val = static_cast( mem_heap_alloc( @@ -1702,8 +1793,11 @@ trx_undo_rec_get_partial_row( if (uf->old_v_val) { continue; } - ulint c = dict_index_get_nth_col(index, uf->field_no)->ind; - *dtuple_get_nth_field(*row, c) = uf->new_val; + const dict_col_t& c = *dict_index_get_nth_col(index, + uf->field_no); + if (!c.is_dropped()) { + *dtuple_get_nth_field(*row, c.ind) = uf->new_val; + } } end_ptr = ptr + mach_read_from_2(ptr); @@ -1714,7 +1808,6 @@ trx_undo_rec_get_partial_row( const byte* field; ulint field_no; const dict_col_t* col; - ulint col_no; ulint len; ulint orig_len; bool is_virtual; @@ -1742,15 +1835,18 @@ trx_undo_rec_get_partial_row( dict_v_col_t* vcol = dict_table_get_nth_v_col( index->table, field_no); col = &vcol->m_col; - col_no = dict_col_get_no(col); dfield = dtuple_get_nth_v_field(*row, vcol->v_pos); dict_col_copy_type( &vcol->m_col, dfield_get_type(dfield)); } else { col = dict_index_get_nth_col(index, field_no); - col_no = dict_col_get_no(col); - dfield = dtuple_get_nth_field(*row, col_no); + + if (col->is_dropped()) { + continue; + } + + dfield = dtuple_get_nth_field(*row, col->ind); ut_ad(dfield->type.mtype == DATA_MISSING || dict_col_type_assert_equal(col, &dfield->type)); @@ -1758,9 +1854,7 @@ trx_undo_rec_get_partial_row( || dfield->len == len || (len != UNIV_SQL_NULL && len >= UNIV_EXTERN_STORAGE_FIELD)); - dict_col_copy_type( - dict_table_get_nth_col(index->table, col_no), - dfield_get_type(dfield)); + dict_col_copy_type(col, dfield_get_type(dfield)); } dfield_set_data(dfield, field, len);