mariadb

Tree: b23a109695

Author	SHA1	Message	Date
Jan Lindström	b23a109695	MDEV-11025: Make number of page cleaner threads variable dynamic New test cases innodb-page-cleaners Modified test cases innodb_page_cleaners_basic New function buf_flush_set_page_cleaner_thread_cnt Increase or decrease the amount of page cleaner worker threads. In case of increase this function creates based on current abount and requested amount how many new threads should be created. In case of decrease this function sets up the requested amount of threads and uses is_requested event to signal workers. Then we wait until all new treads are started, old threads that should exit signal is_finished or shutdown has marked that page cleaner should finish. buf_flush_page_cleaner_worker Store current thread id and thread_no and then signal event is_finished. If number of used page cleaner threads decrease we shut down those threads that have thread_no greater or equal than number of page configured page cleaners - 1 (note that there will be always page cleaner coordinator). Before exiting we signal is_finished. New function innodb_page_cleaners_threads_update Update function for innodb-page-cleaners system variable. innobase_start_or_create_for_mysql If more than one page cleaner threads is configured we use new function buf_flush_set_page_cleaner_thread_cnt to set up the requested threads (-1 coordinator).	8 years ago
Sergey Vojtovich	3722372ae5	MDEV-11549 - Twin include Removed duplicate include.	8 years ago
Marko Mäkelä	c65fdcf7c8	MDEV-14062 Upgrade from 10.0 fails on assertion during ALTER TABLE The fix in MDEV-14022 was incomplete. We must adjust some more code for the fact that SYS_INDEXES records may lack the column MERGE_THRESHOLD that was "instantly" added in MySQL 5.7 and MariaDB 10.2.2. dict_boot(): Hard-core SYS_INDEXES.MERGE_THRESHOLD as DEFAULT NULL. dict_index_t::instant_field_value(), rec_offs_make_valid(): Tolerate the missing SYS_INDEXES.MERGE_THRESHOLD column.	8 years ago
Eugene Kosov	f3ad3bbe77	fix data races	8 years ago
Sergey Vojtovich	7d2a7782fe	Fixed build failure	8 years ago
Eugene Kosov	62b21fc101	fix some data races (#464 ) * fix some data races	8 years ago
Eugene Kosov	a4c23ac720	remove unndeded class member variable	8 years ago
Eugene Kosov	bbfd18c691	fix unsynchronized read (#463 ) * fix unsynchronized read * use relaxed atomic instead of seq_cst	8 years ago
Marko Mäkelä	b3027d4065	MDEV-11369: Test ROW_FORMAT=COMPRESSED innodb.instant_alter: Demonstrate that instant ADD COLUMN is not supported for ROW_FORMAT=COMPRESSED but is for ROW_FORMAT=DYNAMIC. Test adding NOT NULL columns without explict DEFAULT using both ALGORITHM=COPY and ALGORITHM=INPLACE (so that implicit DEFAULT will be used).	8 years ago
Daniel Bartholomew	9a10e7fc6f	bump the VERSION	8 years ago
Marko Mäkelä	6edba392d8	Work around MDEV-14029 Server does not remove #sql.frm files after crash during ALTER TABLE For the two ALTER TABLE statements that are interrupted by killing the server, manually remove the #sql.frm files, because server startup will not remove the files.	8 years ago
Marko Mäkelä	1a4c63238c	MDEV-14022 Upgrade from 10.0/10.1 fails on assertion rec_offs_validate(), rec_get_offsets_func(): Tolerate SYS_INDEXES records where the last column MERGE_THRESHOLD is missing. This column was added in MySQL 5.7 and MariaDB 10.2.2. For tables that were created earlier, the column will be missing.	8 years ago
Sergei Golubchik	2d2f857fb3	fixes for --embedded	8 years ago
Eugene Kosov	e59d080ffa	fix a data race in debug build (#456 ) fix a data race in debug build This particular one flooded TSAN report.	8 years ago
Marko Mäkelä	a4948dafcd	MDEV-11369 Instant ADD COLUMN for InnoDB For InnoDB tables, adding, dropping and reordering columns has required a rebuild of the table and all its indexes. Since MySQL 5.6 (and MariaDB 10.0) this has been supported online (LOCK=NONE), allowing concurrent modification of the tables. This work revises the InnoDB ROW_FORMAT=REDUNDANT, ROW_FORMAT=COMPACT and ROW_FORMAT=DYNAMIC so that columns can be appended instantaneously, with only minor changes performed to the table structure. The counter innodb_instant_alter_column in INFORMATION_SCHEMA.GLOBAL_STATUS is incremented whenever a table rebuild operation is converted into an instant ADD COLUMN operation. ROW_FORMAT=COMPRESSED tables will not support instant ADD COLUMN. Some usability limitations will be addressed in subsequent work: MDEV-13134 Introduce ALTER TABLE attributes ALGORITHM=NOCOPY and ALGORITHM=INSTANT MDEV-14016 Allow instant ADD COLUMN, ADD INDEX, LOCK=NONE The format of the clustered index (PRIMARY KEY) is changed as follows: (1) The FIL_PAGE_TYPE of the root page will be FIL_PAGE_TYPE_INSTANT, and a new field PAGE_INSTANT will contain the original number of fields in the clustered index ('core' fields). If instant ADD COLUMN has not been used or the table becomes empty, or the very first instant ADD COLUMN operation is rolled back, the fields PAGE_INSTANT and FIL_PAGE_TYPE will be reset to 0 and FIL_PAGE_INDEX. (2) A special 'default row' record is inserted into the leftmost leaf, between the page infimum and the first user record. This record is distinguished by the REC_INFO_MIN_REC_FLAG, and it is otherwise in the same format as records that contain values for the instantly added columns. This 'default row' always has the same number of fields as the clustered index according to the table definition. The values of 'core' fields are to be ignored. For other fields, the 'default row' will contain the default values as they were during the ALTER TABLE statement. (If the column default values are changed later, those values will only be stored in the .frm file. The 'default row' will contain the original evaluated values, which must be the same for every row.) The 'default row' must be completely hidden from higher-level access routines. Assertions have been added to ensure that no 'default row' is ever present in the adaptive hash index or in locked records. The 'default row' is never delete-marked. (3) In clustered index leaf page records, the number of fields must reside between the number of 'core' fields (dict_index_t::n_core_fields introduced in this work) and dict_index_t::n_fields. If the number of fields is less than dict_index_t::n_fields, the missing fields are replaced with the column value of the 'default row'. Note: The number of fields in the record may shrink if some of the last instantly added columns are updated to the value that is in the 'default row'. The function btr_cur_trim() implements this 'compression' on update and rollback; dtuple::trim() implements it on insert. (4) In ROW_FORMAT=COMPACT and ROW_FORMAT=DYNAMIC records, the new status value REC_STATUS_COLUMNS_ADDED will indicate the presence of a new record header that will encode n_fields-n_core_fields-1 in 1 or 2 bytes. (In ROW_FORMAT=REDUNDANT records, the record header always explicitly encodes the number of fields.) We introduce the undo log record type TRX_UNDO_INSERT_DEFAULT for covering the insert of the 'default row' record when instant ADD COLUMN is used for the first time. Subsequent instant ADD COLUMN can use TRX_UNDO_UPD_EXIST_REC. This is joint work with Vin Chen (陈福荣) from Tencent. The design that was discussed in April 2017 would not have allowed import or export of data files, because instead of the 'default row' it would have introduced a data dictionary table. The test rpl.rpl_alter_instant is exactly as contributed in pull request #408. The test innodb.instant_alter is based on a contributed test. The redo log record format changes for ROW_FORMAT=DYNAMIC and ROW_FORMAT=COMPACT are as contributed. (With this change present, crash recovery from MariaDB 10.3.1 will fail in spectacular ways!) Also the semantics of higher-level redo log records that modify the PAGE_INSTANT field is changed. The redo log format version identifier was already changed to LOG_HEADER_FORMAT_CURRENT=103 in MariaDB 10.3.1. Everything else has been rewritten by me. Thanks to Elena Stepanova, the code has been tested extensively. When rolling back an instant ADD COLUMN operation, we must empty the PAGE_FREE list after deleting or shortening the 'default row' record, by calling either btr_page_empty() or btr_page_reorganize(). We must know the size of each entry in the PAGE_FREE list. If rollback left a freed copy of the 'default row' in the PAGE_FREE list, we would be unable to determine its size (if it is in ROW_FORMAT=COMPACT or ROW_FORMAT=DYNAMIC) because it would contain more fields than the rolled-back definition of the clustered index. UNIV_SQL_DEFAULT: A new special constant that designates an instantly added column that is not present in the clustered index record. len_is_stored(): Check if a length is an actual length. There are two magic length values: UNIV_SQL_DEFAULT, UNIV_SQL_NULL. dict_col_t::def_val: The 'default row' value of the column. If the column is not added instantly, def_val.len will be UNIV_SQL_DEFAULT. dict_col_t: Add the accessors is_virtual(), is_nullable(), is_instant(), instant_value(). dict_col_t::remove_instant(): Remove the 'instant ADD' status of a column. dict_col_t::name(const dict_table_t& table): Replaces dict_table_get_col_name(). dict_index_t::n_core_fields: The original number of fields. For secondary indexes and if instant ADD COLUMN has not been used, this will be equal to dict_index_t::n_fields. dict_index_t::n_core_null_bytes: Number of bytes needed to represent the null flags; usually equal to UT_BITS_IN_BYTES(n_nullable). dict_index_t::NO_CORE_NULL_BYTES: Magic value signalling that n_core_null_bytes was not initialized yet from the clustered index root page. dict_index_t: Add the accessors is_instant(), is_clust(), get_n_nullable(), instant_field_value(). dict_index_t::instant_add_field(): Adjust clustered index metadata for instant ADD COLUMN. dict_index_t::remove_instant(): Remove the 'instant ADD' status of a clustered index when the table becomes empty, or the very first instant ADD COLUMN operation is rolled back. dict_table_t: Add the accessors is_instant(), is_temporary(), supports_instant(). dict_table_t::instant_add_column(): Adjust metadata for instant ADD COLUMN. dict_table_t::rollback_instant(): Adjust metadata on the rollback of instant ADD COLUMN. prepare_inplace_alter_table_dict(): First create the ctx->new_table, and only then decide if the table really needs to be rebuilt. We must split the creation of table or index metadata from the creation of the dictionary table records and the creation of the data. In this way, we can transform a table-rebuilding operation into an instant ADD COLUMN operation. Dictionary objects will only be added to cache when table rebuilding or index creation is needed. The ctx->instant_table will never be added to cache. dict_table_t::add_to_cache(): Modified and renamed from dict_table_add_to_cache(). Do not modify the table metadata. Let the callers invoke dict_table_add_system_columns() and if needed, set can_be_evicted. dict_create_sys_tables_tuple(), dict_create_table_step(): Omit the system columns (which will now exist in the dict_table_t object already at this point). dict_create_table_step(): Expect the callers to invoke dict_table_add_system_columns(). pars_create_table(): Before creating the table creation execution graph, invoke dict_table_add_system_columns(). row_create_table_for_mysql(): Expect all callers to invoke dict_table_add_system_columns(). create_index_dict(): Replaces row_merge_create_index_graph(). innodb_update_n_cols(): Renamed from innobase_update_n_virtual(). Call my_error() if an error occurs. btr_cur_instant_init(), btr_cur_instant_init_low(), btr_cur_instant_root_init(): Load additional metadata from the clustered index and set dict_index_t::n_core_null_bytes. This is invoked when table metadata is first loaded into the data dictionary. dict_boot(): Initialize n_core_null_bytes for the four hard-coded dictionary tables. dict_create_index_step(): Initialize n_core_null_bytes. This is executed as part of CREATE TABLE. dict_index_build_internal_clust(): Initialize n_core_null_bytes to NO_CORE_NULL_BYTES if table->supports_instant(). row_create_index_for_mysql(): Initialize n_core_null_bytes for CREATE TEMPORARY TABLE. commit_cache_norebuild(): Call the code to rename or enlarge columns in the cache only if instant ADD COLUMN is not being used. (Instant ADD COLUMN would copy all column metadata from instant_table to old_table, including the names and lengths.) PAGE_INSTANT: A new 13-bit field for storing dict_index_t::n_core_fields. This is repurposing the 16-bit field PAGE_DIRECTION, of which only the least significant 3 bits were used. The original byte containing PAGE_DIRECTION will be accessible via the new constant PAGE_DIRECTION_B. page_get_instant(), page_set_instant(): Accessors for the PAGE_INSTANT. page_ptr_get_direction(), page_get_direction(), page_ptr_set_direction(): Accessors for PAGE_DIRECTION. page_direction_reset(): Reset PAGE_DIRECTION, PAGE_N_DIRECTION. page_direction_increment(): Increment PAGE_N_DIRECTION and set PAGE_DIRECTION. rec_get_offsets(): Use the 'leaf' parameter for non-debug purposes, and assume that heap_no is always set. Initialize all dict_index_t::n_fields for ROW_FORMAT=REDUNDANT records, even if the record contains fewer fields. rec_offs_make_valid(): Add the parameter 'leaf'. rec_copy_prefix_to_dtuple(): Assert that the tuple is only built on the core fields. Instant ADD COLUMN only applies to the clustered index, and we should never build a search key that has more than the PRIMARY KEY and possibly DB_TRX_ID,DB_ROLL_PTR. All these columns are always present. dict_index_build_data_tuple(): Remove assertions that would be duplicated in rec_copy_prefix_to_dtuple(). rec_init_offsets(): Support ROW_FORMAT=REDUNDANT records whose number of fields is between n_core_fields and n_fields. cmp_rec_rec_with_match(): Implement the comparison between two MIN_REC_FLAG records. trx_t::in_rollback: Make the field available in non-debug builds. trx_start_for_ddl_low(): Remove dangerous error-tolerance. A dictionary transaction must be flagged as such before it has generated any undo log records. This is because trx_undo_assign_undo() will mark the transaction as a dictionary transaction in the undo log header right before the very first undo log record is being written. btr_index_rec_validate(): Account for instant ADD COLUMN row_undo_ins_remove_clust_rec(): On the rollback of an insert into SYS_COLUMNS, revert instant ADD COLUMN in the cache by removing the last column from the table and the clustered index. row_search_on_row_ref(), row_undo_mod_parse_undo_rec(), row_undo_mod(), trx_undo_update_rec_get_update(): Handle the 'default row' as a special case. dtuple_t::trim(index): Omit a redundant suffix of an index tuple right before insert or update. After instant ADD COLUMN, if the last fields of a clustered index tuple match the 'default row', there is no need to store them. While trimming the entry, we must hold a page latch, so that the table cannot be emptied and the 'default row' be deleted. btr_cur_optimistic_update(), btr_cur_pessimistic_update(), row_upd_clust_rec_by_insert(), row_ins_clust_index_entry_low(): Invoke dtuple_t::trim() if needed. row_ins_clust_index_entry(): Restore dtuple_t::n_fields after calling row_ins_clust_index_entry_low(). rec_get_converted_size(), rec_get_converted_size_comp(): Allow the number of fields to be between n_core_fields and n_fields. Do not support infimum,supremum. They are never supposed to be stored in dtuple_t, because page creation nowadays uses a lower-level method for initializing them. rec_convert_dtuple_to_rec_comp(): Assign the status bits based on the number of fields. btr_cur_trim(): In an update, trim the index entry as needed. For the 'default row', handle rollback specially. For user records, omit fields that match the 'default row'. btr_cur_optimistic_delete_func(), btr_cur_pessimistic_delete(): Skip locking and adaptive hash index for the 'default row'. row_log_table_apply_convert_mrec(): Replace 'default row' values if needed. In the temporary file that is applied by row_log_table_apply(), we must identify whether the records contain the extra header for instantly added columns. For now, we will allocate an additional byte for this for ROW_T_INSERT and ROW_T_UPDATE records when the source table has been subject to instant ADD COLUMN. The ROW_T_DELETE records are fine, as they will be converted and will only contain 'core' columns (PRIMARY KEY and some system columns) that are converted from dtuple_t. rec_get_converted_size_temp(), rec_init_offsets_temp(), rec_convert_dtuple_to_temp(): Add the parameter 'status'. REC_INFO_DEFAULT_ROW = REC_INFO_MIN_REC_FLAG \| REC_STATUS_COLUMNS_ADDED: An info_bits constant for distinguishing the 'default row' record. rec_comp_status_t: An enum of the status bit values. rec_leaf_format: An enum that replaces the bool parameter of rec_init_offsets_comp_ordinary().	8 years ago
Eugene Kosov	3a418242df	fix TSAN build with Clang	8 years ago
Sergei Golubchik	a1a4e8eec1	Merge branch 'bb-10.2-ext' into 10.3	8 years ago
Sergei Golubchik	ac57a30bd9	MDEV-13675 filsort_priority_queue typo fixed	8 years ago
Sergei Golubchik	3e39771551	Merge branch '10.2' into bb-10.2-ext	8 years ago
Sergei Golubchik	08c493c62a	Merge branch '10.1' into 10.2	8 years ago
Sergei Golubchik	8898c1614d	cleanup: remove test include file, clarify the comment	8 years ago
Sergei Golubchik	a62ebf2590	cppcheck harmless warnings MDEV-13953 mariadb-10.2.9/storage/innobase/row/row0import.cc:3602: possible cut'n'paste error ? MDEV-13955 mariadb-10.2.9/sql/log_event.cc:7671: possible wrong operator ?	8 years ago
Sergei Golubchik	ebda6e958f	enable MongoDB support in CONNECT	8 years ago
Vladislav Vaintroub	93fb586572	Fix a truncations warning	8 years ago
Sergei Golubchik	9584c6753e	MDEV-12874 UPDATE statements with the same source and target update engine tests	8 years ago
Marko Mäkelä	2c1067166d	Merge bb-10.2-ext into 10.3	8 years ago
Marko Mäkelä	61b2618d3a	Merge 10.2 into bb-10.2-ext	8 years ago
Marko Mäkelä	6ca35c1428	Replace a non-ASCII character in a comment	8 years ago
Vladislav Vaintroub	8d413c32dc	Fix several truncation and formatting warnings.	8 years ago
Vladislav Vaintroub	4732767981	Fix Windows warnings : fix server_audit not to use my_win_open and Co functions. There is no reason to export IO this functionality from server, if it can be easily implemented with CRT. Note: for the sake of compatibility, we do not use _open() on Windows, but open file with CreateFile, with FILE_SHARE_DELETE flag, so it can be renamed or deleted when open. Also atomic append flag is used.	8 years ago
Vladislav Vaintroub	35a4591e12	Update C/C Fixes some warnings.	8 years ago
Sergei Golubchik	ac2db256d9	MDEV-12874 UPDATE statements with the same source and target fix --ps-protocol	8 years ago
halfspawn	1a74d12da6	MDEV-12874 UPDATE statements with the same source and target	8 years ago
Sergei Golubchik	26ff92f7ac	MDEV-13911 Support ORDER BY and LIMIT in multi-table update	8 years ago
Sergei Golubchik	b6a5be9eaa	cleanup: split multi_update test in two move binlog-dependent part (~6%) into a separate file, to be able to run the rest of multi_update with embedded server (include/have_log_bin.inc disables embedded)	8 years ago
Marko Mäkelä	b716231238	MDEV-13901 Assertion `!space->stop_new_ops' failed in TRUNCATE TABLE with many indexes fil_space_extend_must_retry(): If the table is being truncated, do not call fil_flush_low(). The operation is covered by the truncate log. File extension during TRUNCATE only occurs if there are many indexes on the table. With smaller innodb_page_size, the file extension occurs already with fewer indexes on the table.	8 years ago
Marko Mäkelä	1641879387	Merge 10.2 into bb-10.2-ext	8 years ago
Sergei Petrunia	ebc2f0dad3	Avoid using HA_POS_ERROR constant when passing around values of type double. This is error-prone and causes warnings on Windows	8 years ago
Jan Lindström	325e071c11	Merge pull request #457 from smtalk/10.2 wrep_sst_common and wsrep_sst_xtrabackup-v2 bugfixes to read sst parameters from configs	8 years ago
Alexander Barkov	64bfad6307	Fixing Item_func_hybrid_field_type::date_op(,uint) to date_op(,ulonglong) Fixing the data type for the "fuzzydate" parameter to Item_func_hybrid_field_type::date_op() from uint to ulonglong, for consistency with Item::get_date().	8 years ago
Marko Mäkelä	770231f355	Remove dict_disable_redo_if_temporary() The function dict_disable_redo_if_temporary() was supposed to disable redo logging for temporary tables. It was invoked unnecessarily for two read-only operations: row_undo_search_clust_to_pcur() and dict_stats_update_transient_for_index(). When a table is not temporary and not in the system tablespace, the tablespace should be flagged for MLOG_FILE_NAME logging. We do not need this overhead for temporary tables. Therefore, either mtr_t::set_log_mode() or mtr_t::set_named_space() should be invoked. dict_table_t::is_temporary(): Determine if a table is temporary. dict_table_is_temporary(): Redefined as a macro wrapper for dict_table_t::is_temporary(). dict_disable_redo_if_temporary(): Remove.	8 years ago
Vladislav Vaintroub	c3a44c2701	Fix compiler error on Win64 - do not truncate pointer in DBUG Fix truncation warning on Windows, in a populart header, with a cast.	8 years ago
Alice Sherepa	2cf3e2ea2f	Compressed columns tests (replication and partition)	8 years ago
Alexander Barkov	fcf631eafb	A cleanup for MDEV-10914 ROW data type for stored routine variables Changing datatypes for: - Item_spvar_args::m_table - sp_rcontext::m_var_table - return value of create_virtual_tmp_table() from TABLE* to Virtual_tmp_table* Advantages: - Stricter data type control - Removing the duplicate code (a loop with free_blobs) from destructors ~sp_rcontext() and ~Item_spvar_args(), using "delete m_(var_)table" in both instead. - Using Virtual_tmp_table::delete makes the code call Field::delete, which calls TRASH() for the freed fields, which is good for valgrind test runs.	8 years ago
Alexander Barkov	8ae8cd6348	Merge remote-tracking branch 'origin/10.2' into bb-10.2-ext	8 years ago
Marko Mäkelä	387bdf07ae	Remove MYSQL_REPLACE_TRX_IN_THD The method handlerton::replace_native_transaction_in_thd was added into MySQL 5.7 in WL#6860, Binlogging XA-prepared transaction. In MariaDB we do not have that change yet.	8 years ago
Marko Mäkelä	cc3057fde7	Remove dict_table_t::big_rows The field dict_table_t::big_rows was only used for determining if the adaptive hash index should be used when the internal InnoDB SQL parser is used. That parser is only used for modifying the InnoDB data dictionary, updating persistent tables, and for fulltext indexes.	8 years ago
sachin	bcda03b4fa	MDEV-13950 mysqld_safe could not start Galera node after upgrade ... This is regression caused by patch of mdev-10767. 1st problem :- mktmp is invoked without '-t' or specifing tmp directory. 2nd problem :- Since eval_log_error redirect stderr to stdout '2>' will return nothing. and hence $wr_logfile will be empty. Patch Credit:- Andrii Nikitin	8 years ago
Marko Mäkelä	d6f857ddbc	Remove a constant parameter commit=false	8 years ago
Marko Mäkelä	3c4cff3357	Merge 10.1 into 10.2	8 years ago

1 2 3 4 5 ...

179274 Commits (b23a1096956c21df037bd851494f11509b5514dd) All Branches Search

179274 Commits (b23a1096956c21df037bd851494f11509b5514dd)

All Branches