mariadb

Commit Graph

Author	SHA1	Message	Date
Monty	066f920484	MDEV-35110 Deadlock on Replica during BACKUP STAGE BLOCK_COMMIT on XA transactions This is an extension of MDEV-30423 "Deadlock on Replica during BACKUP STAGE BLOCK_COMMIT on XA transactions" The original commit in MDEV-30423 was not complete as some usage in XA of MDL_BACKUP_COMMIT locks did not set thd->backup_commit_lock. This is required to be set when using parallel replication. Fixed by ensuring that all usage of BACKUP_COMMIT lock i XA is uniform and all sets thd->backup_commit_lock. I also changed all locks to be MDL_EXPLICIT to keep also that part uniform. A regression test is added.	1 year ago
Vlad Lesin	8c7786e7d5	MDEV-34690 lock_rec_unlock_unmodified() causes deadlock lock_rec_unlock_unmodified() is executed either under lock_sys.wr_lock() or under a combination of lock_sys.rd_lock() + record locks hash table cell latch. It also requests page latch to check if locked records were changed by the current transaction or not. Usually InnoDB requests page latch to find the certain record on the page, and then requests lock_sys and/or record lock hash cell latch to request record lock. lock_rec_unlock_unmodified() requests the latches in the opposite order, what causes deadlocks. One of the possible scenario for the deadlock is the following: thread 1 - lock_rec_unlock_unmodified() is invoked under locks hash table cell latch, the latch is acquired; thread 2 - purge thread acquires page latch and tries to remove delete-marked record, it invokes lock_update_delete(), which requests locks hash table cell latch, held by thread 1; thread 1 - requests page latch, held by thread 2. To fix it we need to release lock_sys.latch and/or lock hash cell latch, acquire page latch and re-acquire lock_sys related latches. When lock_sys.latch and/or lock hash cell latch are released in lock_release_on_prepare() and lock_release_on_prepare_try(), the page on which the current lock is held, can be merged. In this case the bitmap of the current lock must be cleared, and the new lock must be added to the end of trx->lock.trx_locks list, or bitmap of already existing lock must be changed. The new field trx_lock_t::set_nth_bit_calls indicates if new locks (bits in existing lock bitmaps or new lock objects) were created during the period when lock_sys was released in trx->lock.trx_locks list iteration loop in lock_release_on_prepare() or lock_release_on_prepare_try(). And, if so, we traverse the list again. The block can be freed during pages merging, what causes assertion failure in buf_page_get_gen(), as btr_block_get() passes BUF_GET as page get mode to it. That's why page_get_mode parameter was added to btr_block_get() to pass BUF_GET_POSSIBLY_FREED from lock_release_on_prepare() and lock_release_on_prepare_try() to buf_page_get_gen(). As searching for id of trx, which modified secondary index record, is quite expensive operation, restrict its usage for master. System variable was added to remove the restriction for testing simplifying. The variable exists only either for debug build or for build with -DINNODB_ENABLE_XAP_UNLOCK_UNMODIFIED_FOR_PRIMARY option to increase the probability of catching bugs for release build with RQG. Note that the code, which does primary index lookup to find out what transaction modified secondary index record, is necessary only when there is no primary key and no unique secondary key on replica with row based replication, because only in this case extra X locks on unmodified records can be set during scan phase. Reviewed by Marko Mäkelä.	1 year ago
Vlad Lesin	92180ad513	MDEV-34466 XA prepare don't release unmodified records for some cases There is no need to exclude exclusive non-gap locks from the procedure of locks releasing on XA PREPARE execution in lock_release_on_prepare_try() after commit `17e59ed3aa` (MDEV-33454), because lock_rec_unlock_unmodified() should check if the record was modified with the XA, and release the lock if it was not. lock_release_on_prepare_try(): don't skip X-locks, let lock_rec_unlock_unmodified() to process them. lock_sec_rec_some_has_impl(): add template parameter for not acquiring trx_t::mutex for the case if a caller already holds the mutex, don't crash if lock's bitmap is clean. row_vers_impl_x_locked(), row_vers_impl_x_locked_low(): add new argument to skip trx_t::mutex acquiring. rw_trx_hash_t::validate_element(): don't acquire trx_t::mutex if the current thread already holds it. Thanks to Andrei Elkin for finding the bug. Reviewed by Marko Mäkelä, Debarun Banerjee.	1 year ago
Marko Mäkelä	1cad1dbde6	MDEV-35235 innodb_snapshot_isolation=ON fails to signal transaction rollback convert_error_code_to_mysql(): Treat DB_DEADLOCK and DB_RECORD_CHANGED in the same way, that is, signal to the SQL layer that the transaction had been rolled back.	1 year ago
Jan Lindström	b3be3c2157	MDEV-30653 : With wsrep_mode=REPLICATE_ARIA only part of mixed-engine transactions is replicated Replication of non-transactional engines is experimental and uses TOI. This naturally means that if there is open transaction with transactional engine it's changes will be rolled back. Fixed by adding error message if non-transactional engine is part of multi-engine transaction with warning. Signed-off-by: Julius Goryavsky <julius.goryavsky@mariadb.com>	1 year ago
Marko Mäkelä	b38edd09ff	MDEV-34830 fixup: Relax an assertion This follows up `1067046b7f`	1 year ago
Kristian Nielsen	45537939e7	MDEV-34859: Pass thorugh -DWITH_BOOST_CONTEXT to libmariadb Signed-off-by: Kristian Nielsen <knielsen@knielsen-hq.org>	1 year ago
Marko Mäkelä	1067046b7f	MDEV-34830 fixup: Relax an assertion It is possible that recv_sys.scanned_lsn is ahead of recv_sys.recovered_lsn by a few 512-byte log blocks in case the last mini-transaction in the log had not been written out completely before the server was killed. This is occasionally the case when running the test innodb.innodb-32k-crash.	1 year ago
Marko Mäkelä	bea4adcb5a	MDEV-35225 Bogus debug assertion failures in innodb.innodb-32k-crash log_sort_flush_list(): Correct some debug assertions that had been added in commit `0d175968d1` (MDEV-31354). The writes of some blocks may be completed and the oldest_modification() set to 1 at any time. The bogus assertion failures led to occasional failures of the test innodb.innodb-32k-crash.	1 year ago
Brandon Nesterenko	1ed30e08af	MDEV-34122: Assertion `entry' failed in Active_tranx::assert_thd_is_waiter If semi-sync is switched off then on while a transaction is in-between binlogging and waiting for an ACK, the semi-sync state of the transaction is removed, leading to a debug assertion that indicates the transaction tried to wait, but cannot receive an ACK signal. More specifically, when semi-sync is switched off, the Active_tranx list is cleared (where a transaction adds an entry to this list during binlogging), and each entry in this list saves the thread which will wait for an ACK, and the thread has the COND variable to signal to wake itself. So if the entry is lost, the Ack_receiver thread won’t be able to find the thread to wake up when an ACK comes in The fix is to ensure that the entry exists before awaiting the ACK, and if there is no entry, skip the wait. In debug builds, an informative message is written explaining that the transaction is skipping its wait. Additional debug-build only logic is added to ensure that the cause of the missing entry is due to semi-sync being turned off and on Reviewed By: ============ Kristian Nielsen <knielsen@knielsen-hq.org>	1 year ago
Vladislav Vaintroub	e8db5c8760	MDEV-35171 OS_FILE_NORMAL and OS_FILE_AIO are misleading Removed 'purpose' parameter from os_file_create() and related functions. Always use FILE_FLAG_OVERLAPPED when opening Windows files. No performance regression was measured, nor there is any measurable improvement.	1 year ago
Marko Mäkelä	7701ccb72d	MDEV-35149 Race condition around SET GLOBAL innodb_lru_scan_depth A debug assertion in buf_LRU_get_free_block() could fail if SET GLOBAL innodb_lru_scan_depth is being executed during a workload that involves allocating buffer pool pages. buf_pool_t::LRU_scan_depth: Replaces srv_LRU_scan_depth. buf_pool_t::flush_neighbors: Replaces srv_flush_neighbors. innodb_buf_pool_update<T>(): Update a parameter of buf_pool while holding buf_pool.mutex.	1 year ago
Sergei Golubchik	70d8ce63c7	update C/C	1 year ago
Sergei Petrunia	a68e74b5a4	MDEV-35164: optimizer_join_limit_pref_ratio: assertion when the ORDER BY table becomes constant Assertion failure has happened due to this scenario: A query was ran with optimizer_join_limit_pref_ratio=1. The query had "ORDER BY t1.col LIMIT N". The optimizer set join->limit_shortcut_applicable=1. Then, table t1 was marked as constant. The code in choose_query_plan() still set join->limit_optimization_mode=1 which caused the optimizer to only consider t1 as the first non-const table. But t1 was already put into the join prefix as the constant table. The optimizer couldn't produce any join order at all and crashed. Fixed by not searching for shortcut plan if ORDER BY table is a constant. We will not try to do sorting anyway in this case (and LIMIT short-cutting will be done for any join order).	1 year ago
Sergei Petrunia	0540eac05c	MDEV-35180: ref_to_range rewrite causes poor query plan (Variant 2: only allow rewrite for ref(const)) make_join_select() has a "ref_to_range" rewrite: it would rewrite any ref access to a range access on the same index if the latter uses more keyparts. It seems, he initial intent of this was to fix poor query plan choice in cases like t.keypart1=const AND t.keypart2 < 'foo' Due to deficiency in cost model, ref access could be picked while range would enumerate fewer rows and be cheaper. However, the condition also forces a rewrite in cases like: t.keypart1=prev_table.col AND t.keypart1<='foo' AND t.keypart2<'bar' Here, it can be that * keypart1=prev_table.col is highly selective * (keypart1, keypart2) <= ('foo', 'bar') is not at all selective. Still, the rewrite would be made and poor query plan chosen. Fixed this by only doing the rewrite if ref access was ref(const) so we can be certain that quick select also used these restrictions and will scan a subset of rows that ref access would scan.	1 year ago
Sergei Golubchik	3a1cf2c85b	MDEV-34679 ER_BAD_FIELD uses non-localizable substrings	1 year ago
Sergei Golubchik	99178311ac	don't disable lto in DEB builds	2 years ago
Sergei Golubchik	a20c79da13	MDEV-25633 MariaDB crashes when compiled with link time optimizations don't force -static-libgcc in gcc builds gcc can decide automatically if it's needed. forcing static libgcc breaks lto.	2 years ago
Sergei Golubchik	3e3fdb25f1	better disable lto for libmysqld_exports.cc e.g. also when -flto is set in CMAKE_CXX_FLAGS_RELWITHDEBINFO	2 years ago
Sergei Golubchik	bfa15f9b34	C/C compilation failures under -flto	1 year ago
Sergei Golubchik	3693fb9581	MDEV-25199 cluster fails to start up if you need innodb in your test - enable it yourself	1 year ago
Sergei Golubchik	e1e836fc76	update results after the merge	1 year ago
Marko Mäkelä	bb47e575de	MDEV-34830: LSN in the future is not being treated as serious corruption The invariant of write-ahead logging is that before any change to a page is written to the data file, the corresponding log record must must first have been durably written. On crash recovery, there were some sloppy checks for this. Let us implement accurate checks and flag an inconsistency as a hard error, so that we can avoid further corruption of a corrupted database. For data extraction from the corrupted database, innodb_force_recovery can be used. Before recovery is reading any data pages or invoking buf_dblwr_t::recover() to recover torn pages from the doublewrite buffer, InnoDB will have parsed the log until the final LSN and updated log_sys.lsn to that. So, we can rely on log_sys.lsn at all times. The doublewrite buffer recovery has been refactored in such a way that the recv_sys.dblwr.pages may be consulted while discovering files and their page sizes, but nothing will be written back to data files before buf_dblwr_t::recover() is invoked. A section of the test mariabackup.innodb_redo_overwrite that is parsing some mariadb-backup --backup output has been removed, because that output "redo log block is overwritten" would often be missing in a Microsoft Windows environment as a result of these changes. recv_max_page_lsn, recv_lsn_checks_on: Remove. recv_sys_t::validate_checkpoint(): Validate the write-ahead-logging condition at the end of the recovery. recv_dblwr_t::validate_page(): Keep track of the maximum LSN (if we are checking a non-doublewrite copy of a page) but do not complain LSN being in the future. The doublewrite buffer is a special case, because it will be read early during recovery. Besides, starting with commit `762bcb81b5` the dblwr=true copies of pages may legitimately be "too new". recv_dblwr_t::find_page(): Find a valid page with the smallest FIL_PAGE_LSN that is in the valid range for recovery. recv_dblwr_t::restore_first_page(): Replaced by find_page(). Only buf_dblwr_t::recover() will write to data files. buf_dblwr_t::recover(): Simplify the message output. Do attempt doublewrite recovery on user page read error. Ignore doublewrite pages whose FIL_PAGE_LSN is outside the usable bounds. Previously, we could wrongly recover a too new page from the doublewrite buffer. It is unlikely that this could have lead to an actual error. Write back all recovered pages from the doublewrite buffer here, including for the first page of any tablespace. buf_page_is_corrupted(): Distinguish the return values CORRUPTED_FUTURE_LSN and CORRUPTED_OTHER. buf_page_check_corrupt(): Return the error code DB_CORRUPTION in case the LSN is in the future. Datafile::read_first_page(): Handle FSP_SPACE_FLAGS=0xffffffff in the same way on both 32-bit and 64-bit architectures. Datafile::read_first_page_flags(): Split from read_first_page(). Take a copy of the first page as a parameter. recv_sys_t::free_corrupted_page(): Take the file as a parameter and return whether a message was displayed. This avoids some duplicated and incomplete error messages. buf_page_t::read_complete(): Remove some redundant output and always display the name of the corrupted file. Never return DB_FAIL; use it only in internal error handling. IORequest::read_complete(): Assume that buf_page_t::read_complete() will have reported any error. fil_space_t::set_corrupted(): Return whether this is the first time the tablespace had been flagged as corrupted. Datafile::validate_first_page(), fil_node_open_file_low(), fil_node_open_file(), fil_space_t::read_page0(), fil_node_t::read_page0(): Add a parameter for a copy of the first page, and a parameter to indicate whether the FIL_PAGE_LSN check should be suppressed. Before buf_dblwr_t::recover() is invoked, we cannot validate the FIL_PAGE_LSN, but we can trust the FSP_SPACE_FLAGS and the tablespace ID that may be present in a potentially too new copy of a page. Reviewed by: Debarun Banerjee	1 year ago
Marko Mäkelä	740519e15a	MDEV-35125: Unnecessary buf_pool.page_hash lookups dict_index_t::clear(), btr_drop_temporary_table(): Make use of the root page guess if it is available. btr_read_autoinc(): Invoke btr_root_block_get() to access the root page. btr_blob_free(): Retain a buffer-fix on the page across mtr_t::commit() in order to avoid a buf_pool.page_hash lookup. dict_load_table_one(): Remove a redundant check for page id. It was already validated in buf_page_t::read_complete(). trx_t::apply_log(): Make use of buf_pool.page_fix() to avoid some mtr_t related overhead. Reviewed by: Thirunarayanan Balathandayuthapani	1 year ago
Sergei Petrunia	89493a9980	MDEV-34993: fix merge into 10.6: OPTIMIZER_ADJ_FIX_CARD_MULT should be ON by default	1 year ago
Marko Mäkelä	a4d2cc931d	MDEV-35174 Possible hang in trx_undo_prev_version() In commit `b7b9f3ce82` (MDEV-34515) we accidentally made the InnoDB MVCC code acquire a shared purge_sys.latch twice. Recursive shared latch acquisition may cause a deadlock of InnoDB threads if another thread in between will start waiting for an exclusive latch. purge_sys_t::latch: In debug builds, use srw_lock_debug instead of srw_spin_lock, so that bugs like this will result in debug assertion failures. trx_undo_report_row_operation(): Pass the view_guard to trx_undo_prev_version() and the rest of the arguments in the same order, so that the work to permute argument registers is minimized.	1 year ago
Sergei Petrunia	9849e3f948	MDEV-35072: Assertion with optimizer_join_limit_pref_ratio and 1-table select Pre-11.0 variant: 1. In recompute_join_cost_with_limit(), add an assertion that that partial_join_cost >= 0.0. 2. best_extension_by_limited_search() subtracts COST_EPS from join->best_read. But it is not subtracted from join->positions[0].read_time, add it back. 2. We could get very small negative partial_join_cost due to rounding errors. For fraction=1.0, we were computing essentially this (denote as EXPR-1): $row_read_cost + $where_cost - ($row_read_cost + $where_cost) which should compute to 0. But the computation was done in the following order (left-to-right): EXPR-2: ($row_read_cost + $where_cost) - $row_read_cost - $where_cost this produced a value of -1.1102230246251565e-16 due to a rounding error. Change the computation use EXPR-1 instead of EXPR-2.	1 year ago
Sergei Petrunia	5619f29384	Replace 0.001 with symbolic name COST_EPS optimize_straight_join and best_extension_by_limited_search() use 0.001 to make choice between plans with identical cost deterministic. Use COST_EPS instead of 0.001, like it's done in newer versions.	1 year ago
Yuchen Pei	cd5577ba4a	Merge branch '10.5' into 10.6	1 year ago
Yuchen Pei	77ed235d50	MDEV-26345 Spider GBH should execute original queries on the data node Stop skipping const items when selecting but skip them when storing their results to spider row to avoid storing in mismatching temporary table fields. Skip auxiliary fields in SELECTing, and do not store the (non-existing) results to the corresponding temporary table accordingly. When there are BOTH auxiliary fields AND const items in the auxiliary field items, do not use the spider GBH. This is a rare occasion if it happens at all and not worth the added complexity to cover it. Use the original item (item_ptr) in constructing GROUP BY and ORDER BY, which also means using item->name instead of field->field_name as aliases in constructing SELECT items. This fixes spurious regressions caused by the above changes in some tests using ORDER BY, such as mdev_24517.test. As a by-product, this also fixes MDEV-29546. Therefore we update mdev_29008.test to include the MDEV-29546 case.	1 year ago
Yuchen Pei	e6daff40e4	MDEV-32524 [fixup] Fixup of spider mem alloc enums missed in a previous merge The merge was 10.4.34->10.5.25	1 year ago
Nayuta Yanagisawa	6080e3af19	MDEV-26912 Spider: Remove dead code related to Oracle OCI Remove the dead-code, in Spider, which is related to the Spider's Oracle OCI support. The code has been disabled for a long time and it is unlikely that the code will be enabled.	4 years ago
Yuchen Pei	03a5c683f9	MDEV-27650 Spider: remove #ifdef SPIDER_HAS_GROUP_BY_HANDLER	1 year ago
Yuchen Pei	0a59aafc5f	MDEV-34659 Bound check in spider cast function query construction During spider query construction of certain cast functions, it locates the last occurrence of a keyword in the output of the Item::print() function and append from there to the constructed query so far. For example, consider the following query SELECT * FROM t2 ORDER BY CAST(c AS INET6); It constructs the following query and executes it at the data node (assuming the data node table is called t0). select cast(t0.`c` as inet6) ``,t0.`c` `c` from `test`.`t1` t0 order by `` When the construction has completed the initial part select cast(t0.`c` It then attempts to construct the " as inet6" part. To that end, it calls print() on the Item_typecast_fbt corresponding to the cast item, and obtains cast(`test`.`t2`.`c` as inet6) It then looks for " as ", and places cursor there for appending: cast(`test`.`t2`.`c` as inet6) ^ In this patch, if the search fails, i.e. there's no " as ...", we make sure that the cursor is not placed before the beginning of the string (out of bound). We also relax the search from " as char" to " as " in the case of CHAR_TYPECAST_FUNC, since there is more than one Item type with this func type. For example, "AS INET6" is an Item_typecast_fbt which has this func type.	1 year ago
Yuchen Pei	98a9c75ea3	MDEV-34659 Use evalp in CREATE SERVER's in init_spider.inc This was already fixed in higher versions.	1 year ago
Yuchen Pei	d3b84ff10d	MDEV-30067 Remove some overly enthusiastic asserts when deleting from a partitioned table When an DDL statement results in a local partition table with partitions not covering all values in the table, a failure is emitted. However, when the table in question is a spider table, the issue does not surface until some future statements (DELETE in the test examples in this commit) are executed. This is consistent with the design of spider which aims to minimise connections with the data node. The resulting error is legitimate and should not result in an assertion failure. Similarly, a partitioned spider table could have misplaced rows, so we remove the other assertion as well.	1 year ago
Yuchen Pei	8a52639ede	MDEV-34716 spider: some trivial cleanups and documentation - document tmp_share, which are temporary spider shares with only one link (no ha) - simplify spider_get_sys_tables_connect_info() where link_idx is always 0	1 year ago
Sergei Golubchik	d5f5062a48	our release packages on rhel8 use bundled readline for historical reasons, but we keep doing it for compatibility	1 year ago
Thirunarayanan Balathandayuthapani	5777d9f282	MDEV-35116 InnoDB fails to set error index for HA_ERR_NULL_IN_SPATIAL - InnoDB fails to set the index information or index number for the spatial index error HA_ERR_NULL_IN_SPATIAL. row_build_spatial_index_key(): Initialize the tmp_mbr array completely. check_if_supported_inplace_alter(): Fix the spelling mistake of alter	1 year ago
Oleksandr Byelkin	a79c9b3812	MDEV-35135 Assertion `!is_cond()' failed in Item_bool_func::val_int / do_select Change val_int with val_bool when it is a condition.	1 year ago
Oleksandr Byelkin	b138f428ea	MDEV-18151 Skipped error returning for GRANT/SET PASSWORD Make message of error not warning.	1 year ago
Oleksandr Byelkin	cc59fbfffa	MDEV-18151 Skipped error returning for GRANT/SET PASSWORD Make error issueing for GRANT and SET PASSWORD the same. Report errors wich were skipped before.	1 year ago
Oleksandr Byelkin	9cd9c3cf16	fix grant5 test to return to the original database.	1 year ago
Marko Mäkelä	cd97caef84	MDEV-33106: Null merge 10.5 into 10.6	1 year ago
Marko Mäkelä	8981ee238a	MDEV-33106 innodb.innodb-lock-inherit-read_commited times out Sometimes, in MariaDB Server 10.5 but apparently not in later branches, the test would hang because con1 and con2 would be blocked in debug_sync (for example, lock_wait_suspend_thread_enter and row_ins_sec_index_entry_dup_locks_created) and therefore blocking the purge of transactions from completing. To prevent an occasional DEBUG_SYNC induced hang in the test, we will wait for everything to be purged, except the last 2 transactions. This change should be null-merged to 10.6, because the test is not failing in 10.6 or later major versions.	1 year ago
Oleksandr Byelkin	1d0e94c55f	Merge branch '10.5' into 10.6	1 year ago
Thirunarayanan Balathandayuthapani	23820f1d79	MDEV-34392 Inplace algorithm violates the foreign key constraint - Fixing the compilation issue for the compiler lesser than gcc-6 Reviewed-by : Marko Mäkelä <marko.makela@mariadb.com>	1 year ago
Sergei Golubchik	4281e0068b	MDEV-35082 HANDLER with FULLTEXT keys is not always rejected need to check for index capabilities also for HANDLER READ NEXT	1 year ago
Sergei Golubchik	23d7dc1ea1	MDEV-35086 Trying to lock mutex when the mutex was already locked (session_tracker.cc), server hangs followup for `9021f40b8e`	1 year ago
Sergei Golubchik	3ea71a2c8e	MDEV-16699 heap-use-after-free in group_concat with compressed or GIS columns Field_blob::store() has special code for GROUP_CONCAT temporary table (to store blob values in Blob_mem_storage - this prevents them from being freed/overwritten when a next row is read). Field_geom and Field_blob_compressed inherit from Field_blob but they have their own ::store() method without this special Blob_mem_storage support. Considering that non-grouping CONCAT() of such fields converts them to plain BLOB, let's do the same for GROUP_CONCAT. To do it, Item_func_group_concat::setup will signal that it's creating a temporary table for GROUP_CONCAT, and Field_blog::make_new_field() override will create base Field_blob when under group concat.	1 year ago

1 2 3 4 5 ...

198847 Commits (066f920484bb186774d65e243ca2763a4293d866) All Branches Search

198847 Commits (066f920484bb186774d65e243ca2763a4293d866)

All Branches