mariadb

Commit Graph

Author	SHA1	Message	Date
Sergey Vojtovich	ecd3ff9f24	MDEV-6089 - MySQL WL#7305 "Improve MDL scalability by using lock-free hash" Removed MDL map partitions. Won't be needed when this MDEV is implemented.	11 years ago
Sergey Vojtovich	903160ef59	MDEV-6089 - MySQL WL#7305 "Improve MDL scalability by using lock-free hash" Removed MDL objects cache. Won't be needed when this MDEV is implemented.	11 years ago
Sergei Golubchik	ce0ed977d5	MDEV-7186 get_lock() crashes on Windows, main.sp_notembedded and main.trigger_notembedded fail in buildbot user locks are not affected by lowed_case_table_names	11 years ago
Sergei Golubchik	2ae7541bcf	cleanup: s/const CHARSET_INFO/CHARSET_INFO/ as CHARSET_INFO is already const, using const on it is redundant and results in compiler warnings (on Windows)	11 years ago
Nirbhay Choubey	3e792e6cbc	MDEV-4018 : Feature Request: microseconds in GET_LOCK() Updated MDL_context's lock methods to accept double precision lock_wait_timeout.	11 years ago
Sergei Golubchik	3620910eea	cleanup: galera merge, simple changes	11 years ago
Jan Lindström	df4dd593f2	MDEV-6247: Merge 10.0-galera to 10.1. Merged lp:maria/maria-10.0-galera up to revision 3879. Added a new functions to handler API to forcefully abort_transaction, producing fake_trx_id, get_checkpoint and set_checkpoint for XA. These were added for future possiblity to add more storage engines that could use galera replication.	11 years ago
Jan Lindström	21b4dec072	Review fixes.	11 years ago
Jan Lindström	5ebb396638	MDEV-6247: Merge 10.0-galera to 10.1. Merged lp:maria/maria-10.0-galera up to revision 3867.	11 years ago
Sergei Golubchik	d81b662b8c	Asserting correct database name lettercase in various places in the code.	12 years ago
Sergey Vojtovich	e4fde57712	MDEV-5864 - Reduce usage of LOCK_open: TABLE_SHARE::tdc.free_tables Let TABLE_SHARE::tdc.free_tables, TABLE_SHARE::tdc.all_tables, TABLE_SHARE::tdc.flushed and corresponding invariants be protected by per-share TABLE_SHARE::tdc.LOCK_table_share instead of global LOCK_open.	12 years ago
Sergey Vojtovich	b95c8ce530	MDEV-5675 - Performance: my_hash_sort_bin is called too often Reduced number of my_hash_sort_bin() calls from 4 to 1 per query. Reduced number of memory accesses done by my_hash_sort_bin(). Details: - let MDL subsystem use pre-calculated hash value for hash inserts and deletes - let table cache use pre-calculated MDL hash value - MDL namespace is excluded from hash value calculation, so that hash value can be used by table cache as is - hash value for MDL is calculated as resulting hash value + MDL namespace - extended hash implementation to accept user defined hash function	12 years ago
Sergey Vojtovich	cd342e78b8	MDEV-4748 - metadata_lock_info plugin - restore find_ticket() private status, declare friend function instead - added metadata_lock_info to DEFAULT_SUITES	12 years ago
Kentoku SHIBA	3771e14ea9	add metadata_lock_info	12 years ago
Seppo Jaakola	b098b7a84c	bzr merge -r3904..3928 lp:codership-mysql/5.5 This is now otherwise on level wsrep-25.9, but storage/innobase has not been fully merged wsrep-5.5 is not good source for that, so we probably have to cherry pick innodb changes from wsrep-5.6	12 years ago
Seppo Jaakola	61bda9a2f0	Merged revisions 3425..3430 from mariadb-galera-5.5	12 years ago
Seppo Jaakola	0a9216835f	Initial merge result with mariaDB 10: lp:maria	13 years ago
Michael Widenius	8075b05b7d	More merge fixes: - mdl.cc and mdl.h merged completely - mysql_system_tables*.sql merged completely - Fixed wrong merge of lock_tables - Added some missing functions: - bool THD::notify_shared_lock() - Dynamic_array::pop, Dynamic_array::del - Added MDL_context_owner to THD - Added metadata_locks_hash_instances	13 years ago
Michael Widenius	5f1f2fc0e4	Applied all changes from Igor and Sanja	13 years ago
Alexey Botchkov	1a600125ff	MDEV-3917 multiple use locks (GET_LOCK) in one connection. The patch contributed by Konstantin Osipov applied. Native comments: Implement multiple user-level locks per connection. GET_LOCK() function in MySQL allows a connection to hold at most one user level lock. Taking a new lock automatically releases the old lock, if any. The limit of one lock per session existed since early versions of MySQL didn't have a deadlock detector for SQL locks. MDL patches in MySQL 5.5 added a deadlock detector, so starting from 5.5 it became possible to take multiple locks in any order -- a deadlock, should it occur, would be detected and an error returned to the client which closed the wait chain. This is exactly what is done in this patch: ULLs are moved to use MDL subsystem.	13 years ago
Michael Widenius	068c61978e	Temporary commit of 10.0-merge	13 years ago
Gleb Shchepa	e5424d196b	Bug #15948123 : SERVER WORKS INCORRECT WITH LONG TABLE ALIASES After-push cleanup: removal of unneeded assertions.	13 years ago
Gleb Shchepa	46cfbf35f8	Bug #15948123 : SERVER WORKS INCORRECT WITH LONG TABLE ALIASES Post-push cleanup: removal of unneeded assertions.	13 years ago
Gleb Shchepa	d538d394b8	Bug #15948123 : SERVER WORKS INCORRECT WITH LONG TABLE ALIASES Code in MDL subsystem assumes that identifiers of objects can't be longer than NAME_LEN characters. This assumption was broken when one tried to construct MDL_key based on table alias, which can have arbitrary length. Since MDL_key's (and MDL locks) are not really used for table aliases this patch changes code to not initialize MDL_key object for table list element representing aliases.	13 years ago
Gleb Shchepa	70cb820e2d	Bug #15948123 : SERVER WORKS INCORRECT WITH LONG TABLE ALIASES Code in MDL subsystem assumes that identifiers of objects can't be longer than NAME_LEN characters. This assumption was broken when one tried to construct MDL_key based on table alias, which can have arbitrary length. Since MDL_key's (and MDL locks) are not really used for table aliases this patch changes code to not initialize MDL_key object for table list element representing aliases.	13 years ago
Sergei Golubchik	474fe6d9d9	fixes for test failures and small collateral changes mysql-test/lib/My/Test.pm: somehow with "print" we get truncated writes sometimes mysql-test/suite/perfschema/r/digest_table_full.result: md5 hashes of statement digests differ, because yacc token codes are different in mariadb mysql-test/suite/perfschema/r/dml_handler.result: host table is not ported over yet mysql-test/suite/perfschema/r/information_schema.result: host table is not ported over yet mysql-test/suite/perfschema/r/nesting.result: this differs, because we don't rewrite general log queries, and multi-statement packets are logged as a one entry. this result file is identical to what mysql-5.6.5 produces with the --log-raw option. mysql-test/suite/perfschema/r/relaylog.result: MariaDB modifies the binlog index file directly, while MySQL 5.6 has a feature "crash-safe binlog index" and modifies a special "crash-safe" shadow copy of the index file and then moves it over. That's why this test shows "NONE" index file writes in MySQL and "MANY" in MariaDB. mysql-test/suite/perfschema/r/server_init.result: MariaDB initializes the "manager" resources from the "manager" thread, and starts this thread only when --flush-time is not 0. MySQL 5.6 initializes "manager" resources unconditionally on server startup. mysql-test/suite/perfschema/r/stage_mdl_global.result: this differs, because MariaDB disables query cache when query_cache_size=0. MySQL does not do that, and this causes useless mutex locks and waits. mysql-test/suite/perfschema/r/statement_digest.result: md5 hashes of statement digests differ, because yacc token codes are different in mariadb mysql-test/suite/perfschema/r/statement_digest_consumers.result: md5 hashes of statement digests differ, because yacc token codes are different in mariadb mysql-test/suite/perfschema/r/statement_digest_long_query.result: md5 hashes of statement digests differ, because yacc token codes are different in mariadb mysql-test/suite/rpl/r/rpl_mixed_drop_create_temp_table.result: will be updated to match 5.6 when alfranio.correia@oracle.com-20110512172919-c1b5kmum4h52g0ni and anders.song@greatopensource.com-20110105052107-zoab0bsf5a6xxk2y are merged mysql-test/suite/rpl/r/rpl_non_direct_mixed_mixing_engines.result: will be updated to match 5.6 when anders.song@greatopensource.com-20110105052107-zoab0bsf5a6xxk2y is merged	13 years ago
Praveenkumar Hulakund	d0766534bd	Bug#13058122 - DML, LOCK/UNLOCK TABLES AND SELECT LEAD TO FOREVER MDL LOCK Analysis: ---------- While granting MDL lock for the lock requests in wait queue, first the lock is granted to the high priority lock types and then to the low priority lock types. MDL Priority Matrix, +-------------+----+---+---+---+----+-----+ \| Locks \| \| \| \| \| \| \| \| has Priority\| \| \| \| \| \| \| \| over ---> \| S \| SR\| SW\| SU\| SNW\| SNRW\| +-------------+----+---+---+---+----+-----+ \| X \| + \| + \| + \| + \| + \| + \| +-------------\|----\|---\|---\|---\|----\|-----\| \| SNRW \| - \| + \| + \| - \| - \| - \| +-------------\|----\|---\|---\|---\|----\|-----\| \| SNW \| - \| - \| + \| - \| - \| - \| +-------------+----+---+---+---+----+-----+ Here '+' means, Lock priority is higher. '-' means, Has same priority In the scenario where, . Lock wait queue has requests of type S/SR/SW/SU. . And locks of high priority X/SNRW/SNW are requested continuously. In this case, while granting lock, always first high priority lock requests(X/SNRW/SNW) are considered. Low priority locks(S/SR/SW/SU) will not get chance and they will wait forever. In the scenario for which this bug is reported, application executed many LOCK TABLES ... WRITE statements concurrently. These statements request SNRW lock. Also there were some connections trying to execute DML statements requesting SR lock. Since SNRW lock request has higher priority (and as they were too many waiting SNRW requests) lock is always granted to it. So, lock request SR will wait forever, resulting in DML starvation. How is this handled in 5.1? --------------------------- Even in 5.1 we have low priority lock starvation issue. But, in 5.1 thread locking, system variable "max_write_lock_count" can be configured to grant some pending read lock requests. After "max_write_lock_count" of write lock grants all the low priority locks are granted. Why this issue is seen in 5.5/trunk? --------------------------------- In 5.5/trunk MDL locking, "max_write_lock_count" system variable exists but not used in MDL, only thread lock uses it. So no effect of "max_write_lock_count" in MDL locking. This means that starvation of metadata locks is possible even if max_write_lock_count is used. Looks like, customer was using "max_write_lock_count" in 5.1 and when upgraded to 5.5, starvation is seen because of not having effect of "max_write_lock_count" in MDL. Fix: ---------- As a fix, support for max_write_lock_count is added to MDL. To maintain write lock counter per MDL_lock object, new member "m_hog_lock_count" is added in MDL_lock. And following logic is added to increment the counter in function reschedule_waiters, (reschedule_waiters function is called while thread is releasing the lock) - After granting lock request from the wait queue. - Check if there are any S/SR/SU/SW exists in the wait queue - If yes then increment the "m_hog_lock_count" And following logic is added in the same function to handle pending S/SU/SR/SW locks - Before granting locks - Check if max_write_lock_count <= m_hog_lock_count - If Yes, then try to grant S/SR/SW/SU locks. (Since all of these has same priority, all locks are granted together. But some lock grant may fail because of grant incompatibility) - Reset m_hog_lock_count if there no low priority lock requests in wait queue. - return Note: -------------------------- In the lock priority matrix explained above, though X has priority over the SNW and SNRW. X locks is taken mostly for RENAME, TRUNCATE, CREATE ... operations. So lock type X may not be requested in loop continuously in real world applications, as compared to other lock request types. So, lock request of type SNW and SNRW are not starved. So, we can grant all S/SR/SU/SW in one shot, without considering SNW & SNRW lock request starvation. ALTER table operations take SU lock first and then upgrade to SNW if required. All S, SR, SW, SU have same lock priority. So while granting SU, request of types SR, SW, S are also granted in one shot. So, lock request of type SU->SNW in loop will not make other low priority lock request to starve. But, when there is request for lock of type SNRW, lock requests of lower priority types are not granted. And if SNRW is requested in loop continuously then all S, SR, SW, SU are starved. This patch addresses the latter scenario. When we have S/SR/SW/SU in wait queue and if there are - Continuous SNRW lock requests - OR one or more X and Continuous SNRW lock requests. - OR one SNW and Continuous SNRW lock requests. - OR one SNW, one or more X and continuous SNRW lock requests. in wait queue then, S/SR/SW/SU lock request are starved.	13 years ago
Dmitry Lenev	082e0b957d	Fix for bug#12695572 - "IMPROVE MDL PERFORMANCE IN PRE-VISTA BY CACHING OR REDUCING CREATEEVENT CALLS". 5.5 versions of MySQL server performed worse than 5.1 versions under single-connection workload in autocommit mode on Windows XP. Part of this slowdown can be attributed to overhead associated with constant creation/destruction of MDL_lock objects in the MDL subsystem. The problem is that creation/destruction of these objects causes creation and destruction of associated synchronization primitives, which are expensive on Windows XP. This patch tries to alleviate this problem by introducing a cache of unused MDL_object_lock objects. Instead of destroying such objects we put them into the cache and then reuse with a new key when creation of a new object is requested. To limit the size of this cache, a new --metadata-locks-cache-size start-up parameter was introduced. mysql-test/r/mysqld--help-notwin.result: Updated test after adding --metadata-locks-cache-size parameter. mysql-test/r/mysqld--help-win.result: Updated test after adding --metadata-locks-cache-size parameter. mysql-test/suite/sys_vars/r/metadata_locks_cache_size_basic.result: Added test coverage for newly introduced --metadata_locks_cache_size start-up parameter and corresponding global read-only variable. mysql-test/suite/sys_vars/t/metadata_locks_cache_size_basic-master.opt: Added test coverage for newly introduced --metadata_locks_cache_size start-up parameter and corresponding global read-only variable. mysql-test/suite/sys_vars/t/metadata_locks_cache_size_basic.test: Added test coverage for newly introduced --metadata_locks_cache_size start-up parameter and corresponding global read-only variable. sql/mdl.cc: Introduced caching of unused MDL_object_lock objects, in order to avoid costs associated with constant creation and destruction of such objects in single-connection workloads run in autocommit mode. Such costs can be pretty high on systems where creation and destruction of synchronization primitives require a system call (e.g. Windows XP). To implement this cache,a list of unused MDL_object_lock instances was added to MDL_map object. Instead of being destroyed MDL_object_lock instances are put into this list and re-used later when creation of a new instance is required. Also added MDL_lock::m_version counter to allow threads having outstanding references to an MDL_object_lock instance to notice that it has been moved to the unused objects list. Added a global variable for a start-up parameter that limits the size of the unused objects list. Note that we don't cache MDL_scoped_lock objects since they are supposed to be created only during execution of DDL statements and therefore should not affect performance much. sql/mdl.h: Added a global variable for start-up parameter that limits the size of the unused MDL_object_lock objects list and constant for its default value. sql/sql_plist.h: Added I_P_List<>::pop_front() function. sql/sys_vars.cc: Introduced --metadata-locks-cache-size start-up parameter for specifying size of the cache of unused MDL_object_lock objects.	14 years ago
Dmitry Lenev	5b225518ff	Fix for bug #12641342 - "61401: UPDATE PERFORMANCE DEGRADES GRADUALLY IF A TRIGGER EXISTS". This bug manifested itself in two ways: - Firstly execution of any data-changing statement which required prelocking (i.e. involved stored function or trigger) as part of transaction slowed down a bit all subsequent statements in this transaction. So performance in transaction which periodically involved such statements gradually degraded over time. - Secondly execution of any data-changing statement which required prelocking as part of transaction prevented concurrent FLUSH TABLES WITH READ LOCK from proceeding until the end of transaction instead of end of particular statement. The problem was caused by incorrect handling of metadata lock used in FTWRL implementation for statements requiring prelocked mode. Each statement which changes data acquires global IX lock with STATEMENT duration. This lock is supposed to block concurrent FTWRL from proceeding until the statement ends. When entering prelocked mode, durations of all metadata locks acquired so far were changed to EXPLICIT, to prevent substatements from releasing these locks. When prelocked mode was left, durations of metadata locks were changed to TRANSACTIONAL (with a few exceptions) so they can be properly released at the end of transaction. Unfortunately, this meant that the global IX lock blocking FTWRL with STATEMENT duration was moved to TRANSACTIONAL duration after execution of statement requiring prelocking. Since each subsequent statement that required prelocking and tried to acquire global IX lock with STATEMENT duration got a new instance of MDL_ticket, which was later moved to TRANSACTIONAL duration, this led to unwarranted growth of number of tickets with TRANSACITONAL duration in this connection's MDL_context. As result searching for other tickets in it became slow and acquisition of other metadata locks by this transaction started to hog CPU. Moreover, this also meant that after execution of statement requiring prelocking concurrent FTWRL was blocked until the end of transaction instead of end of statement. This patch solves this problem by not moving locks to EXPLICIT duration when thread enters prelocked mode (unless it is a real LOCK TABLES mode). This step turned out to be not really necessary as substatements don't try to release metadata locks. Consequently, the global IX lock blocking FTWRL keeps its STATEMENT duration and is properly released at the end of statement and the above issue goes away. mysql-test/r/flush.result: Added test for bug #12641342 - "61401: UPDATE PERFORMANCE DEGRADES GRADUALLY IF A TRIGGER EXISTS". mysql-test/t/flush.test: Added test for bug #12641342 - "61401: UPDATE PERFORMANCE DEGRADES GRADUALLY IF A TRIGGER EXISTS". sql/mdl.h: Added comment describing various types of metadata lock duration. sql/sql_class.cc: Since we no longer change duration of metadata locks to EXPLICIT when entering prelocked mode (unless it is a real LOCK TABLES) there is no need to restore proper duration of the locks when leaving prelocked mode. sql/sql_class.h: Do not change duration of metadata locks to EXPLICIT when entering prelocking mode (unless it is a real LOCK TABLES). This allows to avoid problems with restoring correct duration when leaving this mode. It is possible to do this as substatements won't release metadata locks in any case. sql/sql_parse.cc: Added assert checking that we won't release metadata locks when in substatement.	15 years ago
Sergei Golubchik	0accbd0364	lots of post-merge changes	15 years ago
Davi Arnaut	754381c4b9	Move I_P_List_adapter to sql_plist.h as it is currently being used for code other then in mdl.h. Since the adapter is generic, it is better located in sql_plist.h.	15 years ago
Jon Olav Hauglid	6cae1bb971	This patch fixes the bug which caused the funcs_1.storedproc test case to give valgrind warnings. The problem was that when comparing two MDL key buffers using memcmp(), 1 was added to the buffer length. However, this was no longer needed since the buffer length already included the '\0' terminator.	15 years ago
Dmitry Lenev	6bf6272fda	Patch that refactors global read lock implementation and fixes bug #57006 "Deadlock between HANDLER and FLUSH TABLES WITH READ LOCK" and bug #54673 "It takes too long to get readlock for 'FLUSH TABLES WITH READ LOCK'". The first bug manifested itself as a deadlock which occurred when a connection, which had some table open through HANDLER statement, tried to update some data through DML statement while another connection tried to execute FLUSH TABLES WITH READ LOCK concurrently. What happened was that FTWRL in the second connection managed to perform first step of GRL acquisition and thus blocked all upcoming DML. After that it started to wait for table open through HANDLER statement to be flushed. When the first connection tried to execute DML it has started to wait for GRL/the second connection creating deadlock. The second bug manifested itself as starvation of FLUSH TABLES WITH READ LOCK statements in cases when there was a constant stream of concurrent DML statements (in two or more connections). This has happened because requests for protection against GRL which were acquired by DML statements were ignoring presence of pending GRL and thus the latter was starved. This patch solves both these problems by re-implementing GRL using metadata locks. Similar to the old implementation acquisition of GRL in new implementation is two-step. During the first step we block all concurrent DML and DDL statements by acquiring global S metadata lock (each DML and DDL statement acquires global IX lock for its duration). During the second step we block commits by acquiring global S lock in COMMIT namespace (commit code acquires global IX lock in this namespace). Note that unlike in old implementation acquisition of protection against GRL in DML and DDL is semi-automatic. We assume that any statement which should be blocked by GRL will either open and acquires write-lock on tables or acquires metadata locks on objects it is going to modify. For any such statement global IX metadata lock is automatically acquired for its duration. The first problem is solved because waits for GRL become visible to deadlock detector in metadata locking subsystem and thus deadlocks like one in the first bug become impossible. The second problem is solved because global S locks which are used for GRL implementation are given preference over IX locks which are acquired by concurrent DML (and we can switch to fair scheduling in future if needed). Important change: FTWRL/GRL no longer blocks DML and DDL on temporary tables. Before this patch behavior was not consistent in this respect: in some cases DML/DDL statements on temporary tables were blocked while in others they were not. Since the main use cases for FTWRL are various forms of backups and temporary tables are not preserved during backups we have opted for consistently allowing DML/DDL on temporary tables during FTWRL/GRL. Important change: This patch changes thread state names which are used when DML/DDL of FTWRL is waiting for global read lock. It is now either "Waiting for global read lock" or "Waiting for commit lock" depending on the stage on which FTWRL is. Incompatible change: To solve deadlock in events code which was exposed by this patch we have to replace LOCK_event_metadata mutex with metadata locks on events. As result we have to prohibit DDL on events under LOCK TABLES. This patch also adds extensive test coverage for interaction of DML/DDL and FTWRL. Performance of new and old global read lock implementations in sysbench tests were compared. There were no significant difference between new and old implementations. mysql-test/include/check_ftwrl_compatible.inc: Added helper script which allows to check that a statement is compatible with FLUSH TABLES WITH READ LOCK. mysql-test/include/check_ftwrl_incompatible.inc: Added helper script which allows to check that a statement is incompatible with FLUSH TABLES WITH READ LOCK. mysql-test/include/handler.inc: Adjusted test case to the fact that now DROP TABLE closes open HANDLERs for the table to be dropped before checking if there active FTWRL in this connection. mysql-test/include/wait_show_condition.inc: Fixed small error in the timeout message. The correct name of variable used as parameter for this script is "$condition" and not "$wait_condition". mysql-test/r/delayed.result: Added test coverage for scenario which triggered assert in metadata locking subsystem. mysql-test/r/events_2.result: Updated test results after prohibiting event DDL operations under LOCK TABLES. mysql-test/r/flush.result: Added test coverage for bug #57006 "Deadlock between HANDLER and FLUSH TABLES WITH READ LOCK". mysql-test/r/flush_read_lock.result: Added test coverage for various aspects of FLUSH TABLES WITH READ LOCK functionality. mysql-test/r/flush_read_lock_kill.result: Adjusted test case after replacing custom global read lock implementation with one based on metadata locks. Use new debug_sync point. Do not disable concurrent inserts as now InnoDB we always use InnoDB table. mysql-test/r/handler_innodb.result: Adjusted test case to the fact that now DROP TABLE closes open HANDLERs for the table to be dropped before checking if there active FTWRL in this connection. mysql-test/r/handler_myisam.result: Adjusted test case to the fact that now DROP TABLE closes open HANDLERs for the table to be dropped before checking if there active FTWRL in this connection. mysql-test/r/mdl_sync.result: Adjusted test case after replacing custom global read lock implementation with one based on metadata locks. Replaced usage of GRL-specific debug_sync's with appropriate sync points in MDL subsystem. mysql-test/suite/perfschema/r/dml_setup_instruments.result: Updated test results after removing global COND_global_read_lock condition variable. mysql-test/suite/perfschema/r/func_file_io.result: Ensure that this test doesn't affect subsequent tests. At the end of its execution enable back P_S instrumentation which this test disables at some point. mysql-test/suite/perfschema/r/func_mutex.result: Ensure that this test doesn't affect subsequent tests. At the end of its execution enable back P_S instrumentation which this test disables at some point. mysql-test/suite/perfschema/r/global_read_lock.result: Adjusted test case to take into account that new GRL implementation is based on MDL. mysql-test/suite/perfschema/r/server_init.result: Adjusted test case after replacing custom global read lock implementation with one based on MDL and replacing LOCK_event_metadata mutex with metadata lock. mysql-test/suite/perfschema/t/func_file_io.test: Ensure that this test doesn't affect subsequent tests. At the end of its execution enable back P_S instrumentation which this test disables at some point. mysql-test/suite/perfschema/t/func_mutex.test: Ensure that this test doesn't affect subsequent tests. At the end of its execution enable back P_S instrumentation which this test disables at some point. mysql-test/suite/perfschema/t/global_read_lock.test: Adjusted test case to take into account that new GRL implementation is based on MDL. mysql-test/suite/perfschema/t/server_init.test: Adjusted test case after replacing custom global read lock implementation with one based on MDL and replacing LOCK_event_metadata mutex with metadata lock. mysql-test/suite/rpl/r/rpl_tmp_table_and_DDL.result: Updated test results after prohibiting event DDL under LOCK TABLES. mysql-test/t/delayed.test: Added test coverage for scenario which triggered assert in metadata locking subsystem. mysql-test/t/events_2.test: Updated test case after prohibiting event DDL operations under LOCK TABLES. mysql-test/t/flush.test: Added test coverage for bug #57006 "Deadlock between HANDLER and FLUSH TABLES WITH READ LOCK". mysql-test/t/flush_block_commit.test: Adjusted test case after changing thread state name which is used when COMMIT waits for FLUSH TABLES WITH READ LOCK from "Waiting for release of readlock" to "Waiting for commit lock". mysql-test/t/flush_block_commit_notembedded.test: Adjusted test case after changing thread state name which is used when DML waits for FLUSH TABLES WITH READ LOCK. Now we use "Waiting for global read lock" in this case. mysql-test/t/flush_read_lock.test: Added test coverage for various aspects of FLUSH TABLES WITH READ LOCK functionality. mysql-test/t/flush_read_lock_kill-master.opt: We no longer need to use make_global_read_lock_block_commit_loop debug tag in this test. Instead we rely on an appropriate debug_sync point in MDL code. mysql-test/t/flush_read_lock_kill.test: Adjusted test case after replacing custom global read lock implementation with one based on metadata locks. Use new debug_sync point. Do not disable concurrent inserts as now InnoDB we always use InnoDB table. mysql-test/t/lock_multi.test: Adjusted test case after changing thread state names which are used when DML or DDL waits for FLUSH TABLES WITH READ LOCK to "Waiting for global read lock". mysql-test/t/mdl_sync.test: Adjusted test case after replacing custom global read lock implementation with one based on metadata locks. Replaced usage of GRL-specific debug_sync's with appropriate sync points in MDL subsystem. Updated thread state names which are used when DDL waits for FTWRL. mysql-test/t/trigger_notembedded.test: Adjusted test case after changing thread state names which are used when DML or DDL waits for FLUSH TABLES WITH READ LOCK to "Waiting for global read lock". sql/event_data_objects.cc: Removed Event_queue_element::status/last_executed_changed members and Event_queue_element::update_timing_fields() method. We no longer use this class for updating mysql.events once event is chosen for execution. Accesses to instances of this class in scheduler thread require protection by Event_queue::LOCK_event_queue mutex and we try to avoid updating table while holding this lock. sql/event_data_objects.h: Removed Event_queue_element::status/last_executed_changed members and Event_queue_element::update_timing_fields() method. We no longer use this class for updating mysql.events once event is chosen for execution. Accesses to instances of this class in scheduler thread require protection by Event_queue::LOCK_event_queue mutex and we try to avoid updating table while holding this lock. sql/event_db_repository.cc: - Changed Event_db_repository methods to not release all metadata locks once they are done updating mysql.events table. This allows to keep metadata lock protecting against GRL and lock protecting particular event around until corresponding DDL statement is written to the binary log. - Removed logic for conditional update of "status" and "last_executed" fields from update_timing_fields_for_event() method. In the only case when this method is called now "last_executed" is always modified and tracking change of "status" is too much hassle. sql/event_db_repository.h: Removed logic for conditional update of "status" and "last_executed" fields from Event_db_repository:: update_timing_fields_for_event() method. In the only case when this method is called now "last_executed" is always modified and tracking change of "status" field is too much hassle. sql/event_queue.cc: Changed event scheduler code not to update mysql.events table while holding Event_queue::LOCK_event_queue mutex. Doing so led to a deadlock with a new GRL implementation. This deadlock didn't occur with old implementation due to fact that code acquiring protection against GRL ignored pending GRL requests (which lead to GRL starvation). One of goals of new implementation is to disallow GRL starvation and so we have to solve problem with this deadlock in a different way. sql/events.cc: Changed methods of Events class to acquire protection against GRL while perfoming DDL statement and keep it until statement is written to the binary log. Unfortunately this step together with new GRL implementation exposed deadlock involving Events::LOCK_event_metadata and GRL. To solve it Events::LOCK_event_metadata mutex was replaced with a metadata lock on event. As a side-effect events DDL has to be prohibited under LOCK TABLES even in cases when mysql.events table was explicitly locked for write. sql/events.h: Replaced Events::LOCK_event_metadata mutex with a metadata lock on event. sql/ha_ndbcluster.cc: Updated code after replacing custom global read lock implementation with one based on MDL. Since MDL subsystem should now be able to detect deadlocks involving metadata locks and GRL there is no need for special handling of active GRL. sql/handler.cc: Replaced custom implementation of global read lock with one based on metadata locks. Consequently when doing commit instead of calling method of Global_read_lock class to acquire protection against GRL we simply acquire IX in COMMIT namespace. sql/lock.cc: Replaced custom implementation of global read lock with one based on metadata locks. This step allows to expose wait for GRL to deadlock detector of MDL subsystem and thus succesfully resolve deadlocks similar to one behind bug #57006 "Deadlock between HANDLER and FLUSH TABLES WITH READ LOCK". It also solves problem with GRL starvation described in bug #54673 "It takes too long to get readlock for 'FLUSH TABLES WITH READ LOCK'" since metadata locks used by GRL give preference to FTWRL statement instead of DML statements (if needed in future this can be changed to fair scheduling). Similar to old implementation of acquisition of GRL is two-step. During the first step we block all concurrent DML and DDL statements by acquiring global S metadata lock (each DML and DDL statement acquires global IX lock for its duration). During the second step we block commits by acquiring global S lock in COMMIT namespace (commit code acquires global IX lock in this namespace). Note that unlike in old implementation acquisition of protection against GRL in DML and DDL is semi-automatic. We assume that any statement which should be blocked by GRL will either open and acquires write-lock on tables or acquires metadata locks on objects it is going to modify. For any such statement global IX metadata lock is automatically acquired for its duration. To support this change: - Global_read_lock::lock/unlock_global_read_lock and make_global_read_lock_block_commit methods were changed accordingly. - Global_read_lock::wait_if_global_read_lock() and start_waiting_global_read_lock() methods were dropped. It is now responsibility of code acquiring metadata locks opening tables to acquire protection against GRL by explicitly taking global IX lock with statement duration. - Global variables, mutex and condition variable used by old implementation was removed. - lock_routine_name() was changed to use statement duration for its global IX lock. It was also renamed to lock_object_name() as it now also used to take metadata locks on events. - Global_read_lock::set_explicit_lock_duration() was added which allows not to release locks used for GRL when leaving prelocked mode. sql/lock.h: - Renamed lock_routine_name() to lock_object_name() and changed its signature to allow its usage for events. - Removed broadcast_refresh() function. It is no longer needed with new GRL implementation. sql/log_event.cc: Release metadata locks with statement duration at the end of processing legacy event for LOAD DATA. This ensures that replication thread processing such event properly releases its protection against global read lock. sql/mdl.cc: Changed MDL subsystem to support new MDL-based implementation of global read lock. Added COMMIT and EVENTS namespaces for metadata locks. Changed thread state name for GLOBAL namespace to "Waiting for global read lock". Optimized MDL_map::find_or_insert() method to avoid taking m_mutex mutex when looking up MDL_lock objects for GLOBAL or COMMIT namespaces. We keep pre-created MDL_lock objects for these namespaces around and simply return pointers to these global objects when needed. Changed MDL_lock/MDL_scoped_lock to properly handle notification of insert delayed handler threads when FTWRL takes global S lock. Introduced concept of lock duration. In addition to locks with transaction duration which work in the way which is similar to how locks worked before (i.e. they are released at the end of transaction), locks with statement and explicit duration were introduced. Locks with statement duration are automatically released at the end of statement. Locks with explicit duration require explicit release and obsolete concept of transactional sentinel. * Changed MDL_request and MDL_ticket classes to support notion of duration. * Changed MDL_context to keep locks with different duration in different lists. Changed code handling ticket list to take this into account. * Changed methods responsible for releasing locks to take into account duration of tickets. Particularly public MDL_context::release_lock() method now only can release tickets with explicit duration (there is still internal method which allows to specify duration). To release locks with statement or transaction duration one have to use release_statement/transactional_locks() methods. * Concept of savepoint for MDL subsystem now has to take into account locks with statement duration. Consequently MDL_savepoint class was introduced and methods working with savepoints were updated accordingly. * Added methods which allow to set duration for one or all locks in the context. sql/mdl.h: Changed MDL subsystem to support new MDL-based implementation of global read lock. Added COMMIT and EVENTS namespaces for metadata locks. Introduced concept of lock duration. In addition to locks with transaction duration which work in the way which is similar to how locks worked before (i.e. they are released at the end of transaction), locks with statement and explicit duration were introduced. Locks with statement duration are automatically released at the end of statement. Locks with explicit duration require explicit release and obsolete concept of transactional sentinel. * Changed MDL_request and MDL_ticket classes to support notion of duration. * Changed MDL_context to keep locks with different duration in different lists. Changed code handling ticket list to take this into account. * Changed methods responsible for releasing locks to take into account duration of tickets. Particularly public MDL_context::release_lock() method now only can release tickets with explicit duration (there is still internal method which allows to specify duration). To release locks with statement or transaction duration one have to use release_statement/transactional_locks() methods. * Concept of savepoint for MDL subsystem now has to take into account locks with statement duration. Consequently MDL_savepoint class was introduced and methods working with savepoints were updated accordingly. * Added methods which allow to set duration for one or all locks in the context. sql/mysqld.cc: Removed global mutex and condition variables which were used by old implementation of GRL. Also we no longer need to initialize Events::LOCK_event_metadata mutex as it was replaced with metadata locks on events. sql/mysqld.h: Removed global variable, mutex and condition variables which were used by old implementation of GRL. sql/rpl_rli.cc: When slave thread closes tables which were open for handling of RBR events ensure that it releases global IX lock which was acquired as protection against GRL. sql/sp.cc: Adjusted code to the new signature of lock_object/routine_name(), to the fact that one now needs specify duration of lock when initializing MDL_request and to the fact that savepoints for MDL subsystem are now represented by MDL_savepoint class. sql/sp_head.cc: Ensure that statements in stored procedures release statement metadata locks and thus release their protectiong against GRL in proper moment in time. Adjusted code to the fact that one now needs specify duration of lock when initializing MDL_request. sql/sql_admin.cc: Adjusted code to the fact that one now needs specify duration of lock when initializing MDL_request. sql/sql_base.cc: - Implemented support for new approach to acquiring protection against global read lock. We no longer acquire such protection explicitly on the basis of statement flags. Instead we always rely on code which is responsible for acquiring metadata locks on object to be changed acquiring this protection. This is achieved by acquiring global IX metadata lock with statement duration. Code doing this also responsible for checking that current connection has no active GRL by calling an Global_read_lock::can_acquire_protection() method. Changed code in open_table() and lock_table_names() accordingly. Note that as result of this change DDL and DML on temporary tables is always compatible with GRL (before it was incompatible in some cases and compatible in other cases). - To speed-up code acquiring protection against GRL introduced m_has_protection_against_grl member in Open_table_context class. It indicates that protection was already acquired sometime during open_tables() execution and new attempts can be skipped. - Thanks to new GRL implementation calls to broadcast_refresh() became unnecessary and were removed. - Adjusted code to the fact that one now needs specify duration of lock when initializing MDL_request and to the fact that savepoints for MDL subsystem are now represented by MDL_savepoint class. sql/sql_base.h: Adjusted code to the fact that savepoints for MDL subsystem are now represented by MDL_savepoint class. Also introduced Open_table_context::m_has_protection_against_grl member which allows to avoid acquiring protection against GRL while opening tables if such protection was already acquired. sql/sql_class.cc: Changed THD::leave_locked_tables_mode() after transactional sentinel for metadata locks was obsoleted by introduction of locks with explicit duration. sql/sql_class.h: - Adjusted code to the fact that savepoints for MDL subsystem are now represented by MDL_savepoint class. - Changed Global_read_lock class according to changes in global read lock implementation: * wait_if_global_read_lock and start_waiting_global_read_lock are now gone. Instead code needing protection against GRL has to acquire global IX metadata lock with statement duration itself. To help it new can_acquire_protection() was introduced. Also as result of the above change m_protection_count member is gone too. * Added m_mdl_blocks_commits_lock member to store metadata lock blocking commits. * Adjusted code to the fact that concept of transactional sentinel was obsoleted by concept of lock duration. - Removed CF_PROTECT_AGAINST_GRL flag as it is no longer necessary. New GRL implementation acquires protection against global read lock automagically when statement acquires metadata locks on tables or other objects it is going to change. sql/sql_db.cc: Adjusted code to the fact that one now needs specify duration of lock when initializing MDL_request. sql/sql_handler.cc: Removed call to broadcast_refresh() function. It is no longer needed with new GRL implementation. Adjusted code after introducing duration concept for metadata locks. Particularly to the fact transactional sentinel was replaced with explicit duration. sql/sql_handler.h: Renamed mysql_ha_move_tickets_after_trans_sentinel() to mysql_ha_set_explicit_lock_duration() after transactional sentinel was obsoleted by locks with explicit duration. sql/sql_insert.cc: Adjusted code handling delaying inserts after switching to new GRL implementation. Now connection thread initiating delayed insert has to acquire global IX lock in addition to metadata lock on table being inserted into. This IX lock protects against GRL and similarly to SW lock on table being inserted into has to be passed to handler thread in order to avoid deadlocks. sql/sql_lex.cc: LEX::protect_against_global_read_lock member is no longer necessary since protection against GRL is automatically taken by code acquiring metadata locks/opening tables. sql/sql_lex.h: LEX::protect_against_global_read_lock member is no longer necessary since protection against GRL is automatically taken by code acquiring metadata locks/opening tables. sql/sql_parse.cc: - Implemented support for new approach to acquiring protection against global read lock. We no longer acquire such protection explicitly on the basis of statement flags. Instead we always rely on code which is responsible for acquiring metadata locks on object to be changed acquiring this protection. This is achieved by acquiring global IX metadata lock with statement duration. This lock is automatically released at the end of statement execution. - Changed implementation of CREATE/DROP PROCEDURE/FUNCTION not to release metadata locks and thus protection against of GRL in the middle of statement execution. - Adjusted code to the fact that one now needs specify duration of lock when initializing MDL_request and to the fact that savepoints for MDL subsystem are now represented by MDL_savepoint class. sql/sql_prepare.cc: Adjusted code to the to the fact that savepoints for MDL subsystem are now represented by MDL_savepoint class. sql/sql_rename.cc: With new GRL implementation there is no need to explicitly acquire protection against GRL before renaming tables. This happens automatically in code which acquires metadata locks on tables being renamed. sql/sql_show.cc: Adjusted code to the fact that one now needs specify duration of lock when initializing MDL_request and to the fact that savepoints for MDL subsystem are now represented by MDL_savepoint class. sql/sql_table.cc: - With new GRL implementation there is no need to explicitly acquire protection against GRL before dropping tables. This happens automatically in code which acquires metadata locks on tables being dropped. - Changed mysql_alter_table() not to release lock on new table name explicitly and to rely on automatic release of locks at the end of statement instead. This was necessary since now MDL_context::release_lock() is supported only for locks for explicit duration. sql/sql_trigger.cc: With new GRL implementation there is no need to explicitly acquire protection against GRL before changing table triggers. This happens automatically in code which acquires metadata locks on tables which triggers are to be changed. sql/sql_update.cc: Fix bug exposed by GRL testing. During prepare phase acquire only S metadata locks instead of SW locks to keep prepare of multi-UPDATE compatible with concurrent LOCK TABLES WRITE and global read lock. sql/sql_view.cc: With new GRL implementation there is no need to explicitly acquire protection against GRL before creating view. This happens automatically in code which acquires metadata lock on view to be created. sql/sql_yacc.yy: LEX::protect_against_global_read_lock member is no longer necessary since protection against GRL is automatically taken by code acquiring metadata locks/opening tables. sql/table.cc: Adjusted code to the fact that one now needs specify duration of lock when initializing MDL_request. sql/table.h: Adjusted code to the fact that one now needs specify duration of lock when initializing MDL_request. sql/transaction.cc: Replaced custom implementation of global read lock with one based on metadata locks. Consequently when doing commit instead of calling method of Global_read_lock class to acquire protection against GRL we simply acquire IX in COMMIT namespace. Also adjusted code to the fact that MDL savepoint is now represented by MDL_savepoint class.	15 years ago
Dmitry Lenev	476939cb45	Reverted a temporary workaround for bug #56405 "Deadlock in the MDL deadlock detector". It is no longer needed as a better fix for this bug has been pushed.	15 years ago
Dmitry Lenev	f052fa3aad	A temporary workaround for bug #56405 "Deadlock in the MDL deadlock detector". Deadlock could have occurred when workload containing mix of DML, DDL and FLUSH TABLES statements affecting same set of tables was executed in heavily concurrent environment. This deadlock occurred when several connections tried to perform deadlock detection in metadata locking subsystem. The first connection started traversing wait-for graph, encountered sub-graph representing wait for flush, acquired LOCK_open and dived into sub-graph inspection. When it has encounterd sub-graph corresponding to wait for metadata lock and blocked while trying to acquire rd-lock on MDL_lock::m_rwlock (*) protecting this subgraph, since some other thread had wr-lock on it. When this wr-lock was released it could have happened (if there was other pending wr-lock against this rwlock) that rd-lock from the first connection was left unsatisfied but at the same time new rd-lock request from the second connection sneaked in and was satisfied (for this to be possible second rd- request should come exactly after wr-lock is released but before pending wr-lock manages to grab rwlock, which is possible both on Linux and in our own rwlock implementation). If this second connection continued traversing wait-for graph and encountered sub-graph representing wait for flush it tried to acquire LOCK_open and thus deadlock was created. This patch tries to workaround this problem but not allowing deadlock detector to lock LOCK_open mutex if some other thread doing deadlock detection already owns it and current search depth is greater than 0. Instead deadlock is reported. Other possible solutions are either known to have negative effects on performance or require much more time for proper implementation and testing. No test case is provided as this bug is very hard to repeat in MTR environment but is repeatable with the help of RQG tests. sql/mdl.cc: Moved Deadlock_detection_visitor::m_current_search_depth to parent class to make it available in TABLE_SHARE::visit_subgraph(). Added MDL_wait_for_graph_visitor::abort_traversal() method which allows to abort traversal of a wait-for graph and report a deadlock. sql/mdl.h: Moved Deadlock_detection_visitor::m_current_search_depth to parent class to make it available in TABLE_SHARE::visit_subgraph(). Added MDL_wait_for_graph_visitor::abort_traversal() method which allows to abort traversal of a wait-for graph and report a deadlock. sql/sql_base.cc: Added dd_owns_lock_open counter and mutex protecting it to track number of connections which do deadlock detection and own or try to acquire LOCK_open. sql/sql_base.h: Added dd_owns_lock_open counter and mutex protecting it to track number of connections which do deadlock detection and own or try to acquire LOCK_open. sql/table.cc: Workaround bug #56405 but not allowing MDL deadlock detector to lock LOCK_open mutex if some other thread doing deadlock detection already owns it and current search depth is greater than 0. Instead report deadlock.	15 years ago
Dmitry Lenev	8d0dc9b58b	Part of fix for bug#52044 "FLUSH TABLES WITH READ LOCK and FLUSH TABLES <list> WITH READ LOCK are incompatible" to be pushed as separate patch. Replaced thread state name "Waiting for table", which was used by threads waiting for a metadata lock or table flush, with a set of names which better reflect types of resources being waited for. Also replaced "Table lock" thread state name, which was used by threads waiting on thr_lock.c table level lock, with more elaborate "Waiting for table level lock", to make it more consistent with other thread state names. Updated test cases and their results according to these changes. Fixed sys_vars.query_cache_wlock_invalidate_func test to not to wait for timeout of wait_condition.inc script. mysql-test/r/query_cache.result: Added test coverage for query_cache_wlock_invalidate behavior for implicitly locked tables. mysql-test/suite/sys_vars/r/query_cache_wlock_invalidate_func.result: Fixed sys_vars.query_cache_wlock_invalidate_func test to not to wait for timeout of wait_condition.inc script. Reverted changes to test which introduced timeout and replaced waiting condition with a more appropriate one. Test coverage for query_cache_wlock_invalidate behavior for implicitly locked tables was added to query_cache.test. mysql-test/suite/sys_vars/t/query_cache_wlock_invalidate_func.test: Fixed sys_vars.query_cache_wlock_invalidate_func test to not to wait for timeout of wait_condition.inc script. Reverted changes to test which introduced timeout and replaced waiting condition with a more appropriate one. Test coverage for query_cache_wlock_invalidate behavior for implicitly locked tables was added to query_cache.test. mysql-test/t/query_cache.test: Added test coverage for query_cache_wlock_invalidate behavior for implicitly locked tables. mysys/thr_lock.c: Replaced "Table lock" thread state name, which was used by threads waiting on thr_lock.c table level lock, with more elaborate "Waiting for table level lock", to make it consistent with thread state names which are used while waiting for metadata locks and table flush. sql/mdl.cc: Replaced thread state name "Waiting for table", which was used by threads waiting for a metadata lock or table flush, with a set of names which better reflect types of resources being waited for. To implement this: - Adjusted MDL_wait::timed_wait() to take thread state name as parameter. - Introduced method of MDL_key class which allows to get thread state name to be used while waiting for resource corresponding to the key and changed code to use it. Added array translating namespaces to thread state names as part of this change. sql/mdl.h: To implement this: - Adjusted MDL_wait::timed_wait() to take thread state name as parameter. - Introduced method of MDL_key class which allows to get thread state name to be used while waiting for resource corresponding to the key and changed code to use it. Added array translating namespaces to thread state names as part of this change. sql/sql_base.cc: Replaced thread state name "Waiting for table", which was used by threads waiting for table flush, with a more elaborate "Waiting for table flush".	16 years ago
Dmitry Lenev	00496b7acd	Fix for bug #52044 "FLUSH TABLES WITH READ LOCK and FLUSH TABLES <list> WITH READ LOCK are incompatible". The problem was that FLUSH TABLES <list> WITH READ LOCK which was issued when other connection has acquired global read lock using FLUSH TABLES WITH READ LOCK was blocked and has to wait until global read lock is released. This issue stemmed from the fact that FLUSH TABLES <list> WITH READ LOCK implementation has acquired X metadata locks on tables to be flushed. Since these locks required acquiring of global IX lock this statement was incompatible with global read lock. This patch addresses problem by using SNW metadata type of lock for tables to be flushed by FLUSH TABLES <list> WITH READ LOCK. It is OK to acquire them without global IX lock as long as we won't try to upgrade those locks. Since SNW locks allow concurrent statements using same table FLUSH TABLE <list> WITH READ LOCK now has to wait until old versions of tables to be flushed go away after acquiring metadata locks. Since such waiting can lead to deadlock MDL deadlock detector was extended to take into account waits for flush and resolve such deadlocks. As a bonus code in open_tables() which was responsible for waiting old versions of tables to go away was refactored. Now when we encounter old version of table in open_table() we don't back-off and wait for all old version to go away, but instead wait for this particular table to be flushed. Such approach supported by deadlock detection should reduce number of scenarios in which FLUSH TABLES aborts concurrent multi-statement transactions. Note that active FLUSH TABLES <list> WITH READ LOCK still blocks concurrent FLUSH TABLES WITH READ LOCK statement as the former keeps tables open and thus prevents the latter statement from doing flush. mysql-test/include/handler.inc: Adjusted test case after changing status which is set when FLUSH TABLES waits for tables to be flushed from "Flushing tables" to "Waiting for table". mysql-test/r/flush.result: Added test which checks that "flush tables <list> with read lock" is compatible with active "flush tables with read lock" but not vice-versa. This test also covers bug #52044 "FLUSH TABLES WITH READ LOCK and FLUSH TABLES <list> WITH READ LOCK are incompatible". mysql-test/r/mdl_sync.result: Added scenarios in which wait for table to be flushed causes deadlocks to the coverage of MDL deadlock detector. mysql-test/suite/perfschema/r/dml_setup_instruments.result: Adjusted test results after removal of COND_refresh condition variable. mysql-test/suite/perfschema/r/server_init.result: Adjusted test and its results after removal of COND_refresh condition variable. mysql-test/suite/perfschema/t/server_init.test: Adjusted test and its results after removal of COND_refresh condition variable. mysql-test/t/flush.test: Added test which checks that "flush tables <list> with read lock" is compatible with active "flush tables with read lock" but not vice-versa. This test also covers bug #52044 "FLUSH TABLES WITH READ LOCK and FLUSH TABLES <list> WITH READ LOCK are incompatible". mysql-test/t/kill.test: Adjusted test case after changing status which is set when FLUSH TABLES waits for tables to be flushed from "Flushing tables" to "Waiting for table". mysql-test/t/lock_multi.test: Adjusted test case after changing status which is set when FLUSH TABLES waits for tables to be flushed from "Flushing tables" to "Waiting for table". mysql-test/t/mdl_sync.test: Added scenarios in which wait for table to be flushed causes deadlocks to the coverage of MDL deadlock detector. sql/ha_ndbcluster.cc: Adjusted code after adding one more parameter for close_cached_tables() call - timeout for waiting for table to be flushed. sql/ha_ndbcluster_binlog.cc: Adjusted code after adding one more parameter for close_cached_tables() call - timeout for waiting for table to be flushed. sql/lock.cc: Removed COND_refresh condition variable. See comment for sql_base.cc for details. sql/mdl.cc: Now MDL deadlock detector takes into account information about waits for table flushes when searching for deadlock. To implement this change: - Declaration of enum_deadlock_weight and Deadlock_detection_visitor were moved to mdl.h header to make them available to the code in table.cc which implements deadlock detector traversal through edges of waiters graph representing waiting for flush. - Since now MDL_context may wait not only for metadata lock but also for table to be flushed an abstract Wait_for_edge class was introduced. Its descendants MDL_ticket and Flush_ticket incapsulate specifics of inspecting waiters graph when following through edge representing wait of particular type. We no longer require global IX metadata lock when acquiring SNW or SNRW locks. Such locks are needed only when metadata locks of these types are upgraded to X locks. This allows to use SNW locks in FLUSH TABLES <list> WITH READ LOCK implementation and keep the latter compatible with global read lock. sql/mdl.h: Now MDL deadlock detector takes into account information about waits for table flushes when searching for deadlock. To implement this change: - Declaration of enum_deadlock_weight and Deadlock_detection_visitor were moved to mdl.h header to make them available to the code in table.cc which implements deadlock detector traversal through edges of waiters graph representing waiting for flush. - Since now MDL_context may wait not only for metadata lock but also for table to be flushed an abstract Wait_for_edge class was introduced. Its descendants MDL_ticket and Flush_ticket incapsulate specifics of inspecting waiters graph when following through edge representing wait of particular type. - Deadlock_detection_visitor now has m_table_shares_visited member which allows to support recursive locking for LOCK_open. This is required when deadlock detector inspects waiters graph which contains several edges representing waits for flushes or needs to come through the such edge more than once. sql/mysqld.cc: Removed COND_refresh condition variable. See comment for sql_base.cc for details. sql/mysqld.h: Removed COND_refresh condition variable. See comment for sql_base.cc for details. sql/sql_base.cc: Changed approach to how threads are waiting for table to be flushed. Now thread that wants to wait for old table to go away subscribes for notification by adding Flush_ticket to table's share and waits using MDL_context::m_wait object. Once table gets flushed (i.e. all tables are closed and table share is ready to be destroyed) all such waiters are notified individually. Thanks to this change MDL deadlock detector can take such waits into account. To implement this/as result of this change: - tdc_wait_for_old_versions() was replaced with tdc_wait_for_old_version() which waits for individual old share to go away and which is called by open_table() after finding out that share is outdated. We don't need to perform back-off before such waiting thanks to the fact that deadlock detector now sees such waits. - As result Open_table_ctx::m_mdl_requests became unnecessary and was removed. We no longer allocate copies of MDL_request objects on MEM_ROOT when MYSQL_OPEN_FORCE_SHARED/SHARED_HIGH_PRIO flags are in effect. - close_cached_tables() and tdc_wait_for_old_version() share code which implements waiting for share to be flushed - the both use TABLE_SHARE::wait_until_flush() method. Thanks to this close_cached_tables() supports timeouts and has extra parameter for this. - Open_table_context::OT_MDL_CONFLICT enum element was renamed to OT_CONFLICT as it is now also used in cases when back-off is required to resolve deadlock caused by waiting for flush and not metadata lock. - In cases when we discover that current connection tries to open tables from different generation we now simply back-off and restart process of opening tables. To support this Open_table_context::OT_REOPEN_TABLES enum element was added. - COND_refresh condition variable became unnecessary and was removed. - mysql_notify_thread_having_shared_lock() no longer wakes up connections waiting for flush as all such connections can be waken up by deadlock detector if necessary. sql/sql_base.h: - close_cached_tables() now has one more parameter - timeout for waiting for table to be flushed. - Open_table_context::OT_MDL_CONFLICT enum element was renamed to OT_CONFLICT as it is now also used in cases when back-off is required to resolve deadlock caused by waiting for flush and not metadata lock. Added new OT_REOPEN_TABLES enum element to be used in cases when we need to restart open tables process even in the middle of transaction. - Open_table_ctx::m_mdl_requests became unnecessary and was removed. sql/sql_class.h: Added assert ensuring that we won't use LOCK_open mutex with THD::enter_cond(). Otherwise deadlocks can arise in MDL deadlock detector. sql/sql_parse.cc: Changed FLUSH TABLES <list> WITH READ LOCK to take SNW metadata locks instead of X locks on tables to be flushed. Since we no longer require global IX lock to be taken when SNW locks are taken this makes this statement compatible with FLUSH TABLES WITH READ LOCK statement. Since SNW locks allow other connections to have table opened FLUSH TABLES <list> WITH READ LOCK now has to wait during open_tables() for old version to go away. Such waits can lead to deadlocks which will be detected by MDL deadlock detector which now takes waits for table to be flushed into account. Also adjusted code after adding one more parameter for close_cached_tables() call - timeout for waiting for table to be flushed. sql/sql_yacc.yy: FLUSH TABLES <list> WITH READ LOCK now needs only SNW metadata locks on tables. sql/sys_vars.cc: Adjusted code after adding one more parameter for close_cached_tables() call - timeout for waiting for table to be flushed. sql/table.cc: Implemented new approach to how threads are waiting for table to be flushed. Now thread that wants to wait for old table to go away subscribes for notification by adding Flush_ticket to table's share and waits using MDL_context::m_wait object. Once table gets flushed (i.e. all tables are closed and table share is ready to be destroyed) all such waiters are notified individually. This change allows to make such waits visible inside of MDL deadlock detector. To do it: - Added list of waiters/Flush_tickets to TABLE_SHARE class. - Changed free_table_share() to postpone freeing of share memory until last waiter goes away and to wake up subscribed waiters. - Added TABLE_SHARE::wait_until_flushed() method which implements subscription to the list of waiters for table to be flushed and waiting for this event. Implemented interface which allows to expose waits for flushes to MDL deadlock detector: - Introduced Flush_ticket class a descendant of Wait_for_edge class. - Added TABLE_SHARE::find_deadlock() method which allows deadlock detector to find out what contexts are still using old version of table in question (i.e. to find out what contexts are waited for by owner of Flush_ticket). sql/table.h: In order to support new strategy of waiting for table flush (see comment for table.cc for details) added list of waiters/Flush_tickets to TABLE_SHARE class. Implemented interface which allows to expose waits for flushes to MDL deadlock detector: - Introduced Flush_ticket class a descendant of Wait_for_edge class. - Added TABLE_SHARE::find_deadlock() method which allows deadlock detector to find out what contexts are still using old version of table in question (i.e. to find out what contexts are waited for by owner of Flush_ticket).	16 years ago
Jon Olav Hauglid	ec002090b9	Bug #55223 assert in Protocol::end_statement during CREATE DATABASE The problem was that a statement could cause an assert if it was aborted by KILL QUERY while it waited on a metadata lock. This assert checks that a statement either sends OK or an error to the client. If the bug was triggered on release builds, it caused OK to be sent to the client instead of ER_QUERY_INTERRUPTED. The root cause of the problem was that there are two separate ways to tell if a statement is killed: thd->killed and mysys_var->abort. KILL QUERY causes both to be set, thd->killed before mysys_var->abort. Also, both values are reset at the end of statement execution. This means that it is possible for KILL QUERY to first set thd->killed, then have the killed statement reset both thd->killed and mysys_var->abort and finally have KILL QUERY set mysys_var->abort. This means that the connection with the killed statement will start executing the next statement with the two values out of sync - i.e. thd->killed not set but mysys_var->abort set. Since mysys_var->abort is used to check if a wait for a metadata lock should be aborted, the next statement would immediately abort any such waiting. When waiting is aborted, no OK message is sent and thd->killed is checked to see if ER_QUERY_INTERRUPTED should be sent to the client. But since the->killed had been reset, neither OK nor an error message was sent to the client. This then triggered the assert. This patch fixes the problem by changing the metadata lock waiting code to check thd->killed. No test case added as reproducing the assert is dependent on very exact timing of two (or more) threads. The patch has been checked using RQG and the grammar posted on the bug report.	16 years ago
Dmitry Lenev	a6a8e10c79	A pre-requisite for patch fixing bug #52044 "FLUSH TABLES WITH READ LOCK and FLUSH TABLES <list> WITH READ LOCK are incompatible", which adds information about waits caused by FLUSH TABLES statement to deadlock detector in MDL subsystem. Remove API supporting caching of pointers to TABLE_SHARE object in MDL subsystem and all code related to it. The problem was that locking requirements of code implementing this API conflicted with locking requirements of code which adds information about waits caused by flushes to deadlock detector in MDL subsystem (the former needed to lock LOCK_open or its future equivalent while having write-lock on MDL_lock's rwlock, and the latter needs to be able to read-lock MDL_lock rwlock while owning LOCK_open or its future equivalent). Since caching of pointers to TABLE_SHARE objects in MDL subsystem didn't bring expected performance benefits we decided to remove caching API rather than try to come up with some complex solution for this problem.	16 years ago
Jon Olav Hauglid	9ff272fbbd	A 5.5 version of the fix for Bug #54360 "Deadlock DROP/ALTER/CREATE DATABASE with open HANDLER" Remove LOCK_create_db, database name locks, and use metadata locks instead. This exposes CREATE/DROP/ALTER DATABASE statements to the graph-based deadlock detector in MDL, and paves the way for a safe, deadlock-free implementation of RENAME DATABASE. Database DDL statements will now take exclusive metadata locks on the database name, while table/view/routine DDL statements take intention exclusive locks on the database name. This prevents race conditions between database DDL and table/view/routine DDL. (e.g. DROP DATABASE with concurrent CREATE/ALTER/DROP TABLE) By adding database name locks, this patch implements WL#4450 "DDL locking: CREATE/DROP DATABASE must use database locks" and WL#4985 "DDL locking: namespace/hierarchical locks". The patch also changes code to use init_one_table() where appropriate. The new lock_table_names() function requires TABLE_LIST::db_length to be set correctly, and this is taken care of by init_one_table(). This patch also adds a simple template to help work with the mysys HASH data structure. Most of the patch was written by Konstantin Osipov.	16 years ago
Konstantin Osipov	429454f76e	A new implementation for the TABLE_SHARE cache in MDL subsystem. Fix a number of caveates that the previous implementation suffered from, including unprotected access to shared data and lax resource accounting (share->ref_count) that could lead to deadlocks. The new implementation still suffers from a number of potential deadlocks in some edge cases, and this is still not enabled by default. Especially since performance testing has shown that it gives only marginable (not even exceeding measuring accuracy) improvements. @todo: - Remove calls to close_cached_tables() with REFRESH_FAST, and have_lock, because they break the MDL cache. - rework FLUSH TABLES <list> to not use close_cached_tables() - make sure that whenever we set TABLE_SHARE::version to 0 we free MDL cache references to it. sql/mdl.cc: We may cache references to TABLE_SHARE objects in MDL_lock objects for tables. Create a separate MDL_lock class to represent a table. sql/mdl.h: Adjust the MDL caching API to avoid races. sql/sql_base.cc: Move all caching functionality close together. Implement a solution for deadlocks caused by close_cached_tables() when MDL cache is enabled (incomplete). sql/sql_yacc.yy: Adjust FLUSH rule to do the necessary initialization of TABLE_LIST elements used in for FLUSH TABLES <list>, and thus work OK with flush_mdl_cache() function.	16 years ago
Dmitry Lenev	9dbd9ce185	Patch that changes approach to how we acquire metadata locks for DML statements and changes the way MDL locks are acquired/granted in contended case. Instead of backing-off when a lock conflict is encountered and waiting for it to go away before restarting open_tables() process we now wait for lock to be released without releasing any previously acquired locks. If conflicting lock goes away we resume opening tables. If waiting leads to a deadlock we try to resolve it by backing-off and restarting open_tables() immediately. As result both waiting for possibility to acquire and acquiring of a metadata lock now always happen within the same MDL API call. This has allowed to make release of a lock and granting it to the most appropriate pending request an atomic operation. Thanks to this it became possible to wake up during release of lock only those waiters which requests can be satisfied at the moment as well as wake up only one waiter in case when granting its request would prevent all other requests from being satisfied. This solves thundering herd problem which occured in cases when we were releasing some lock and woke up many waiters for SNRW or X locks (this was the issue in bug#52289 "performance regression for MyISAM in sysbench OLTP_RW test". This also allowed to implement more fair (FIFO) scheduling among waiters with the same priority. It also opens the door for introducing new types of requests for metadata locks such as low-prio SNRW lock which is necessary in order to support LOCK TABLES LOW_PRIORITY WRITE. Notice that after this sometimes can report ER_LOCK_DEADLOCK error in cases in which it has not happened before. Particularly we will always report this error if waiting for conflicting lock has happened in the middle of transaction and resulted in a deadlock. Before this patch the error was not reported if deadlock could have been resolved by backing off all metadata locks acquired by the current statement. mysql-test/r/mdl_sync.result: Added test coverage for some aspects of deadlock handling in metadata locking subsystem. Adjusted test case after removing back-off in general case when conflicting metadata lock is encountered during open_tables() (now this happens only if waiting for conflicting lock to go away leads to a deadlock). mysql-test/r/sp_sync.result: Adjusted test case after removing back-off in general case when conflicting metadata lock is encountered during open_tables() (now this happens only if waiting for conflicting lock to go away leads to a deadlock). mysql-test/suite/perfschema/r/dml_setup_instruments.result: Adjusted test results after renaming MDL_context:: m_waiting_for_lock rwlock to m_LOCK_waiting_for. mysql-test/suite/rpl/r/rpl_sp.result: Adjusted test case after implementing new approach to acquiring metadata locks in open_tables(). We no longer release all MDL locks acquired by statement before waiting for conflicting lock to go away. As result DROP FUNCTION statement has to wait for DML statement which managed to acquire metadata lock on function being dropped and now waits for other conflicting metadata lock to go away. mysql-test/suite/rpl/t/rpl_sp.test: Adjusted test case after implementing new approach to acquiring metadata locks in open_tables(). We no longer release all MDL locks acquired by statement before waiting for conflicting lock to go away. As result DROP FUNCTION statement has to wait for DML statement which managed to acquire metadata lock on function being dropped and now waits for other conflicting metadata lock to go away. mysql-test/t/mdl_sync.test: Added test coverage for some aspects of deadlock handling in metadata locking subsystem. Adjusted test case after removing back-off in general case when conflicting metadata lock is encountered during open_tables() (now this happens only if waiting for conflicting lock to go away leads to a deadlock). mysql-test/t/sp_sync.test: Adjusted test case after removing back-off in general case when conflicting metadata lock is encountered during open_tables() (now this happens only if waiting for conflicting lock to go away leads to a deadlock). sql/mdl.cc: Changed MDL subsystem to support new approach to acquring metadata locks in open tables and more fair and efficient scheduling of metadata locks. To implement this: - Made releasing of the lock and granting it to the most appropriate pending request atomic operation. As result it became possible to wake up only those waiters requests from which can be satisfied at the moment as well as wake-up only one waiter in case when granting its request would prevent all other requests from being satisfied. This solved thundering herd problem which occured in cases when we were releasing some lock and woke up many waiters for SNRW or X locks (this was the issue in Bug #52289 "performance regression for MyISAM in sysbench OLTP_RW test". To emphasize above changes wake_up_waiters() was renamed to MDL_context::reschedule_waiters(). - Changed code to add tickets for new requests to the back of waiters queue and to select tickets to be satisfied from the head of the queue if possible (this makes scheduling of requests with the same priority fair). To be able to do this efficiently we now use for waiting and granted queues version of I_P_List class which provides fast push_back() method. - Members and methods of MDL_context related to sending and waiting for signal were moved to separate MDL_wait class. - Since in order to avoid race conditions we must grant the lock only to the context which was not chosen as a victim of deadlock, killed or aborted due to timeout MDL_wait::set_status() (former awake()) was changed not to send signal if signal slot is already occupied and to indicate this fact through its return value. As another consequence MDL_wait::timed_wait() method was changed to handle timeout (optionally) and abort due to kill as signals which make signal slot occupied. - Renamed MDL_context::acquire_lock_impl() to acquire_lock(). Changed it to be able correctly process requests for shared locks when there are open HANDLERs, made this method more optimized for acquisition of shared locks. As part of this change moved code common between try_acquire_lock() and acquire_lock() to new try_acquire_lock_impl() method. Also adjusted acquire_lock()'s code to take into account the fact that in cases when lock is granted as result of MDL_context::reschedule_waiters() call (i.e. when it is granted after waiting for lock conflict to go away) updating MDL_lock state is responsibility of the thread calling reschedule_waiters(). - Changed MDL_context::find_deadlock() to send VICTIM signal even if victim is the context which has initiated deadlock detection. This is required in order to avoid races in cases when the same context simultaneously is chosen as a victim and its request for lock is satisfied. As result return value of this method became unnecessary and it was changed to return void. Adjusted MDL_lock::find_deadlock() method to take into account that now there can be a discrepancy between MDL_context::m_waiting_for value being set and real state of the ticket this member points to. - Renamed MDL_context::m_waiting_for_lock to m_LOCK_waiting_for and MDL_context::stop_waiting() to done_waiting_for(). - Finally, removed MDL_context::wait_for_lock() method. sql/mdl.h: Changed MDL subsystem to support new approach to acquring metadata locks in open tables and more fair and efficient scheduling of metadata locks. To implement this: - Members and methods of MDL_context related to sending and waiting for signal were moved to separate MDL_wait class. - Since now in order to avoid race conditions we must grant the lock only to the context which was not chosen as a victim of deadlock, killed or aborted due to timeout MDL_wait::set_status (former awake()) was changed not to send signal if signal slot is already occupied and to indicate this fact through its return value. Also NORMAL_WAKE_UP signal became GRANTED, and timeouts and aborts due to kill became full blown signals rather than simple return values. - MDL_wait::timed_wait() now takes extra parameter that indicates whether signal should be set if timeout is reached. - Enabled fast push_back() operation in MDL_context::m_tickets list to make move_ticket_after_trans_sentinel() method more efficient. - Removed MDL_context::wait_for_lock() method. - Renamed MDL_context::m_waiting_for_lock to m_LOCK_waiting_for and MDL_context::stop_waiting() to done_waiting_for(). - MDL_context::acquire_lock_impl() became acquire_lock(). - Introduced MDL_context::try_acquire_lock_impl() as a place for code shared by try_acquire_lock and acquire_lock(). - Due to fact that now VICTIM signal is sent even if victim is the context which has initiated deadlock detection find_deadlock() no longer needs a return value. sql/sql_base.cc: Implemented new approach to acquiring metadata locks in open_tables(). We no longer perform back-off when conflicting metadata lock is encountered. Instead we wait for this lock to go away while holding all locks which were acquired so far. Back-off is only used in situation when further waiting will cause a deadlock which could be avoided by performing back-off and restarting open_tables() process. Absence of waiting between back-off and restart of acquiring metadata locks can't lead to livelocks as MDL subsystem was changed to make release of lock and granting it to waiting lock an atomic action, so back-off will automatically give way to other participants of deadlock loop. Accordingly: - open_table_get_mdl_lock() and open_and_process_routine() were changed to wait for conflicting metadata lock to go away without back-off. Only if such wait leads to a deadlock back-off is requested. As part of this change new error handler class was introduced which converts, if possible, ER_LOCK_DEADLOCK error to a request for back-off and re-start of open_tables() process. - Open_table_context::recover_from_failed_open() was changed not to wait in case of metadata lock conflict. Instead we immediately proceed to re-acquiring locks. - Open_table_context::request_backoff_action() now always emits error if back-off is requested in the middle of transaction as we can't be sure that releasing lock which were acquired only by current statement will resolve a deadlock. Before this patch such situations were successfully detected thanks to the fact that we called MDL_context::wait_for_lock() method in recover_from_failed_open(). - In order to avoid deadlocks open_tables() code was adjusted to flush open HANDLERs for which there are pending requests for X locks before restarting the process of acquiring metadata locks. - Changed close_tables_for_reopen() not to reset MDL_request for tables belonging to the tail of prelocking list. It is no longer necessary as these MDL_request objects won't be used for any waiting. - Adjusted comment in tdc_wait_for_old_version() to avoid mentioning removed MDL_context::wait_for_lock() method. sql/sql_base.h: As we no longer wait for conflicting metadata lock away in Open_table_context::recover_from_failed_open() method, Open_table_context::OT_WAIT_MDL_LOCK action was renamed to OT_MDL_CONFLICT. Also Open_table_context::m_failed_mdl_request became unnecessary and was removed. sql/sql_plist.h: Extended I_P_List template to support efficient push_back() operation if it is parameterized with an appropriate policy class. sql/sql_show.cc: Adjusted code after removal of MDL_context::wait_for_lock() method. Now if one needs to acquire metadata lock with waiting one has to use a variant of MDL_context::acquire_lock() method.	16 years ago
Konstantin Osipov	e7854c86a7	A code review comment for Bug#52289. Encapsulate the deadlock detection functionality into a visitor class, and separate it from the wait-for graph traversal code. Use "Internal iterator" and "Visitor" patterns to achieve the desired separation of responsibilities. Add comments. sql/mdl.cc: Encapsulate deadlock detection into a class. sql/mdl.h: Adjust for a rename of a class.	16 years ago
Konstantin Osipov	8867c0a52c	Add comments to a few MDL deadlock-search related variables and methods. sql/mdl.cc: Add comments. sql/mdl.h: Add a comment.	16 years ago
Davi Arnaut	a8c288054e	Bug#53445: Build with -Wall and fix warnings that it generates Fix various mismatches between function's language linkage. Any particular function that is declared in C++ but should be callable from C must have C linkage. Note that function types with different linkages are also distinct. Thus, if a function type is declared in C code, it will have C linkage (same if declared in a extern "C" block). client/mysql.cc: Mismatch between prototype and declaration. client/mysqltest.cc: mysqltest used to be C code. Use C linkage where appropriate. cmd-line-utils/readline/input.c: Isolate unreachable code. include/my_alloc.h: Function type must have C linkage. include/my_base.h: Function type must have C linkage. include/my_global.h: Add helper macros to avoid spurious namespace indentation. include/mysql.h.pp: Update ABI file. mysys/my_gethwaddr.c: Remove stray carriage return and fix coding style. plugin/semisync/semisync_master_plugin.cc: Callback function types have C linkage. plugin/semisync/semisync_slave_plugin.cc: Callback function types have C linkage. sql/derror.cc: Expected function type has C linkage. sql/field.cc: Use helper macro and fix indentation. sql/handler.cc: Expected function type has C linkage. sql/item_sum.cc: Correct function linkages. Remove now unnecessary cast. sql/item_sum.h: Add prototypes with the appropriate linkage as otherwise they are distinct. sql/mysqld.cc: Wrap functions in C linkage mode. sql/opt_range.cc: C language linkage is ignored for class member functions. sql/partition_info.cc: Add wrapper functions with C linkage for class member functions. sql/rpl_utility.h: Use helper macro and fix indentation. sql/sql_class.cc: Change type of thd argument -- THD is a class. Use helper macro and fix indentation. sql/sql_class.h: Change type of thd argument -- THD is a class. sql/sql_select.cc: Expected function type has C linkage. sql/sql_select.h: Move prototype to sql_test.h sql/sql_show.cc: Expected function type has C linkage. sql/sql_test.cc: Fix required function prototype and fix coding style. sql/sql_test.h: Removed unnecessary export and add another. storage/myisammrg/ha_myisammrg.cc: Expected function type has C linkage. storage/perfschema/pfs.cc: PSI headers are declared with C language linkage, which also applies to function types.	16 years ago
Dmitry Lenev	bee0f214fd	Pre-requisite patch for bug #51263 "Deadlock between transactional SELECT and ALTER TABLE ... REBUILD PARTITION". The goal of this patch is to decouple type of metadata lock acquired for table by open_tables() from type of table-level lock to be acquired on it. To achieve this we change approach to how we determine what type of metadata lock should be acquired on table to be open. Now instead of inferring it at open_tables() time from flags and type of table-level lock we rely on that type of metadata lock is properly set at parsing time and is not changed further. sql/ha_ndbcluster.cc: Now one needs to properly initialize table list element's MDL_request object before calling mysql_rm_table_part2(). sql/lock.cc: lock_table_names() no longer initializes table list elements' MDL_request objects. Now proper initialization of these requests is a responsibility of the caller. sql/lock.h: Removed MYSQL_OPEN_TAKE_UPGRADABLE_MDL flag which became unnecessary. Thanks to the fact that we don't reset type of requests for metadata locks between re-executions we now can figure out that upgradable locks are requested by simply looking at their type which were set in the parser. As result this flag became redundant. sql/mdl.h: Added version of new operator which simplifies allocation of MDL_request objects on a MEM_ROOT. sql/sp_head.cc: Added comment explaining why it is OK to infer type of metadata lock to request from type of table-level lock for prelocking. Added enum_mdl_type argument to sp_add_to_query_tables() to simplify its usage in trigger implementation. sql/sp_head.h: Added enum_mdl_type argument to sp_add_to_query_tables() to simplify its usage in trigger implementation. sql/sql_base.cc: - open_table_get_mdl_lock(): Preserve type of MDL_request for table list element which was set in the parser by creating MDL_request objects on memory root if MYSQL_OPEN_FORCE_SHARED_MDL or MYSQL_OPEN_FORCE_SHARED_HIGH_PRIO_MDL flag were specified. Thanks to this and to the fact that we no longer reset type of requests for metadata locks between re-executions we no longer need to acquire exclusive metadata lock on table to be created in a special way. This lock is acquired by code handling acquiring of upgradable locks. Also changed signature/calling convention for this function to simplify its usage. - Accordingly special lock strategy for table list elements which was used for such locks became unnecessary and was removed. Other strategies were renamed. - Since we no longer have guarantee that MDL_request object which were not satisfied due to lock conflict belongs to table list element Open_table_context class and its methods were extended to remember pointer to MDL_request which has caused problem at request_backoff_action() time and use it in recover_from_failed_open(). Similar approach is used for cases when problem from which we need to recover is not related to MDL but to the table itself. In this case we store pointer to the element of table list. - Changed open_tables()/open_tables_check_upgradable_mdl()/ open_tables_acquire_upgradable_mdl() not to rely on MYSQL_OPEN_TAKE_UPGRADABLE_MDL flag to understand when upgradable metadata locks should be acquired and not to infer type of MDL lock from type of table-level lock. Instead we assume that type of MDL to be acquired was set in the parser (we can do this as type of MDL_request is no longer reset between re-executions). sql/sql_class.h: Since we no longer have guarantee that MDL_request object which were not satisfied due to lock conflict belongs to table list element Open_table_context class and its methods were extended to remember pointer to MDL_request which has caused problem at request_backoff_action() time and use it in recover_from_failed_open(). Similar approach is used for cases when problem from which we need to recover is not related to MDL but to the table itself. In this case we store pointer to the element of table list. sql/sql_db.cc: Now one needs to properly initialize table list element's MDL_request object before calling mysql_rm_table_part2() or mysql_rename_tables(). sql/sql_lex.cc: st_select_lex/st_select_lex_node::add_table_to_list() method now has argument which allows specify type of metadata lock to be requested for table list element being added. sql/sql_lex.h: - st_select_lex/st_select_lex_node::add_table_to_list() method now has argument which specifies type of metadata lock to be requested for table list element being added. This allows to explicitly set type of MDL lock to be acquired for a DDL statement in parser. It is also more future-proof than inferring type of MDL request from type of table-level lock. - Added Yacc_state::m_mdl_type member which specifies which type of metadata lock should be requested for tables to be added to table list by a grammar rule in cases when the same rule is used in several statements requiring different kinds of metadata locks. sql/sql_parse.cc: - st_select_lex::add_table_to_list() method now has argument which specifies type of metadata lock to be requested for table list element being added. This allows to explicitly set type of MDL lock to be acquired for a DDL statement in parser. It is also more future-proof than inferring type of MDL request from type of table-level lock. - EXCLUSIVE_DOWNGRADABLE_MDL lock strategy has a new name - OTLS_DOWNGRADE_IF_EXISTS. - Adjusted LOCK TABLES implementation to the fact that we no longer infer type of metadata lock to be acquired from table level lock and that type of MDL request is set at parsing. And thus MYSQL_OPEN_TAKE_UPGRADABLE_MDL flag became unnecessary. sql/sql_prepare.cc: TABLE_LIST's lock strategy SHARED_MDL was renamed to OTLS_NONE as now it means that metadata lock should not be changed during call to open_table() (if it has been already acquired) and is also used for exclusive metadata lock. sql/sql_show.cc: st_select_lex::add_table_to_list() method now has argument which specifies type of metadata lock to be requested for table list element being added. sql/sql_table.cc: - Adjusted mysql_admin_table()'s code to the fact that open_tables() no longer determines what kind of metadata lock should be obtained basing on type of table-level lock and flags. Instead type of metadata lock for table to be open should be set before calling open_tables(). - Changed mysql_alter_table() code to the facts: a) that now it is responsibility of caller to properly initalize MDL_request in table list elements before calling lock_table_names() b) and that MYSQL_OPEN_TAKE_UPGRADABLE_MDL is no longer necessary since type of metadata lock to be obtained at open_tables() time is set during parsing. - Changed code of mysql_recreate_table() to properly set type of metadata and table-level lock to be obtained by mysql_alter_table() which it calls. sql/sql_trigger.cc: Instead of relying on MYSQL_OPEN_TAKE_UPGRADABLE_MDL flag to force open_tables() to take an upgradable lock we now specify exact type of lock to be taken when constructing table list element for table to be open for CREATE/DROP TRIGGER. sql/sql_view.cc: We no longer use TABLE_LIST::EXCLUSIVE_MDL strategy to force open_tables() to take an exclusive metadata lock on view to be created. Instead we rely on parser setting proper type of metadata lock to request and open_tables() acquiring it. This became possible thanks to the fact that we no longer reset type of MDL_request between statement re-executions. sql/sql_yacc.yy: Instead of inferring type of MDL_request for table to be open from type of table-level lock and flags passed to open_tables() we now explicitly specify them at parsing. This became possible thanks to the fact that we no longer reset type of MDL_request between statement re-executions. In future this should allow to decouple type of metadata lock from type of table-level lock. The only exception to this approach is statements implemented through mysql_admin_table() which re-uses same table list element several times with different types of table-level and metadata locks. We now also properly initialize MDL_request objects for table list elements which are later passed to lock_table_names() function. sql/table.cc: Do not reset type of MDL_request between statement re-executions. This became unnecessesary as we no longer change type of MDL_request residing in table list element. In its turn this change allows to set type of MDL_request only once - at parsing time. sql/table.h: Got rid of TABLE_LIST::EXCLUSIVE_MDL lock strategy. Now we can specify that we need to acquire exclusive lock on table to be processed by open_tables() through setting an appropriate type of MDL_request at parsing time (this became possible thanks to the fact that we no longer reset types of MDL_request's belonging to table list elements between statement re-execution). Strategy SHARED_MDL was renamed to OTLS_NONE as now it means that metadata lock should not be changed during call to open_table() (if it has been already acquired) and is also used for exclusive metadata lock. Strategy EXCLUSIVE_DOWNGRADABLE_MDL was renamed to OTLS_DOWNGRADE_IF_EXISTS.	16 years ago
Joerg Bruehe	056e670542	Fix an issue with the IBM C++ compiler which was detected during the build of 5.5.3-m3. This requires version 9 of IBM C++ to help. More fixes will still be needed, it is very strict (or rather: a bit picky?) about templates.	16 years ago
Marc Alff	a7c9bf2ccf	Bug#51295 Build warnings in mdl.cc Before this fix, the performance schema instrumentation in mdl.h / mdl.cc was incomplete, causing: - build warnings, - no data collection for the performance schema This fix: - added instrumentation helpers for the new preferred reader read write lock, mysql_prlock_* - implemented completely the performance schema instrumentation of mdl.h / mdl.cc	16 years ago
Dmitry Lenev	6b5c4a9ef6	Fix for bug #51105 "MDL deadlock in rqg_mdl_stability test on Windows". On platforms where read-write lock implementation does not prefer readers by default (Windows, Solaris) server might have deadlocked while detecting MDL deadlock. MDL deadlock detector relies on the fact that read-write locks which are used in its implementation prefer readers (see new comment for MDL_lock::m_rwlock for details). So far MDL code assumed that default implementation of read/write locks for the system has this property. Indeed, this turned out ot be wrong, for example, for Windows or Solaris. Thus MDL deadlock detector might have deadlocked on these systems. This fix simply adds portable implementation of read/write lock which prefer readers and changes MDL code to use this new type of synchronization primitive. No test case is added as existing rqg_mdl_stability test can serve as one. config.h.cmake: Check for presence of pthread_rwlockattr_setkind_np to be able to determine if system natively supports read-write locks for which we can specify if readers or writers should be preferred. configure.cmake: Check for presence of pthread_rwlockattr_setkind_np to be able to determine if system natively supports read-write locks for which we can specify if readers or writers should be preferred. configure.in: Check for presence of pthread_rwlockattr_setkind_np to be able to determine if system natively supports read-write locks for which we can specify if readers or writers should be preferred. include/my_pthread.h: Added support for portable read-write locks which prefer readers. To do so extended existing my_rw_lock_t implementation to support selection of whom to prefer depending on a flag. mysys/thr_rwlock.c: Extended existing my_rw_lock_t implementation to support selection of whom to prefer depending on a flag. Added rw_pr_init() function implementing initialization of read-write locks preferring readers. sql/mdl.cc: Use portable read-write locks which prefer readers instead of relying on that system implementation of read-write locks has this property (this was true for Linux/NPTL but was false, for example, for Windows and Solaris). Added comment explaining why preferring readers is important for MDL deadlock detector (thanks to Serg for example!). sql/mdl.h: Use portable read-write locks which prefer readers instead of relying on that system implementation of read-write locks has this property (this was true for Linux/NPTL but was false, for example, for Windows and Solaris).	16 years ago
Dmitry Lenev	be3e256d25	Fix for bug #51136 "Crash in pthread_rwlock_rdlock on TEMPORARY + HANDLER + LOCK + SP". Server crashed when one: 1) Opened HANDLER or acquired global read lock 2) Then locked one or several temporary tables with LOCK TABLES statement (but no base tables). 3) Then issued any statement causing commit (explicit or implicit). 4) Issued statement which should have closed HANDLER or released global read lock. The problem was that when entering LOCK TABLES mode in the scenario described above we incorrectly set transactional MDL sentinel to zero. As result during commit all metadata locks were released (including lock for open HANDLER or global metadata shared lock). Indeed, attempt to release metadata lock for the second time which happened during HANLDER CLOSE or during release of GLR caused crash. This patch fixes problem by changing MDL_context's set_trans_sentinel() method to set sentinel to correct value (it should point to the most recent ticket). mysql-test/include/handler.inc: Added test for bug #51136 "Crash in pthread_rwlock_rdlock on TEMPORARY + HANDLER + LOCK + SP". mysql-test/r/flush.result: Updated test results (see flush.test). mysql-test/r/handler_innodb.result: Updated test results (see include/handler.inc). mysql-test/r/handler_myisam.result: Updated test results (see include/handler.inc). mysql-test/t/flush.test: Added additional coverage for bug #51136 "Crash in pthread_rwlock_rdlock on TEMPORARY + HANDLER + LOCK + SP". sql/mdl.h: When setting new value of transactional sentinel use pointer to the most recent ticket instead of value returned by MDL_context::mdl_savepoint(). This allows to handle correctly situation when the new value of sentinel should be the same as its current value (MDL_context::mdl_savepoint() returns NULL in this case).	16 years ago

1 2

84 Commits (ecd3ff9f24d8cf0c51ad564d9c8b335299dcdd9e)