mariadb

Tree: 445ad6bd37

Author	SHA1	Message	Date
unknown	22a9f9b243	Fix merge-big.test to properly restore debug settings. Otherwise following tests go crazy in the log, writing gigabytes of data and causing havoc.	16 years ago
Michael Widenius	67579fd290	Added missing fix from last commit	17 years ago
unknown	64d17dd1c2	Bug#34376 - merge-big test fails After changes to the bug fix for bug 26379 (Combination of FLUSH TABLE and REPAIR TABLE corrupts a MERGE table) the test case merge-big failed. Repaired the test case. Removed tests for INSERT ... SELECT, which is disabled for MERGE. Test case change only. mysql-test/r/merge-big.result: Bug#34376 - merge-big test fails Removed result for removed tests. mysql-test/t/merge-big.test: Bug#34376 - merge-big test fails Repaired the test case. Removed tests for INSERT ... SELECT, which is disabled for MERGE.	18 years ago
unknown	c8450b278d	Bug#26379 - Combination of FLUSH TABLE and REPAIR TABLE corrupts a MERGE table Bug 26867 - LOCK TABLES + REPAIR + merge table result in memory/cpu hogging Bug 26377 - Deadlock with MERGE and FLUSH TABLE Bug 25038 - Waiting TRUNCATE Bug 25700 - merge base tables get corrupted by optimize/analyze/repair table Bug 30275 - Merge tables: flush tables or unlock tables causes server to crash Bug 19627 - temporary merge table locking Bug 27660 - Falcon: merge table possible Bug 30273 - merge tables: Can't lock file (errno: 155) The problems were: Bug 26379 - Combination of FLUSH TABLE and REPAIR TABLE corrupts a MERGE table 1. A thread trying to lock a MERGE table performs busy waiting while REPAIR TABLE or a similar table administration task is ongoing on one or more of its MyISAM tables. 2. A thread trying to lock a MERGE table performs busy waiting until all threads that did REPAIR TABLE or similar table administration tasks on one or more of its MyISAM tables in LOCK TABLES segments do UNLOCK TABLES. The difference against problem #1 is that the busy waiting takes place after the administration task. It is terminated by UNLOCK TABLES only. 3. Two FLUSH TABLES within a LOCK TABLES segment can invalidate the lock. This does not require a MERGE table. The first FLUSH TABLES can be replaced by any statement that requires other threads to reopen the table. In 5.0 and 5.1 a single FLUSH TABLES can provoke the problem. Bug 26867 - LOCK TABLES + REPAIR + merge table result in memory/cpu hogging Trying DML on a MERGE table, which has a child locked and repaired by another thread, made an infinite loop in the server. Bug 26377 - Deadlock with MERGE and FLUSH TABLE Locking a MERGE table and its children in parent-child order and flushing the child deadlocked the server. Bug 25038 - Waiting TRUNCATE Truncating a MERGE child, while the MERGE table was in use, let the truncate fail instead of waiting for the table to become free. Bug 25700 - merge base tables get corrupted by optimize/analyze/repair table Repairing a child of an open MERGE table corrupted the child. It was necessary to FLUSH the child first. Bug 30275 - Merge tables: flush tables or unlock tables causes server to crash Flushing and optimizing locked MERGE children crashed the server. Bug 19627 - temporary merge table locking Use of a temporary MERGE table with non-temporary children could corrupt the children. Temporary tables are never locked. So we do now prohibit non-temporary chidlren of a temporary MERGE table. Bug 27660 - Falcon: merge table possible It was possible to create a MERGE table with non-MyISAM children. Bug 30273 - merge tables: Can't lock file (errno: 155) This was a Windows-only bug. Table administration statements sometimes failed with "Can't lock file (errno: 155)". These bugs are fixed by a new implementation of MERGE table open. When opening a MERGE table in open_tables() we do now add the child tables to the list of tables to be opened by open_tables() (the "query_list"). The children are not opened in the handler at this stage. After opening the parent, open_tables() opens each child from the now extended query_list. When the last child is opened, we remove the children from the query_list again and attach the children to the parent. This behaves similar to the old open. However it does not open the MyISAM tables directly, but grabs them from the already open children. When closing a MERGE table in close_thread_table() we detach the children only. Closing of the children is done implicitly because they are in thd->open_tables. For more detail see the comment at the top of ha_myisammrg.cc. Changed from open_ltable() to open_and_lock_tables() in all places that can be relevant for MERGE tables. The latter can handle tables added to the list on the fly. When open_ltable() was used in a loop over a list of tables, the list must be temporarily terminated after every table for open_and_lock_tables(). table_list->required_type is set to FRMTYPE_TABLE to avoid open of special tables. Handling of derived tables is suppressed. These details are handled by the new function open_n_lock_single_table(), which has nearly the same signature as open_ltable() and can replace it in most cases. In reopen_tables() some of the tables open by a thread can be closed and reopened. When a MERGE child is affected, the parent must be closed and reopened too. Closing of the parent is forced before the first child is closed. Reopen happens in the order of thd->open_tables. MERGE parents do not attach their children automatically at open. This is done after all tables are reopened. So all children are open when attaching them. Special lock handling like mysql_lock_abort() or mysql_lock_remove() needs to be suppressed for MERGE children or forwarded to the parent. This depends on the situation. In loops over all open tables one suppresses child lock handling. When a single table is touched, forwarding is done. Behavioral changes: =================== This patch changes the behavior of temporary MERGE tables. Temporary MERGE must have temporary children. The old behavior was wrong. A temporary table is not locked. Hence even non-temporary children were not locked. See Bug 19627 - temporary merge table locking. You cannot change the union list of a non-temporary MERGE table when LOCK TABLES is in effect. The following does not work: CREATE TABLE m1 ... ENGINE=MRG_MYISAM ...; LOCK TABLES t1 WRITE, t2 WRITE, m1 WRITE; ALTER TABLE m1 ... UNION=(t1,t2) ...; However, you can do this with a temporary MERGE table. You cannot create a MERGE table with CREATE ... SELECT, neither as a temporary MERGE table, nor as a non-temporary MERGE table. CREATE TABLE m1 ... ENGINE=MRG_MYISAM ... SELECT ...; Gives error message: table is not BASE TABLE. include/my_base.h: Bug#26379 - Combination of FLUSH TABLE and REPAIR TABLE corrupts a MERGE table Added HA_EXTRA_ATTACH_CHILDREN and HA_EXTRA_DETACH_CHILDREN. include/myisammrg.h: Bug#26379 - Combination of FLUSH TABLE and REPAIR TABLE corrupts a MERGE table Added element 'children_attached' to MYRG_INFO. Added declarations for myrg_parent_open(), myrg_attach_children() and myrg_detach_children() for the new MERGE table open approach. mysql-test/extra/binlog_tests/blackhole.test: Bug#26379 - Combination of FLUSH TABLE and REPAIR TABLE corrupts a MERGE table Preliminarily added new error message with a comment. mysql-test/r/create.result: Bug#26379 - Combination of FLUSH TABLE and REPAIR TABLE corrupts a MERGE table Fixed test result. mysql-test/r/delayed.result: Bug#26379 - Combination of FLUSH TABLE and REPAIR TABLE corrupts a MERGE table Moved test result from here to merge.result. mysql-test/r/merge.result: Bug#26379 - Combination of FLUSH TABLE and REPAIR TABLE corrupts a MERGE table Fixed/added test result. mysql-test/r/myisam.result: Bug#26379 - Combination of FLUSH TABLE and REPAIR TABLE corrupts a MERGE table Moved test result for bug 8306 from here to merge.result. mysql-test/suite/binlog/r/binlog_stm_blackhole.result: Bug#26379 - Combination of FLUSH TABLE and REPAIR TABLE corrupts a MERGE table Fixed test result. mysql-test/t/create.test: Bug#26379 - Combination of FLUSH TABLE and REPAIR TABLE corrupts a MERGE table Fixed error number. mysql-test/t/delayed.test: Bug#26379 - Combination of FLUSH TABLE and REPAIR TABLE corrupts a MERGE table Moved test from here to merge.test. mysql-test/t/merge.test: Bug#26379 - Combination of FLUSH TABLE and REPAIR TABLE corrupts a MERGE table Fixed test for new temporary MERGE table behavior. Exchanged error numbers by symbolic codes. Added tests. Included are tests for bugs 8306 (moved from myisam.test), 26379, 19627, 25038, 25700, 26377, 26867, 27660, 30275, and 30273. Fixed changes resulting from disabled CREATE...SELECT. Integrated tests moved from delayed.test and myisam.test to here. mysql-test/t/myisam.test: Bug#26379 - Combination of FLUSH TABLE and REPAIR TABLE corrupts a MERGE table Moved test for bug 8306 from here to merge.test. mysys/thr_lock.c: Bug#26379 - Combination of FLUSH TABLE and REPAIR TABLE corrupts a MERGE table Added code to let the owner of a high priority lock (TL_WRITE_ONLY) to bypass its own lock. sql/ha_ndbcluster_binlog.cc: Bug#26379 - Combination of FLUSH TABLE and REPAIR TABLE corrupts a MERGE table Added 'thd' argument to init_tmp_table_share(). sql/handler.cc: Bug#26379 - Combination of FLUSH TABLE and REPAIR TABLE corrupts a MERGE table Added 'thd' argument to init_tmp_table_share(). sql/mysql_priv.h: Bug#26379 - Combination of FLUSH TABLE and REPAIR TABLE corrupts a MERGE table Removed declaration of check_merge_table_access(). It is now static in sql_parse.cc. Added declaration for fix_merge_after_open(). Renamed open_and_lock_tables() to open_and_lock_tables_derived() with additional parameter 'derived'. Added inline functions simple_open_n_lock_tables() and open_and_lock_tables(), which call open_and_lock_tables_derived() and add the argument for 'derived'. Added new function open_n_lock_single_table(), which can be used as an replacement for open_ltable() in most situations. Internally it calls simple_open_n_lock_tables() so hat it is appropriate for MERGE tables. Added 'thd' argument to init_tmp_table_share(). sql/slave.cc: ug#26379 - Combination of FLUSH TABLE and REPAIR TABLE corrupts a MERGE table Added comment. sql/sql_base.cc: Bug#26379 - Combination of FLUSH TABLE and REPAIR TABLE corrupts a MERGE table Defined new functions add_merge_table_list(), attach_merge_children(), detach_merge_children(), and fix_merge_after_open() for the new MERGE table open approach. Added calls of the new functions to close_handle_and_leave_table_as_lock(), close_thread_tables(), close_thread_table(), unlink_open_table(), reopen_name_locked_table(), reopen_table(), drop_locked_tables(), close_temporary_table(), and open_tables() respectively. Prevented special lock handling of merge children (like mysql_lock_remove, mysql_lock_merge or mysql_lock_abort) at many places. Some of these calls are forwarded to the parent table instead. Added code to set thd->some_tables_deleted for every thread that has a table open that we are flushing. Added code for MERGE tables to unlink_open_table(). Added MERGE children to the list of unusable tables in open_table(). Added MERGE table handling to reopen_table(). Added lock handling and closing of a parent before the children in close_data_files_and_morph_locks(). Added code for re-attaching children in reopen_tables(). Added MYSQL_LOCK_NOTIFY_IF_NEED_REOPEN to the locking flags and error reporting after mysql_lock_tables() in reopen_tables(). Added lock handling and closing of a parent before the children in close_old_data_files(). Added lock handling and detaching in drop_locked_tables(). Added code for removing the children list from the statement list to prepare for a repetition in open_tables(). Added new function open_n_lock_single_table(), which can be used as an replacement for open_ltable() in most situations. Internally it calls simple_open_n_lock_tables() so hat it is appropriate for MERGE tables. Disabled use of open_ltable() for MERGE tables. Removed function simple_open_n_lock_tables(). It is now inline declared in mysql_priv.h. Renamed open_and_lock_tables() to open_and_lock_tables_derived() with additional parameter 'derived'. open_and_lock_tables() is now inline declared in mysql_priv.h. Added a check for end-of-list in two loops in lock_tables(). Added 'thd' argument to init_tmp_table_share(). sql/sql_insert.cc: Bug#26379 - Combination of FLUSH TABLE and REPAIR TABLE corrupts a MERGE table Changed from open_ltable() to open_n_lock_single_table() in handle_delayed_insert(). Reestablished LEX settings after lex initialization. Added 'thd' argument to init_tmp_table_share(). sql/sql_parse.cc: Bug#26379 - Combination of FLUSH TABLE and REPAIR TABLE corrupts a MERGE table Made check_merge_table_access() a static function. Disabled use of CREATE...SELECT for MERGE tables. sql/sql_partition.cc: Bug#26379 - Combination of FLUSH TABLE and REPAIR TABLE corrupts a MERGE table Fixed comment typo. sql/sql_select.cc: Bug#26379 - Combination of FLUSH TABLE and REPAIR TABLE corrupts a MERGE table Added 'thd' argument to init_tmp_table_share(). sql/sql_table.cc: Bug#26379 - Combination of FLUSH TABLE and REPAIR TABLE corrupts a MERGE table Optimized use of mysql_ha_flush() in mysql_rm_table_part2(). Disabled the use of MERGE tables with prepare_for_restore() and prepare_for_repair(). Changed from open_ltable() to open_n_lock_single_table() in mysql_alter_table() and mysql_checksum_table(). Disabled change of child list under LOCK TABLES. Initialized table_list->table in mysql_recreate_table(). sql/sql_trigger.cc: Bug#26379 - Combination of FLUSH TABLE and REPAIR TABLE corrupts a MERGE table Added code for allowing CREATE TRIGGER under LOCK TABLE, to be able to test it with MERGE tables. sql/table.cc: Bug#26379 - Combination of FLUSH TABLE and REPAIR TABLE corrupts a MERGE table Added 'thd' argument to init_tmp_table_share(). Setting table_map_id from query_id in init_tmp_table_share(). Added member function TABLE::is_children_attached(). sql/table.h: Bug#26379 - Combination of FLUSH TABLE and REPAIR TABLE corrupts a MERGE table Added access method get_table_def_version() to TABLE_SHARE. Added elements for MERGE tables to TABLE and TABLE_LIST. storage/myisam/ha_myisam.cc: Bug#26379 - Combination of FLUSH TABLE and REPAIR TABLE corrupts a MERGE table Added an unrelated comment to the function comment of table2myisam(). storage/myisam/ha_myisam.h: Bug#26379 - Combination of FLUSH TABLE and REPAIR TABLE corrupts a MERGE table Added new member function MI_INFO::file_ptr(). storage/myisammrg/ha_myisammrg.cc: Bug#26379 - Combination of FLUSH TABLE and REPAIR TABLE corrupts a MERGE table Added callback functions to support parent open and children attach of MERGE tables. Changed ha_myisammrg::open() to initialize storage engine structures and create a list of child tables only. Child tables are not opened. Added ha_myisammrg::attach_children(), which does now the main part of MERGE open. Added ha_myisammrg::detach_children(). Added calls to ::attach_children() and ::detach_children() to ::extra() on HA_EXTRA_ATTACH_CHILDREN and HA_EXTRA_DETACH_CHILDREN respectively. Added a check for matching TEMPORARY type for children against parent. Added a check for table def version. Added support for thd->open_options to attach_children(). Changed child path name generation for temporary tables so that it does nothing special for temporary tables. storage/myisammrg/ha_myisammrg.h: Bug#26379 - Combination of FLUSH TABLE and REPAIR TABLE corrupts a MERGE table Added elements to class ha_myisammrg to support the new open approach. Changed empty destructor definition to a declaration. Implemented in ha_myisammrg.cc. Added declaration for methods attach_children() and detach_children(). Added definition for method table_ptr() for use with callback functions. storage/myisammrg/myrg_close.c: Bug#26379 - Combination of FLUSH TABLE and REPAIR TABLE corrupts a MERGE table Added a check to avoid closing of MyISAM tables when the child tables are not attached. Added freeing of rec_per_key_part when the child tables are not attached. storage/myisammrg/myrg_extra.c: Bug#26379 - Combination of FLUSH TABLE and REPAIR TABLE corrupts a MERGE table Some ::extra() functions and ::reset() can be called when children are detached. storage/myisammrg/myrg_open.c: Bug#26379 - Combination of FLUSH TABLE and REPAIR TABLE corrupts a MERGE table Kept old myrg_open() for MERGE use independent from MySQL. Removed an always true condition in myrg_open(). Set children_attached for independent MERGE use in myrg_open(). Added myrg_parent_open(), myrg_attach_children(), and myrg_detach_children() for the new MERGE table open approach. mysql-test/r/merge-big.result: Bug#26379 - Combination of FLUSH TABLE and REPAIR TABLE corrupts a MERGE table New test result mysql-test/t/merge-big.test: Bug#26379 - Combination of FLUSH TABLE and REPAIR TABLE corrupts a MERGE table New test case	18 years ago

4 Commits (445ad6bd37495c3bc767e8f11ca767eb98bf4598)