mariadb

Tree: 7a22b6a655

Author	SHA1	Message	Date
unknown	7a22b6a655	MDEV-4506: Parallel replication. Fix uninitialised variable.	12 years ago
unknown	ee8a816208	MDEV-4506: Parallel replication. Implement --slave-parallel-max-queue to limit memory usage of SQL thread read-ahead in the relay log.	12 years ago
unknown	96a4f1f628	MDEV-4506: Parallel replication: Update some comments.	12 years ago
unknown	a09d2b105f	MDEV-4506: Parallel replication. Fix some more parts of old-style position updates. Now we save in rgi some coordinates for master log and relay log, so that in do_update_pos() we can use the right set of coordinates with the right events. The Rotate_log_event::do_update_pos() is fixed in the parallel case to not directly update relay-log.info (as Rotate event runs directly in the driver SQL thread, ahead of actual event execution). Instead, group_master_log_file is updated as part of do_update_pos() in each event execution. In the parallel case, position updates happen in parallel without any ordering, but taking care that position is not updated backwards. Since position update happens only after event execution this leads to the right result. Also fix an access-after-free introduced in an earlier commit.	12 years ago
unknown	7681c6aa78	MDEV-4506: Parallel replication: Intermediate commit. Fix some part of update of old-style coordinates in parallel replication: - Ignore XtraDB request for old-style coordinates, not meaningful for parallel replication (must use GTID to get crash-safe parallel slave). - Only update relay log coordinates forward, not backwards, to ensure that parallel threads do not conflict with each other. - Move future_event_relay_log_pos to rgi.	12 years ago
Michael Widenius	fcaf1e6a82	Flush the proc file after every modifications. This will avoid errors of type "Table './mysql/proc' is marked as crashed and should be repaired"	12 years ago
Michael Widenius	5748eb3ec6	Moved the remaining variables, that depends on sql execution, from Relay_log_info to rpl_group_info: -row_stmt_start_timestamp -last_event_start_time -long_find_row_note -trans_retries Added slave_executed_entries_lock to protect rli->executed_entries Added primitives for thread safe 64 bit increment Update rli->executed_entries when event has executed, not when event has been sent to sql execution thread sql/log_event.cc: row_stmt_start and long_find_row_note is now in rpl_group_info sql/mysqld.cc: Added slave_executed_entries_lock to protect rli->executed_entries sql/mysqld.h: Added slave_executed_entries_lock to protect rli->executed_entries Added primitives for thread safe 64 bit increment sql/rpl_parallel.cc: Update rli->executed_entries when event has executed, not when event has been sent to sql execution thread sql/rpl_rli.cc: Moved row_stmt_start_timestamp, last_event_start_time and long_find_row_note from Relay_log_info to rpl_group_info sql/rpl_rli.h: Moved trans_retries, row_stmt_start_timestamp, last_event_start_time and long_find_row_note from Relay_log_info to rpl_group_info sql/slave.cc: Use rgi for trans_retries and last_event_start_time Update rli->executed_entries when event has executed, not when event has been sent to sql execution thread Reset trans_retries when object is created	12 years ago
unknown	2842f6b5dc	MDEV-4506: Parallel replication: error handling. Add an error code to the wait_for_commit facility. Now, when a transaction fails, it can signal the error to any subsequent transaction that is waiting for it to commit. The waiting transactions then receive the error code back from wait_for_prior_commit() and can handle the error appropriately. Also fix one race that could cause crash if @@slave_parallel_threads were changed several times quickly in succession.	12 years ago
Michael Widenius	2e100cc5a4	Fixes for parallel slave: - Made slaves temporary table multi-thread slave safe by adding mutex around save_temporary_table usage. - rli->save_temporary_tables is the active list of all used temporary tables - This is copied to THD->temporary_tables when temporary tables are opened and updated when temporary tables are closed - Added THD->lock_temporary_tables() and THD->unlock_temporary_tables() to simplify this. - Relay_log_info->sql_thd renamed to Relay_log_info->sql_driver_thd to avoid wrong usage for merged code. - Added is_part_of_group() to mark functions that are part of the next function. This replaces setting IN_STMT when events are executed. - Added is_begin(), is_commit() and is_rollback() functions to Query_log_event to simplify code. - If slave_skip_counter is set run things in single threaded mode. This simplifies code for skipping events. - Updating state of relay log (IN_STMT and IN_TRANSACTION) is moved to one single function: update_state_of_relay_log() We can't use OPTION_BEGIN to check for the state anymore as the sql_driver and sql execution threads may be different. Clear IN_STMT and IN_TRANSACTION in init_relay_log_pos() and Relay_log_info::cleanup_context() to ensure the flags doesn't survive slave restarts is_in_group() is now independent of state of executed transaction. - Reset thd->transaction.all.modified_non_trans_table() if we did set it for single table row events. This was mainly for keeping the flag as documented. - Changed slave_open_temp_tables to uint32 to be able to use atomic operators on it. - Relay_log_info::sleep_lock -> rpl_group_info::sleep_lock - Relay_log_info::sleep_cond -> rpl_group_info::sleep_cond - Changed some functions to take rpl_group_info instead of Relay_log_info to make them multi-slave safe and to simplify usage - do_shall_skip() - continue_group() - sql_slave_killed() - next_event() - Simplifed arguments to io_salve_killed(), check_io_slave_killed() and sql_slave_killed(); No reason to supply THD as this is part of the given structure. - set_thd_in_use_temporary_tables() removed as in_use is set on usage - Added information to thd_proc_info() which thread is waiting for slave mutex to exit. - In open_table() reuse code from find_temporary_table() Other things: - More DBUG statements - Fixed the rpl_incident.test can be run with --debug - More comments - Disabled not used function rpl_connect_master() mysql-test/suite/perfschema/r/all_instances.result: Moved sleep_lock and sleep_cond to rpl_group_info mysql-test/suite/rpl/r/rpl_incident.result: Updated result mysql-test/suite/rpl/t/rpl_incident-master.opt: Not needed anymore mysql-test/suite/rpl/t/rpl_incident.test: Fixed that test can be run with --debug sql/handler.cc: More DBUG_PRINT sql/log.cc: More comments sql/log_event.cc: Added DBUG statements do_shall_skip(), continue_group() now takes rpl_group_info param Use is_begin(), is_commit() and is_rollback() functions instead of inspecting query string We don't have set slaves temporary tables 'in_use' as this is now done when tables are opened. Removed IN_STMT flag setting. This is now done in update_state_of_relay_log() Use IN_TRANSACTION flag to test state of relay log. In rows_event_stmt_cleanup() reset thd->transaction.all.modified_non_trans_table if we had set this before. sql/log_event.h: do_shall_skip(), continue_group() now takes rpl_group_info param Added is_part_of_group() to mark events that are part of the next event. This replaces setting IN_STMT when events are executed. Added is_begin(), is_commit() and is_rollback() functions to Query_log_event to simplify code. sql/log_event_old.cc: Removed IN_STMT flag setting. This is now done in update_state_of_relay_log() do_shall_skip(), continue_group() now takes rpl_group_info param sql/log_event_old.h: Added is_part_of_group() to mark events that are part of the next event. do_shall_skip(), continue_group() now takes rpl_group_info param sql/mysqld.cc: Changed slave_open_temp_tables to uint32 to be able to use atomic operators on it. Relay_log_info::sleep_lock -> Rpl_group_info::sleep_lock Relay_log_info::sleep_cond -> Rpl_group_info::sleep_cond sql/mysqld.h: Updated types and names sql/rpl_gtid.cc: More DBUG sql/rpl_parallel.cc: Updated TODO section Set thd for event that is execution Use new is_begin(), is_commit() and is_rollback() functions. More comments sql/rpl_rli.cc: sql_thd -> sql_driver_thd Relay_log_info::sleep_lock -> rpl_group_info::sleep_lock Relay_log_info::sleep_cond -> rpl_group_info::sleep_cond Clear IN_STMT and IN_TRANSACTION in init_relay_log_pos() and Relay_log_info::cleanup_context() to ensure the flags doesn't survive slave restarts. Reset table->in_use for temporary tables as the table may have been used by another THD. Use IN_TRANSACTION instead of OPTION_BEGIN to check state of relay log. Removed IN_STMT flag setting. This is now done in update_state_of_relay_log() sql/rpl_rli.h: Changed relay log state flags to bit masks instead of bit positions (most other code we have uses bit masks) Added IN_TRANSACTION to mark if we are in a BEGIN ... COMMIT section. save_temporary_tables is now thread safe Relay_log_info::sleep_lock -> rpl_group_info::sleep_lock Relay_log_info::sleep_cond -> rpl_group_info::sleep_cond Relay_log_info->sql_thd renamed to Relay_log_info->sql_driver_thd to avoid wrong usage for merged code is_in_group() is now independent of state of executed transaction. sql/slave.cc: Simplifed arguments to io_salve_killed(), sql_slave_killed() and check_io_slave_killed(); No reason to supply THD as this is part of the given structure. set_thd_in_use_temporary_tables() removed as in_use is set on usage in sql_base.cc sql_thd -> sql_driver_thd More DBUG Added update_state_of_relay_log() which will calculate the IN_STMT and IN_TRANSACTION state of the relay log after the current element is executed. If slave_skip_counter is set run things in single threaded mode. Simplifed arguments to io_salve_killed(), check_io_slave_killed() and sql_slave_killed(); No reason to supply THD as this is part of the given structure. Added information to thd_proc_info() which thread is waiting for slave mutex to exit. Disabled not used function rpl_connect_master() Updated argument to next_event() sql/sql_base.cc: Added mutex around usage of slave's temporary tables. The active list is always kept up to date in sql->rgi_slave->save_temporary_tables. Clear thd->temporary_tables after query (safety) More DBUG When using temporary table, set table->in_use to current thd as the THD may be different for slave threads. Some code is ifdef:ed with REMOVE_AFTER_MERGE_WITH_10 as the given code in 10.0 is not yet in this tree. In open_table() reuse code from find_temporary_table() sql/sql_binlog.cc: rli->sql_thd -> rli->sql_driver_thd Remove duplicate setting of rgi->rli sql/sql_class.cc: Added helper functions rgi_lock_temporary_tables() and rgi_unlock_temporary_tables() Would have been nicer to have these inline, but there was no easy way to do that sql/sql_class.h: Added functions to protect slaves temporary tables sql/sql_parse.cc: Added DBUG_PRINT sql/transaction.cc: Added comment	12 years ago
Michael Widenius	3784432256	Give a warning, not an error, if the log file size in innodb doesn't match what is on disk This helps when moving from MariaDB 5.5 to MariaDB 10.0 as sometimes the log file size is rounded differently. storage/innobase/srv/srv0start.c: Give a warning, not an error, if the log file size in innodb doesn't match what is on disk storage/xtradb/srv/srv0start.c: Give a warning, not an error, if the log file size in innodb doesn't match what is on disk	12 years ago
unknown	12c760ef71	MDEV-4506: Parallel replication. Improve STOP SLAVE in parallel mode. Now, the parallel part will queue the current event group to the end, and then stop queing any more events. Each worker will complete the current event group, and then just skip any further queued events.	12 years ago
unknown	45c3c71513	MDEV-4506: Parallel replication. .result file updates + a few comment updates.	12 years ago
unknown	976606d031	MDEV-4506: Parallel replication: After-review fixes.	12 years ago
unknown	5d1d20e409	MDEV-4506: parallel replication. Remove some unnecessary mutex locking.	12 years ago
unknown	c168c49d77	MDEV-4506: Parallel replication: Fix Windows compiler failure.	12 years ago
unknown	36df41ed95	MDEV-4506: Parallel replication. Add another test case.	12 years ago
unknown	39794dc72c	MDEV-4506: Parallel replication. Add another test case, using DEBUG_SYNC. Fix one bug found.	12 years ago
unknown	7781cdb79b	MDEV-4506: parallel replication. Add comments explaining tricky memory barrier semantics and suggestions for future changes.	12 years ago
unknown	5633dd8227	MDEV-4506: parallel replication. Add a simple test case. Fix bugs found.	12 years ago
unknown	d107bdaa01	MDEV-4506, parallel replication. Some after-review fixes.	12 years ago
unknown	13fddb32de	MDEV-4506: Parallel replication: Intermediate commit. Move annotate-event stuff from Relay_log_info to rpl_group_info, to make it thread safe.	13 years ago
unknown	47f8e0ef6e	MDEV-4506: Parallel replication: Intermediate commit Remove Relay_log_info::group_info. (It is not thread safe).	13 years ago
unknown	ba4b937af2	MDEV-4506: Parallel replication: Intermediate commit Move the deferred event stuff from Relay_log_info to rpl_group_info to make it thread safe for parallel replication.	13 years ago
unknown	6d5f237e09	MDEV-4506: Parallel replication: Intermediate commit. Fix a number of failures in the test suite.	13 years ago
unknown	a99356fbe7	MDEV-4506: Parallel replication: intermediate commit. Fix a bunch of issues found with locking, ordering, and non-thread-safe stuff in Relay_log_info. Now able to do a simple benchmark, showing 4.5 times speedup for applying a binlog with 10000 REPLACE statements.	13 years ago
unknown	e654be3865	MDEV-4506: Parallel replication: Intermediate commit. Impement options --binlog-commit-wait-count and --binlog-commit-wait-usec. These options permit the DBA to deliberately increase latency of an individual commit to get more transactions in each binlog group commit. This increases the opportunity for parallel replication on the slave, and can also decrease I/O load on the master. The options also make it easier to test the parallel replication with mysql-test-run.	13 years ago
unknown	b5a496a777	MDEV-4506: Parallel replication: Intermediate commit. Fix some bugs around waiting for worker threads to end during SQL slave stop. Free Log_event after parallel execution (still needs to be made thread-safe by using rpl_group_info rather than rli).	13 years ago
unknown	a1cfd47346	MDEV-4506: Parallel replication: Intermediate commit. Wait for all worker threads to finish when stopping the SQL thread. (Only a basic wait; this still needs to be fixed to include timeout logic as in sql_slave_killed()).	13 years ago
unknown	592e464a02	MDEV-4506: Parallel replication. Intermediate commit. Pass down rpl_group_info * to remove one instance of non-threadsafe use of rli->group_info.	13 years ago
unknown	31a5edb5c2	MDEV-4506: Parallel replication. Intermediate commit. Hook in the wait-for-prior-commit logic (not really tested yet). Clean up some resource maintenance around rpl_group_info (may still be some smaller issues there though). Add a ToDo list at the top of rpl_parallel.cc	13 years ago
unknown	1b3dc66e31	MDEV-4506: Parallel replication: Intermediate commit. First step of splitting out part of Relay_log_info, so that different event groups being applied in parallel can each use their own copy.	13 years ago
unknown	7e5dc4f074	MDEV-4506: Parallel replication. Intermediate commit. Implement facility for the commit in one thread to wait for the commit of another to complete first. The wait is done in a way that does not hinder that a waiter and a waitee can group commit together with a single fsync() in both binlog and InnoDB. The wait is done efficiently with respect to locking. The patch was originally made to support TaoBao parallel replication with in-order commit; now it will be adapted to also be used for parallel replication of group-committed transactions. A waiter THD registers itself with a prior waitee THD. The waiter will then complete its commit at the earliest in the same group commit of the waitee (when using binlog). The wait can also be done explicitly by the waitee.	13 years ago
unknown	535de71728	MDEV-4506: Parallel replication: intermediate commit. Fix typo in worker thread free list management. Simple parallel INSERT from worker threads runs now.	13 years ago
unknown	6d1e55f518	MDEV-4506: Parallel replication: Intermediate commit. A few fixes following tests. Now can apply one INSERT event in a separate worker thread.	13 years ago
unknown	26a9fbc416	MDEV-4506: Parallel replication of group-committed transactions: Intermediate commit First very rough sketch. We spawn and retire a pool of slave threads. Test main.alias works, most likely not much else does.	13 years ago
unknown	6a0a4f00a1	Forgotten .result file update.	13 years ago
unknown	b5fcf33d24	MDEV-4490: Old-style master position points at the last GTID event after slave restart Now whenever we reach the GTID point requested from the slave (when using GTID position to connect), we send a fake Gtid_list event. This event is used by the slave to know the current old-style position for MASTER_POS_WAIT(), and later the similar binlog position for MASTER_GTID_WAIT(). Without this fake event, if the slave is already fully up-to-date with the master, there may be no events sent at the given position for an indeterminate time.	13 years ago
unknown	03aa4876e1	MDEV-4486: Allow to start old-style replication even if mysql.rpl_slave_state is unavailable If the mysql.gtid_slave_pos table is not available, we cannot load nor update the current GTID position persistently. This can happen eg. after an upgrade, before mysql_upgrade_db is run, or if the table is InnoDB and the server is restarted without the InnoDB storage engine enabled. Before, replication always failed to start if the table was unavailable. With this patch, we try to continue with old-style replication, after suitable complaints in the error log. In strict mode, or if slave is configured to use GTID, slave still refuses to start.	13 years ago
unknown	dbe2c5060e	MDEV-4591:Setting gtid* values from inside a transaction might cause unexpected results Now we give an error on attempts to set @@SESSION.gtid_domain_id or @@SESSION.gtid_seq_no when a transaction is active.	13 years ago
unknown	7b6ab5638a	MDEV-4483: CHANGE MASTER TO master_use_gtid=xxx looses old-style coordinates. There was some old code that cleared the position in CHANGE MASTER, it was forgotten to be removed. In addition, add code that saves/restores the old-style position when we nuke the old relay logs as part of GTID slave start. Normally we will not use these, but it could be useful in case the GTID connect fails and user wants to go back to the old-style coordinates.	13 years ago
Sergei Golubchik	4749d40c63	5.5 merge	13 years ago
Vladislav Vaintroub	1ff1cb10fc	fix compile error	13 years ago
Michael Widenius	5cf5a9a1e8	Fixed timing failure in myisam-metadata.test mysql-test/include/wait_show_condition.inc: Print failing statement if timeout mysql-test/r/myisam-metadata.result: Updated DBUG_SYNC mysql-test/t/myisam-metadata.test: Updated DBUG_SYNC. Removed wait_show_condtion, as this is not needed when we use DBUG_SYNC This should fix timing issues with the test mysys/thr_mutex.c: Added comments sql/sql_acl.cc: atoi -> atoll() (Safety) storage/myisam/ha_myisam.cc: Send signal before mi_repair_by_sort.	13 years ago
unknown	64e53a0f81	Fix two small problems in previous push.	13 years ago
unknown	5cb486d159	MDEV-26: Global transaction ID. Fix problems related to reconnect. When we need to reconnect (ie. explict stop/start of just the IO thread by user, or automatic reconnect due to loosing network connection with the master), it is a bit complex to correctly resume at the right point without causing duplicate or missing events in the relay log. The previous code had multiple problems in this regard. With this patch, the problem is solved as follows. The IO thread keeps track (in memory) of which GTID was last queued to the relay log. If it needs to reconnect, it resumes at that GTID position. It also counts number of events received within the last, possibly partial, event group, and skips the same number of events after a reconnect, so that events already enqueued before the reconnect are not duplicated. (There is no need to keep any persistent state; whenever we restart slave threads after both of them being stopped (such as after server restart), we erase the relay logs and start over from the last GTID applied by SQL thread. But while the SQL thread is running, this patch is needed to get correct relay log).	13 years ago
unknown	7ad47ab0e0	MDEV-4605: Failing to load GTID slave position from rpl.gtid_slave_pos There were several cases where the slave GTID position was not loaded correctly before being used. This caused various failures such as corrupting the position at slave start and empty values of @@gtid_slave_pos and @@gtid_current_pos. Fixed by adding more checks for loaded position, and by always loading the position at server startup.	13 years ago
Vladislav Vaintroub	33ef993773	Fix a compile warning on NetBSD	13 years ago
Vladislav Vaintroub	689c1b44a9	MDEV-4607 : libreadline-related compilation problems on NetBSD. Problem : libreadline.so was already present on the machine, however the cmake check NEW_READLINE_INTERFACE was unsuccessfull indicating, thus bundled library had to be used instead of system library. The problem was that the value for HAVE_HIST_ENTRY cmake variable was cached with incorrect value (1 on NetBSD). The fix is to change HAVE_HIST_ENTRY to 0 with CACHE FORCE, after switching to bundled readline.	13 years ago
unknown	22b60fa95c	MDEV-4520: Assertion `0' fails in Query_cache::end_of_result on concurrent drop event and event executio Fix for embedded library, where thd->net.vio is not set which efficently switched off QC in emmbedded server for previous patch.	13 years ago
unknown	6feadb1082	MDEV-4485: Incorrect error handling in record_gtid(). Fix the error handling when access to the table mysql.gtid_slave_pos fails for whatever reason. Add some test cases.	13 years ago

1 2 3 4 5 ...

77630 Commits (7a22b6a6550ef6f798a0f5ff21d645ae43b3cbb2) All Branches Search

77630 Commits (7a22b6a6550ef6f798a0f5ff21d645ae43b3cbb2)

All Branches