mariadb

Tree: e9735a8185

Author	SHA1	Message	Date
Marko Mäkelä	e9735a8185	MDEV-25975 innodb_disallow_writes causes shutdown to hang We will remove the parameter innodb_disallow_writes because it is badly designed and implemented. The parameter was never allowed at startup. It was only internally used by Galera snapshot transfer. If a user executed SET GLOBAL innodb_disallow_writes=ON; the server could hang even on subsequent read operations. During Galera snapshot transfer, we will block writes to implement an rsync friendly snapshot, as follows: sst_flush_tables() will acquire a global lock by executing FLUSH TABLES WITH READ LOCK, which will block any writes at the high level. sst_disable_innodb_writes(), invoked via ha_disable_internal_writes(true), will suspend or disable InnoDB background tasks or threads that could initiate writes. As part of this, log_make_checkpoint() will be invoked to ensure that anything in the InnoDB buf_pool.flush_list will be written to the data files. This has the nice side effect that the Galera joiner will avoid crash recovery. The changes to sql/wsrep.cc and to the tests are based on a prototype that was developed by Jan Lindström. Reviewed by: Jan Lindström	4 years ago
Marko Mäkelä	ea2d44d01b	MDEV-18802 Assertion table->stat_initialized failed in dict_stats_update_if_needed() When a table has been evicted from dict_sys and reloaded internally by InnoDB for FOREIGN KEY processing, statistics may not be initialized, but nevertheless row_update_cascade_for_mysql() could invoke dict_stats_update_if_needed(). In that case, we cannot really update the statistics. For tables that have STATS_PERSISTENT=1 and STATS_AUTO_RECALC=1, ANALYZE TABLE might have to be executed later. dict_stats_update_if_needed(): Replace the assertion with a conditional early return.	5 years ago
Marko Mäkelä	581df0df89	MDEV-7962: Follow-up fix for 10.2 dict_stats_update_if_needed(): Replace the parameter THD* with const trx_t& so that trx_t::is_wsrep() can be invoked instead of the more expensive wsrep_on(). Replace also other occurrences of wsrep_on() with trx_t::is_wsrep().	6 years ago
Marko Mäkelä	0c7c61019d	Remove the wrappers ut_time(), ut_difftime(), ib_time_t	6 years ago
Vicențiu Ciorbaru	c0ac0b8860	Update FSF address	7 years ago
Jan Lindström	d0ebb155fe	MDEV-18577: Indexes problem on import dump SQL Problem was that we skipped background persistent statistics calculation on applier nodes if thread is marked as high priority (a.k.a BF). However, on applier nodes all DDL which is replicate will be executed as high priority i.e BF. Fixed by allowing background persistent statistics calculation on applier nodes even when thread is marked as BF. This could lead BF lock waits but for queries on that node needs that statistics.	7 years ago
Marko Mäkelä	ff88e4bb8a	Remove many redundant #include from InnoDB	7 years ago
Marko Mäkelä	769f6d2db7	Fix -Wclass-memaccess in WSREP,InnoDB,XtraDB	7 years ago
Marko Mäkelä	ba43914ec4	Replace dict_table_is_temporary(table) with table->is_temporary()	8 years ago
Marko Mäkelä	2b27ac8282	Fix many -Wunused-parameter Remove unused InnoDB function parameters and functions. i_s_sys_virtual_fill_table(): Do not allocate heap memory. mtr_is_block_fix(): Replace with mtr_memo_contains(). mtr_is_page_fix(): Replace with mtr_memo_contains_page().	8 years ago
Marko Mäkelä	715e4f4320	MDEV-12218 Clean up InnoDB parameter validation Bind more InnoDB parameters directly to MYSQL_SYSVAR and remove "shadow variables". innodb_change_buffering: Declare as ENUM, not STRING. innodb_flush_method: Declare as ENUM, not STRING. innodb_log_buffer_size: Bind directly to srv_log_buffer_size, without rounding it to a multiple of innodb_page_size. LOG_BUFFER_SIZE: Remove. SysTablespace::normalize_size(): Renamed from normalize(). innodb_init_params(): A new function to initialize and validate InnoDB startup parameters. innodb_init(): Renamed from innobase_init(). Invoke innodb_init_params() before actually trying to start up InnoDB. srv_start(bool): Renamed from innobase_start_or_create_for_mysql(). Added the input parameter create_new_db. SRV_ALL_O_DIRECT_FSYNC: Define only for _WIN32. xb_normalize_init_values(): Merge to innodb_init_param().	8 years ago
Marko Mäkelä	4f8555f1f6	MDEV-14941 Timeouts on persistent statistics tables caused by MDEV-14511 MDEV-14511 tried to avoid some consistency problems related to InnoDB persistent statistics. The persistent statistics are being written by an InnoDB internal SQL interpreter that requires the InnoDB data dictionary cache to be locked. Before MDEV-14511, the statistics were written during DDL in separate transactions, which could unnecessarily reduce performance (each commit would require a redo log flush) and break atomicity, because the statistics would be updated separately from the dictionary transaction. However, because it is unacceptable to hold the InnoDB data dictionary cache locked while suspending the execution for waiting for a transactional lock (in the mysql.innodb_index_stats or mysql.innodb_table_stats tables) to be released, any lock conflict was immediately be reported as "lock wait timeout". To fix MDEV-14941, an attempt to reduce these lock conflicts by acquiring transactional locks on the user tables in both the statistics and DDL operations was made, but it would still not entirely prevent lock conflicts on the mysql.innodb_index_stats and mysql.innodb_table_stats tables. Fixing the remaining problems would require a change that is too intrusive for a GA release series, such as MariaDB 10.2. Thefefore, we revert the change MDEV-14511. To silence the MDEV-13201 assertion, we use the pre-existing flag trx_t::internal.	8 years ago
Marko Mäkelä	40eee1da17	MDEV-14614 InnoDB: Failing assertion: trx->error_state == DB_SUCCESS or lock wait timeout upon saving statistics The assertion failure was caused by MDEV-14511 Use fewer transactions for updating InnoDB persistent statistics We are reusing a transaction object after commit, and sometimes, even after a successful operation, the trx_t::error_state may be something else than DB_SUCCESS. Reset the field when needed.	8 years ago
Marko Mäkelä	447931c6ab	Post-fix for MDEV-14587 dict_stats_process_entry_from_defrag_pool(): Release the mutex	8 years ago
Marko Mäkelä	77fb7ccba4	Follow-up fix to MDEV-13201 Assertion `srv_undo_sources \|\| ...` failed on shutdown during DDL operation Introduce the debug flag trx_t::persistent_stats to suppress the assertion for the updates of persistent statistics during fast shutdown. dict_stats_exec_sql(): Do execute the statement even though shutdown has been initiated.	8 years ago
Marko Mäkelä	7dc6066dea	MDEV-14511 Use fewer transactions for updating InnoDB persistent statistics dict_stats_exec_sql(): Expect the caller to always provide a transaction. Remove some redundant assertions. The caller must hold dict_sys->mutex, but holding dict_operation_lock is only necessary for accessing data dictionary tables, which we are not accessing. dict_stats_save_index_stat(): Acquire dict_sys->mutex for invoking dict_stats_exec_sql(). dict_stats_save(), dict_stats_update_for_index(), dict_stats_update(), dict_stats_drop_index(), dict_stats_delete_from_table_stats(), dict_stats_delete_from_index_stats(), dict_stats_drop_table(), dict_stats_rename_in_table_stats(), dict_stats_rename_in_index_stats(), dict_stats_rename_table(): Use a single caller-provided transaction that is started and committed or rolled back by the caller. dict_stats_process_entry_from_recalc_pool(): Let the caller provide a transaction object. ha_innobase::open(): Pass a transaction to dict_stats_init(). ha_innobase::create(), ha_innobase::discard_or_import_tablespace(): Pass a transaction to dict_stats_update(). ha_innobase::rename_table(): Pass a transaction to dict_stats_rename_table(). We do not use the same transaction as the one that updated the data dictionary tables, because we already released the dict_operation_lock. (FIXME: there is a race condition; a lock wait on SYS_* tables could occur in another DDL transaction until the data dictionary transaction is committed.) ha_innobase::info_low(): Pass a transaction to dict_stats_update() when calculating persistent statistics. alter_stats_norebuild(), alter_stats_rebuild(): Update the persistent statistics as well. In this way, a single transaction will be used for updating the statistics of a whole table, even for partitioned tables. ha_innobase::commit_inplace_alter_table(): Drop statistics for all partitions when adding or dropping virtual columns, so that the statistics will be recalculated on the next handler::open(). This is a refactored version of Oracle Bug#22469660 fix. RecLock::add_to_waitq(), lock_table_enqueue_waiting(): Do not allow a lock wait to occur for updating statistics in a data dictionary transaction, such as DROP TABLE. Instead, return the previously unused error code DB_QUE_THR_SUSPENDED. row_merge_lock_table(), row_mysql_lock_table(): Remove dead code for handling DB_QUE_THR_SUSPENDED. row_drop_table_for_mysql(), row_truncate_table_for_mysql(): Drop the statistics as part of the data dictionary transaction. After TRUNCATE TABLE, the statistics will be recalculated on subsequent ha_innobase::open(), similar to how the logic after the above-mentioned Oracle Bug#22469660 fix in ha_innobase::commit_inplace_alter_table() works. btr_defragment_thread(): Use a single transaction object for updating defragmentation statistics. dict_stats_save_defrag_stats(), dict_stats_save_defrag_stats(), dict_stats_process_entry_from_defrag_pool(), dict_defrag_process_entries_from_defrag_pool(), dict_stats_save_defrag_summary(), dict_stats_save_defrag_stats(): Add a parameter for the transaction. dict_stats_empty_table(): Make public. This will be called by row_truncate_table_for_mysql() after dropping persistent statistics, to clear the memory-based statistics as well.	8 years ago
Marko Mäkelä	63cbb98275	MDEV-14587 dict_stats_process_entry_from_defrag_pool() fails to call dict_table_close() when index==NULL dict_stats_process_entry_from_defrag_pool(): Simplify the logic, and always call dict_table_close() when dict_table_open() returned a non-NULL handle.	8 years ago
Marko Mäkelä	8be7548085	Follow-up to MDEV-12698: Adjust some comments The function dict_stats_update_if_needed() replaced row_update_statistics_if_needed(). Adjust the comments accordingly.	8 years ago
Vicențiu Ciorbaru	4ef64e01a7	5.6.38	8 years ago
Marko Mäkelä	417434f12d	MDEV-13039 innodb_fast_shutdown=0 may fail to purge all undo log When a slow shutdown is performed soon after spawning some work for background threads that can create or commit transactions, it is possible that new transactions are started or committed after the purge has finished. This is violating the specification of innodb_fast_shutdown=0, namely that the purge must be completed. (None of the history of the recent transactions would be purged.) Also, it is possible that the purge threads would exit in slow shutdown while there exist active transactions, such as recovered incomplete transactions that are being rolled back. Thus, the slow shutdown could fail to purge some undo log that becomes purgeable after the transaction commit or rollback. srv_undo_sources: A flag that indicates if undo log can be generated or the persistent, whether by background threads or by user SQL. Even when this flag is clear, active transactions that already exist in the system may be committed or rolled back. innodb_shutdown(): Renamed from innobase_shutdown_for_mysql(). Do not return an error code; the operation never fails. Clear the srv_undo_sources flag, and also ensure that the background DROP TABLE queue is empty. srv_purge_should_exit(): Do not allow the purge to exit if srv_undo_sources are active or the background DROP TABLE queue is not empty, or in slow shutdown, if any active transactions exist (and are being rolled back). srv_purge_coordinator_thread(): Remove some previous workarounds for this bug. innobase_start_or_create_for_mysql(): Set buf_page_cleaner_is_active and srv_dict_stats_thread_active directly. Set srv_undo_sources before starting the purge subsystem, to prevent immediate shutdown of the purge. Create dict_stats_thread and fts_optimize_thread immediately after setting srv_undo_sources, so that shutdown can use this flag to determine if these subsystems were started. dict_stats_shutdown(): Shut down dict_stats_thread. Backported from 10.2. srv_shutdown_table_bg_threads(): Remove (unused).	9 years ago
Marko Mäkelä	0bfa3dff8b	MDEV-12698 innodb.innodb_stats_del_mark test failure In my merge of the MySQL fix for Oracle Bug#23333990 / WL#9513 I overlooked some subsequent revisions to the test, and I also failed to notice that the test is actually always failing. Oracle introduced the parameter innodb_stats_include_delete_marked but failed to consistently take it into account in FOREIGN KEY constraints that involve CASCADE or SET NULL. When innodb_stats_include_delete_marked=ON, obviously the purge of delete-marked records should update the statistics as well. One more omission was that statistics were never updated on ROLLBACK. We are fixing that as well, properly taking into account the parameter innodb_stats_include_delete_marked. dict_stats_analyze_index_level(): Simplify an expression. (Using the ternary operator with a constant operand is unnecessary obfuscation.) page_scan_method_t: Revert the change done by Oracle. Instead, examine srv_stats_include_delete_marked directly where it is needed. dict_stats_update_if_needed(): Renamed from row_update_statistics_if_needed(). row_update_for_mysql_using_upd_graph(): Assert that the table statistics are initialized, as guaranteed by ha_innobase::open(). Update the statistics in a consistent way, both for FOREIGN KEY triggers and for the main table. If FOREIGN KEY constraints exist, do not dereference a freed pointer, but cache the proper value of node->is_delete so that it matches prebuilt->table. row_purge_record_func(): Update statistics if innodb_stats_include_delete_marked=ON. row_undo_ins(): Update statistics (on ROLLBACK of a fresh INSERT). This is independent of the parameter; the record is not delete-marked. row_undo_mod(): Update statistics on the ROLLBACK of updating key columns, or (if innodb_stats_include_delete_marked=OFF) updating delete-marks. innodb.innodb_stats_persistent: Renamed and extended from innodb.innodb_stats_del_mark. Reduced the unnecessarily large dataset from 262,144 to 32 rows. Test both values of the configuration parameter innodb_stats_include_delete_marked. Test that purge is updating the statistics. innodb_fts.innodb_fts_multiple_index: Adjust the result. The test is performing a ROLLBACK of an INSERT, which now affects the statistics. include/wait_all_purged.inc: Moved from innodb.innodb_truncate_debug to its own file.	9 years ago
Vicențiu Ciorbaru	0af9818240	5.6.36	9 years ago
Marko Mäkelä	849af74a48	MariaDB adjustments for Oracle Bug#23070734 fix Split the test case so that a server restart is not needed. Reduce the test cases and use a simpler mechanism for triggering and waiting for purge. fil_table_accessible(): Check if a table can be accessed without enjoying MDL protection.	9 years ago
Aditya A	62dca454e7	Bug #23070734 CONCURRENT TRUNCATE TABLES CAUSE STALLS PROBLEM When truncating single tablespace tables, we need to scan the entire buffer pool to remove the pages of the table from the buffer pool. During this scan and removal dict_sys->mutex is being held ,causing stalls in other DDL operations. FIX Release the dict_sys->mutex during the scan and reacquire it after the scan. Make sure that purge thread doesn't purge the records of the table being truncated and background stats collection thread skips the updation of stats for the table being truncated. [#rb 14564 Approved by Jimmy and satya ]	9 years ago
Marko Mäkelä	4e1116b2c6	MDEV-12271 Port MySQL 8.0 Bug#23150562 REMOVE UNIV_MUST_NOT_INLINE AND UNIV_NONINL Also, remove empty .ic files that were not removed by my MySQL commit. Problem: InnoDB used to support a compilation mode that allowed to choose whether the function definitions in .ic files are to be inlined or not. This stopped making sense when InnoDB moved to C++ in MySQL 5.6 (and ha_innodb.cc started to #include .ic files), and more so in MySQL 5.7 when inline methods and functions were introduced in .h files. Solution: Remove all references to UNIV_NONINL and UNIV_MUST_NOT_INLINE from all files, assuming that the symbols are never defined. Remove the files fut0fut.cc and ut0byte.cc which only mattered when UNIV_NONINL was defined.	10 years ago
Marko Mäkelä	a13a636c74	MDEV-11802 innodb.innodb_bug14676111 fails The function trx_purge_stop() was calling os_event_reset(purge_sys->event) before calling rw_lock_x_lock(&purge_sys->latch). The os_event_set() call in srv_purge_coordinator_suspend() is protected by that X-latch. It would seem a good idea to consistently protect both os_event_set() and os_event_reset() calls with a common mutex or rw-lock in those cases where os_event_set() and os_event_reset() are used like condition variables, tied to changes of shared state. For each os_event_t, we try to document the mutex or rw-lock that is being used. For some events, frequent calls to os_event_set() seem to try to avoid hangs. Some events are never waited for infinitely, only timed waits, and os_event_set() is used for early termination of these waits. os_aio_simulated_put_read_threads_to_sleep(): Define as a null macro on other systems than Windows. TODO: remove this altogether and disable innodb_use_native_aio on Windows. os_aio_segment_wait_events[]: Initialize only if innodb_use_native_aio=0. log_write_flush_to_disk_low(): Invoke log_mutex_enter() at the end, to avoid race conditions when changing the system state. (No potential race condition existed before MySQL 5.7.)	9 years ago
Marko Mäkelä	13493078e9	MDEV-11802 innodb.innodb_bug14676111 fails The function trx_purge_stop() was calling os_event_reset(purge_sys->event) before calling rw_lock_x_lock(&purge_sys->latch). The os_event_set() call in srv_purge_coordinator_suspend() is protected by that X-latch. It would seem a good idea to consistently protect both os_event_set() and os_event_reset() calls with a common mutex or rw-lock in those cases where os_event_set() and os_event_reset() are used like condition variables, tied to changes of shared state. For each os_event_t, we try to document the mutex or rw-lock that is being used. For some events, frequent calls to os_event_set() seem to try to avoid hangs. Some events are never waited for infinitely, only timed waits, and os_event_set() is used for early termination of these waits. os_aio_simulated_put_read_threads_to_sleep(): Define as a null macro on other systems than Windows. TODO: remove this altogether and disable innodb_use_native_aio on Windows. os_aio_segment_wait_events[]: Initialize only if innodb_use_native_aio=0.	9 years ago
Marko Mäkelä	719321e78e	MDEV-11638 Encryption causes race conditions in InnoDB shutdown InnoDB shutdown failed to properly take fil_crypt_thread() into account. The encryption threads were signalled to shut down together with other non-critical tasks. This could be much too early in case of slow shutdown, which could need minutes to complete the purge. Furthermore, InnoDB failed to wait for the fil_crypt_thread() to actually exit before proceeding to the final steps of shutdown, causing the race conditions. Furthermore, the log_scrub_thread() was shut down way too early. Also it should remain until the SRV_SHUTDOWN_FLUSH_PHASE. fil_crypt_threads_end(): Remove. This would cause the threads to be terminated way too early. srv_buf_dump_thread_active, srv_dict_stats_thread_active, lock_sys->timeout_thread_active, log_scrub_thread_active, srv_monitor_active, srv_error_monitor_active: Remove a race condition between startup and shutdown, by setting these in the startup thread that creates threads, not in each created thread. In this way, once the flag is cleared, it will remain cleared during shutdown. srv_n_fil_crypt_threads_started, fil_crypt_threads_event: Declare in global rather than static scope. log_scrub_event, srv_log_scrub_thread_active, log_scrub_thread(): Declare in static rather than global scope. Let these be created by log_init() and freed by log_shutdown(). rotate_thread_t::should_shutdown(): Do not shut down before the SRV_SHUTDOWN_FLUSH_PHASE. srv_any_background_threads_are_active(): Remove. These checks now exist in logs_empty_and_mark_files_at_shutdown(). logs_empty_and_mark_files_at_shutdown(): Shut down the threads in the proper order. Keep fil_crypt_thread() and log_scrub_thread() alive until SRV_SHUTDOWN_FLUSH_PHASE, and check that they actually terminate.	9 years ago
kevg	780db8e252	fix build and some warnings	9 years ago
Jan Lindström	fec844aca8	Merge InnoDB 5.7 from mysql-5.7.14. Contains also: MDEV-10549 mysqld: sql/handler.cc:2692: int handler::ha_index_first(uchar*): Assertion `table_share->tmp_table != NO_TMP_TABLE \|\| m_lock_type != 2' failed. (branch bb-10.2-jan) Unlike MySQL, InnoDB still uses THR_LOCK in MariaDB MDEV-10548 Some of the debug sync waits do not work with InnoDB 5.7 (branch bb-10.2-jan) enable tests that were fixed in MDEV-10549 MDEV-10548 Some of the debug sync waits do not work with InnoDB 5.7 (branch bb-10.2-jan) fix main.innodb_mysql_sync - re-enable online alter for partitioned innodb tables	9 years ago
Jan Lindström	2e814d4702	Merge InnoDB 5.7 from mysql-5.7.9. Contains also MDEV-10547: Test multi_update_innodb fails with InnoDB 5.7 The failure happened because 5.7 has changed the signature of the bool handler::primary_key_is_clustered() const virtual function ("const" was added). InnoDB was using the old signature which caused the function not to be used. MDEV-10550: Parallel replication lock waits/deadlock handling does not work with InnoDB 5.7 Fixed mutexing problem on lock_trx_handle_wait. Note that rpl_parallel and rpl_optimistic_parallel tests still fail. MDEV-10156 : Group commit tests fail on 10.2 InnoDB (branch bb-10.2-jan) Reason: incorrect merge MDEV-10550: Parallel replication can't sync with master in InnoDB 5.7 (branch bb-10.2-jan) Reason: incorrect merge	9 years ago
Sergei Golubchik	720e04ff67	5.6.31	10 years ago
Sergei Golubchik	6d06fbbd1d	move to storage/innobase	11 years ago
Sergei Golubchik	14a142fca6	move to storage/xtradb	11 years ago
Sergei Golubchik	0175860925	initial commit into innodb-5.6 mergetree	12 years ago
Sergei Golubchik	52c26f7a1f	Percona-Server-5.6.14-rel62.0.tar.gz	12 years ago

1 2

62 Commits (e9735a81859f172b73f75eccc043540a58d91cbc)