mariadb

Commit Graph

Author	SHA1	Message	Date
Marko Mäkelä	8c4df595b8	MDEV-13807 mariabackup --apply-log-only does generate redo log by performing rollback and possibly other tasks Skip rollback and other redo-log-generating tasks if srv_apply_log_only is set. Instead of assigning the debug variable recv_no_log_write = FALSE, assign it to srv_apply_log_only, so that any unwanted writes are caught.	8 years ago
Marko Mäkelä	38ca9be4de	MDEV-13684 InnoDB race condition between fil_crypt_thread and btr_scrub_init There is a race condition in InnoDB startup. A number of fil_crypt_thread are created by fil_crypt_threads_init(). These threads may call btr_scrub_complete_space() before btr_scrub_init() was called. Those too early calls would be accessing an uninitialized scrub_stat_mutex. innobase_start_or_create_for_mysql(): Invoke btr_scrub_init() before fil_crypt_threads_init(). fil_crypt_complete_rotate_space(): Only invoke btr_scrub_complete_space() if scrubbing is enabled. There is no need to update the statistics if it is not enabled.	8 years ago
Jan Lindström	b29f26d774	Fix test failures on embedded server. Problem was incorrect definition of wsrep_recovery, trx_sys_update_wsrep_checkpoint and trx_sys_read_wsrep_checkpoint functions causing innodb_plugin not to load as there was undefined symbols.	8 years ago
Jan Lindström	352d27ce36	MDEV-13557: Startup failure, unable to decrypt ibdata1 Fixes also MDEV-13488: InnoDB writes CRYPT_INFO even though encryption is not enabled. Problem was that we created encryption metadata (crypt_data) for system tablespace even when no encryption was enabled and too early. System tablespace can be encrypted only using key rotation. Test innodb-key-rotation-disable, innodb_encryption, innodb_lotoftables require adjustment because INFORMATION_SCHEMA INNODB_TABLESPACES_ENCRYPTION contain row only if tablespace really has encryption metadata. fil_crypt_set_thread_cnt: Send message to background encryption threads if they exits when they are ready. This is required to find tablespaces requiring key rotation if no other changes happen. fil_crypt_find_space_to_rotate: Decrease the amount of time waiting when nothing happens to better enable key rotation on startup. fsp_header_init: Write encryption metadata to page 0 only if tablespace is encrypted or encryption is disabled by table option. i_s_dict_fill_tablespaces_encryption : Skip tablespaces that do not contain encryption metadata. This is required to avoid too early wait condition trigger in encrypted -> unencrypted state transfer. open_or_create_data_files: Do not create encryption metadata by default to system tablespace.	8 years ago
Jan Lindström	61096ff214	MDEV-13591: InnoDB: Database page corruption on disk or a failed file read and assertion failure Problem is that page 0 and its possible enrryption information is not read for undo tablespaces. fil_crypt_get_latest_key_version(): Do not send event to encryption threads if event does not yet exists. Seen on regression testing. fil_read_first_page: Add new parameter does page belong to undo tablespace and if it does, we do not read FSP_HEADER. srv_undo_tablespace_open : Read first page of the tablespace to get crypt_data if it exists and pass it to fil_space_create. Tested using innodb_encryption with combinations with innodb-undo-tablespaces.	8 years ago
Marko Mäkelä	06106c0148	MDEV-12988 backup fails if innodb_undo_tablespaces>0 srv_undo_tablespaces_init(): In Mariabackup backup mode, do initialize the array of undo_tablespace_ids[].	8 years ago
Marko Mäkelä	4325041df6	MDEV-13057 innodb_read_only=1 should avoid creating buf_flush_page_cleaner_thread When the server is started in innodb_read_only mode, there cannot be any writes to persistent InnoDB/XtraDB files. Just like the creation of buf_flush_page_cleaner_thread is skipped in this case, also the creation of the XtraDB-specific buf_flush_lru_manager_thread should be skipped.	9 years ago
Marko Mäkelä	417434f12d	MDEV-13039 innodb_fast_shutdown=0 may fail to purge all undo log When a slow shutdown is performed soon after spawning some work for background threads that can create or commit transactions, it is possible that new transactions are started or committed after the purge has finished. This is violating the specification of innodb_fast_shutdown=0, namely that the purge must be completed. (None of the history of the recent transactions would be purged.) Also, it is possible that the purge threads would exit in slow shutdown while there exist active transactions, such as recovered incomplete transactions that are being rolled back. Thus, the slow shutdown could fail to purge some undo log that becomes purgeable after the transaction commit or rollback. srv_undo_sources: A flag that indicates if undo log can be generated or the persistent, whether by background threads or by user SQL. Even when this flag is clear, active transactions that already exist in the system may be committed or rolled back. innodb_shutdown(): Renamed from innobase_shutdown_for_mysql(). Do not return an error code; the operation never fails. Clear the srv_undo_sources flag, and also ensure that the background DROP TABLE queue is empty. srv_purge_should_exit(): Do not allow the purge to exit if srv_undo_sources are active or the background DROP TABLE queue is not empty, or in slow shutdown, if any active transactions exist (and are being rolled back). srv_purge_coordinator_thread(): Remove some previous workarounds for this bug. innobase_start_or_create_for_mysql(): Set buf_page_cleaner_is_active and srv_dict_stats_thread_active directly. Set srv_undo_sources before starting the purge subsystem, to prevent immediate shutdown of the purge. Create dict_stats_thread and fts_optimize_thread immediately after setting srv_undo_sources, so that shutdown can use this flag to determine if these subsystems were started. dict_stats_shutdown(): Shut down dict_stats_thread. Backported from 10.2. srv_shutdown_table_bg_threads(): Remove (unused).	9 years ago
Marko Mäkelä	fbeb9489cd	Cleanup of MDEV-12600: crash during install_db with innodb_page_size=32K and ibdata1=3M The doublewrite buffer pages must fit in the first InnoDB system tablespace data file. The checks that were added in the initial patch (commit `112b21da37`) were at too high level and did not cover all cases. innodb.log_data_file_size: Test all innodb_page_size combinations. fsp_header_init(): Never return an error. Move the change buffer creation to the only caller that needs to do it. btr_create(): Clean up the logic. Remove the error log messages. buf_dblwr_create(): Try to return an error on non-fatal failure. Check that the first data file is big enough for creating the doublewrite buffers. buf_dblwr_process(): Check if the doublewrite buffer is available. Display the message only if it is available. recv_recovery_from_checkpoint_start_func(): Remove a redundant message about FIL_PAGE_FILE_FLUSH_LSN mismatch when crash recovery has already been initiated. fil_report_invalid_page_access(): Simplify the message. fseg_create_general(): Do not emit messages to the error log. innobase_init(): Revert the changes. trx_rseg_create(): Refactor (no functional change).	9 years ago
Marko Mäkelä	68890fe7d4	Revert part of MDEV-12113 commit `1af8bf39ca` added unnecessary calls to fil_write_flushed_lsn() during redo log resizing at InnoDB server startup. Because fil_write_flushed_lsn() is neither redo-logged nor doublewrite buffered, the call is risky and should be avoided, because if the server killed during the write call, the whole InnoDB instance can become inaccessible (corrupted page 0 in the system tablespace). In the best case, this call might prevent a diagnostic message from being emitted to the error log on the next startup.	9 years ago
Jan Lindström	112b21da37	MDEV-12600: crash during install_db with innodb_page_size=32K and ibdata1=3M; Problem was that all doublewrite buffer pages must fit to first system datafile. Ported commit `27a34df788` Author: Shaohua Wang <shaohua.wang@oracle.com> Date: Wed Aug 12 15:55:19 2015 +0800 BUG#21551464 - SEGFAULT WHILE INITIALIZING DATABASE WHEN INNODB_DATA_FILE SIZE IS SMALL To 10.1 (with extended error printout). btr_create(): If ibuf header page allocation fails report error and return FIL_NULL. Similarly if root page allocation fails return a error. dict_build_table_def_step: If fsp_header_init fails return error code. fsp_header_init: returns true if header initialization succeeds and false if not. fseg_create_general: report error if segment or page allocation fails. innobase_init: If first datafile is smaller than 3M and could not contain all doublewrite buffer pages report error and fail to initialize InnoDB plugin. row_truncate_table_for_mysql: report error if fsp header init fails. srv_init_abort: New function to report database initialization errors. srv_undo_tablespaces_init, innobase_start_or_create_for_mysql: If database initialization fails report error and abort. trx_rseg_create: If segment header creation fails return.	9 years ago
Jan Lindström	1af8bf39ca	MDEV-12113: install_db shows corruption for rest encryption with innodb_data_file_path=ibdata1:3M; Problem was that FIL_PAGE_FLUSH_LSN_OR_KEY_VERSION field that for encrypted pages even in system datafiles should contain key_version except very first page (0:0) is after encryption overwritten with flush lsn. Ported WL#7990 Repurpose FIL_PAGE_FLUSH_LSN to 10.1 The field FIL_PAGE_FLUSH_LSN_OR_KEY_VERSION is consulted during InnoDB startup. At startup, InnoDB reads the FIL_PAGE_FLUSH_LSN_OR_KEY_VERSION from the first page of each file in the InnoDB system tablespace. If there are multiple files, the minimum and maximum LSN can differ. These numbers are passed to InnoDB startup. Having the number in other files than the first file of the InnoDB system tablespace is not providing much additional value. It is conflicting with other use of the field, such as on InnoDB R-tree index pages and encryption key_version. This worklog will stop writing FIL_PAGE_FLUSH_LSN_OR_KEY_VERSION to other files than the first file of the InnoDB system tablespace (page number 0:0) when system tablespace is encrypted. If tablespace is not encrypted we continue writing FIL_PAGE_FLUSH_LSN_OR_KEY_VERSION to all first pages of system tablespace to avoid unnecessary warnings on downgrade. open_or_create_data_files(): pass only one flushed_lsn parameter xb_load_tablespaces(): pass only one flushed_lsn parameter. buf_page_create(): Improve comment about where FIL_PAGE_FIL_FLUSH_LSN_OR_KEY_VERSION is set. fil_write_flushed_lsn(): A new function, merged from fil_write_lsn_and_arch_no_to_file() and fil_write_flushed_lsn_to_data_files(). Only write to the first page of the system tablespace (page 0:0) if tablespace is encrypted, or write all first pages of system tablespace and invoke fil_flush_file_spaces(FIL_TYPE_TABLESPACE) afterwards. fil_read_first_page(): read flush_lsn and crypt_data only from first datafile. fil_open_single_table_tablespace(): Remove output of LSN, because it was only valid for the system tablespace and the undo tablespaces, not user tablespaces. fil_validate_single_table_tablespace(): Remove output of LSN. checkpoint_now_set(): Use fil_write_flushed_lsn and output a error if operation fails. Remove lsn variable from fsp_open_info. recv_recovery_from_checkpoint_start(): Remove unnecessary second flush_lsn parameter. log_empty_and_mark_files_at_shutdown(): Use fil_writte_flushed_lsn and output error if it fails. open_or_create_data_files(): Pass only one flushed_lsn variable.	9 years ago
Marko Mäkelä	2f29fc3c1c	10.1 additions for MDEV-12052 Shutdown crash presumably due to master thread activity btr_defragment_thread(): Create the thread in the same place as other threads. Do not invoke btr_defragment_shutdown(), because row_drop_tables_for_mysql_in_background() in the master thread can still keep invoking btr_defragment_remove_table(). logs_empty_and_mark_files_at_shutdown(): Wait for btr_defragment_thread() to exit. innobase_start_or_create_for_mysql(), innobase_shutdown_for_mysql(): Skip encryption and scrubbing in innodb_read_only_mode. srv_export_innodb_status(): Do not export encryption or scrubbing statistics in innodb_read_only mode, because the threads will not be running.	9 years ago
Vicențiu Ciorbaru	360a4a0372	5.6.36-82.0	9 years ago
Vladislav Vaintroub	9c4b7cad27	MDEV-9566 Prepare xtradb for xtrabackup These changes are comparable to Percona's modifications in innodb in the Percona Xtrabackup repository. - If functions are used in backup as well as in innodb, make them non-static. - Define IS_XTRABACKUP() macro for special handling of innodb running inside backup. - Extend some functions for backup. fil_space_for_table_exists_in_mem() gets additional parameter 'remove_from_data_dict_if_does_not_exist', for partial backups fil_load_single_table_tablespaces() gets an optional parameter predicate which tells whether to load tablespace based on database or table name, also for partial backups. srv_undo_tablespaces_init() gets an optional parameter 'backup_mode' - Allow single redo log file (for backup "prepare") - Do not read doublewrite buffer pages in backup, they are outdated - Add function fil_remove_invalid_table_from_data_dict(), to remove non-existing tables from data dictionary in case of partial backups. - On Windows, fix file share modes when opening tablespaces, to allow mariabackup to read tablespaces while server is online. - Avoid access to THDVARs in backup, because innodb plugin is not loaded, and THDVAR would crash in this case.	9 years ago
Jan Lindström	765a43605a	MDEV-12253: Buffer pool blocks are accessed after they have been freed Problem was that bpage was referenced after it was already freed from LRU. Fixed by adding a new variable encrypted that is passed down to buf_page_check_corrupt() and used in buf_page_get_gen() to stop processing page read. This patch should also address following test failures and bugs: MDEV-12419: IMPORT should not look up tablespace in PageConverter::validate(). This is now removed. MDEV-10099: encryption.innodb_onlinealter_encryption fails sporadically in buildbot MDEV-11420: encryption.innodb_encryption-page-compression failed in buildbot MDEV-11222: encryption.encrypt_and_grep failed in buildbot on P8 Removed dict_table_t::is_encrypted and dict_table_t::ibd_file_missing and replaced these with dict_table_t::file_unreadable. Table ibd file is missing if fil_get_space(space_id) returns NULL and encrypted if not. Removed dict_table_t::is_corrupted field. Ported FilSpace class from 10.2 and using that on buf_page_check_corrupt(), buf_page_decrypt_after_read(), buf_page_encrypt_before_write(), buf_dblwr_process(), buf_read_page(), dict_stats_save_defrag_stats(). Added test cases when enrypted page could be read while doing redo log crash recovery. Also added test case for row compressed blobs. btr_cur_open_at_index_side_func(), btr_cur_open_at_rnd_pos_func(): Avoid referencing block that is NULL. buf_page_get_zip(): Issue error if page read fails. buf_page_get_gen(): Use dberr_t for error detection and do not reference bpage after we hare freed it. buf_mark_space_corrupt(): remove bpage from LRU also when it is encrypted. buf_page_check_corrupt(): @return DB_SUCCESS if page has been read and is not corrupted, DB_PAGE_CORRUPTED if page based on checksum check is corrupted, DB_DECRYPTION_FAILED if page post encryption checksum matches but after decryption normal page checksum does not match. In read case only DB_SUCCESS is possible. buf_page_io_complete(): use dberr_t for error handling. buf_flush_write_block_low(), buf_read_ahead_random(), buf_read_page_async(), buf_read_ahead_linear(), buf_read_ibuf_merge_pages(), buf_read_recv_pages(), fil_aio_wait(): Issue error if page read fails. btr_pcur_move_to_next_page(): Do not reference page if it is NULL. Introduced dict_table_t::is_readable() and dict_index_t::is_readable() that will return true if tablespace exists and pages read from tablespace are not corrupted or page decryption failed. Removed buf_page_t::key_version. After page decryption the key version is not removed from page frame. For unencrypted pages, old key_version is removed at buf_page_encrypt_before_write() dict_stats_update_transient_for_index(), dict_stats_update_transient() Do not continue if table decryption failed or table is corrupted. dict0stats.cc: Introduced a dict_stats_report_error function to avoid code duplication. fil_parse_write_crypt_data(): Check that key read from redo log entry is found from encryption plugin and if it is not, refuse to start. PageConverter::validate(): Removed access to fil_space_t as tablespace is not available during import. Fixed error code on innodb.innodb test. Merged test cased innodb-bad-key-change5 and innodb-bad-key-shutdown to innodb-bad-key-change2. Removed innodb-bad-key-change5 test. Decreased unnecessary complexity on some long lasting tests. Removed fil_inc_pending_ops(), fil_decr_pending_ops(), fil_get_first_space(), fil_get_next_space(), fil_get_first_space_safe(), fil_get_next_space_safe() functions. fil_space_verify_crypt_checksum(): Fixed bug found using ASAN where FIL_PAGE_END_LSN_OLD_CHECKSUM field was incorrectly accessed from row compressed tables. Fixed out of page frame bug for row compressed tables in fil_space_verify_crypt_checksum() found using ASAN. Incorrect function was called for compressed table. Added new tests for discard, rename table and drop (we should allow them even when page decryption fails). Alter table rename is not allowed. Added test for restart with innodb-force-recovery=1 when page read on redo-recovery cant be decrypted. Added test for corrupted table where both page data and FIL_PAGE_FILE_FLUSH_LSN_OR_KEY_VERSION is corrupted. Adjusted the test case innodb_bug14147491 so that it does not anymore expect crash. Instead table is just mostly not usable. fil0fil.h: fil_space_acquire_low is not visible function and fil_space_acquire and fil_space_acquire_silent are inline functions. FilSpace class uses fil_space_acquire_low directly. recv_apply_hashed_log_recs() does not return anything.	9 years ago
Jan Lindström	50eb40a2a8	MDEV-11738: Mariadb uses 100% of several of my 8 cpus doing nothing MDEV-11581: Mariadb starts InnoDB encryption threads when key has not changed or data scrubbing turned off Background: Key rotation is based on background threads (innodb-encryption-threads) periodically going through all tablespaces on fil_system. For each tablespace current used key version is compared to max key age (innodb-encryption-rotate-key-age). This process naturally takes CPU. Similarly, in same time need for scrubbing is investigated. Currently, key rotation is fully supported on Amazon AWS key management plugin only but InnoDB does not have knowledge what key management plugin is used. This patch re-purposes innodb-encryption-rotate-key-age=0 to disable key rotation and background data scrubbing. All new tables are added to special list for key rotation and key rotation is based on sending a event to background encryption threads instead of using periodic checking (i.e. timeout). fil0fil.cc: Added functions fil_space_acquire_low() to acquire a tablespace when it could be dropped concurrently. This function is used from fil_space_acquire() or fil_space_acquire_silent() that will not print any messages if we try to acquire space that does not exist. fil_space_release() to release a acquired tablespace. fil_space_next() to iterate tablespaces in fil_system using fil_space_acquire() and fil_space_release(). Similarly, fil_space_keyrotation_next() to iterate new list fil_system->rotation_list where new tables. are added if key rotation is disabled. Removed unnecessary functions fil_get_first_space_safe() fil_get_next_space_safe() fil_node_open_file(): After page 0 is read read also crypt_info if it is not yet read. btr_scrub_lock_dict_func() buf_page_check_corrupt() buf_page_encrypt_before_write() buf_merge_or_delete_for_page() lock_print_info_all_transactions() row_fts_psort_info_init() row_truncate_table_for_mysql() row_drop_table_for_mysql() Use fil_space_acquire()/release() to access fil_space_t. buf_page_decrypt_after_read(): Use fil_space_get_crypt_data() because at this point we might not yet have read page 0. fil0crypt.cc/fil0fil.h: Lot of changes. Pass fil_space_t* directly to functions needing it and store fil_space_t* to rotation state. Use fil_space_acquire()/release() when iterating tablespaces and removed unnecessary is_closing from fil_crypt_t. Use fil_space_t::is_stopping() to detect when access to tablespace should be stopped. Removed unnecessary fil_space_get_crypt_data(). fil_space_create(): Inform key rotation that there could be something to do if key rotation is disabled and new table with encryption enabled is created. Remove unnecessary functions fil_get_first_space_safe() and fil_get_next_space_safe(). fil_space_acquire() and fil_space_release() are used instead. Moved fil_space_get_crypt_data() and fil_space_set_crypt_data() to fil0crypt.cc. fsp_header_init(): Acquire fil_space_t, write crypt_data and release space. check_table_options() Renamed FIL_SPACE_ENCRYPTION_ TO FIL_ENCRYPTION_* i_s.cc: Added ROTATING_OR_FLUSHING field to information_schema.innodb_tablespace_encryption to show current status of key rotation.	9 years ago
Marko Mäkelä	74fe0e03d5	Remove unused declarations.	9 years ago
Vicențiu Ciorbaru	8d69ce7b82	5.6.35-80.0	9 years ago
Marko Mäkelä	ec4cf111c0	MDEV-11520 after-merge fix for 10.1: Use sparse files. If page_compression (introduced in MariaDB Server 10.1) is enabled, the logical action is to not preallocate space to the data files, but to only logically extend the files with zeroes. fil_create_new_single_table_tablespace(): Create smaller files for ROW_FORMAT=COMPRESSED tables, but adhere to the minimum file size of 4*innodb_page_size. fil_space_extend_must_retry(), os_file_set_size(): On Windows, use SetFileInformationByHandle() and FILE_END_OF_FILE_INFO, which depends on bumping _WIN32_WINNT to 0x0600. FIXME: The files are not yet set up as sparse, so this will currently end up physically extending (preallocating) the files, wasting storage for unused pages. os_file_set_size(): Add the parameter "bool sparse=false" to declare that the file is to be extended logically, instead of being preallocated. The only caller with sparse=true is fil_create_new_single_table_tablespace(). (The system tablespace cannot be created with page_compression.) fil_space_extend_must_retry(), os_file_set_size(): Outside Windows, use ftruncate() to extend files that are supposed to be sparse. On systems where ftruncate() is limited to files less than 4GiB (if there are any), fil_space_extend_must_retry() retains the old logic of physically extending the file.	9 years ago
Marko Mäkelä	f162704570	Rewrite the innodb.log_file_size test with DBUG_EXECUTE_IF. Remove the debug parameter innodb_force_recovery_crash that was introduced into MySQL 5.6 by me in WL#6494 which allowed InnoDB to resize the redo log on startup. Let innodb.log_file_size actually start up the server, but ensure that the InnoDB storage engine refuses to start up in each of the scenarios.	9 years ago
Marko Mäkelä	afb461587c	MDEV-11915 Detect InnoDB system tablespace size mismatch early InnoDB would refuse to start up if there is a mismatch on the size of the system tablespace files. However, before this check is conducted, the system tablespace may already have been heavily modified. InnoDB should perform the size check as early as possible. recv_recovery_from_checkpoint_finish(): Move the recv_apply_hashed_log_recs() call to innobase_start_or_create_for_mysql(). innobase_start_or_create_for_mysql(): Test the mutex functionality before doing anything else. Use a compile_time_assert() for a sizeof() constraint. Check the size of the system tablespace as early as possible.	9 years ago
Marko Mäkelä	ab1e6fefd8	MDEV-11623 MariaDB 10.1 fails to start datadir created with MariaDB 10.0/MySQL 5.6 using innodb-page-size!=16K The storage format of FSP_SPACE_FLAGS was accidentally broken already in MariaDB 10.1.0. This fix is bringing the format in line with other MySQL and MariaDB release series. Please refer to the comments that were added to fsp0fsp.h for details. This is an INCOMPATIBLE CHANGE that affects users of page_compression and non-default innodb_page_size. Upgrading to this release will correct the flags in the data files. If you want to downgrade to earlier MariaDB 10.1.x, please refer to the test innodb.101_compatibility how to reset the FSP_SPACE_FLAGS in the files. NOTE: MariaDB 10.1.0 to 10.1.20 can misinterpret uncompressed data files with innodb_page_size=4k or 64k as compressed innodb_page_size=16k files, and then probably fail when trying to access the pages. See the comments in the function fsp_flags_convert_from_101() for detailed analysis. Move PAGE_COMPRESSION to FSP_SPACE_FLAGS bit position 16. In this way, compressed innodb_page_size=16k tablespaces will not be mistaken for uncompressed ones by MariaDB 10.1.0 to 10.1.20. Derive PAGE_COMPRESSION_LEVEL, ATOMIC_WRITES and DATA_DIR from the dict_table_t::flags when the table is available, in fil_space_for_table_exists_in_mem() or fil_open_single_table_tablespace(). During crash recovery, fil_load_single_table_tablespace() will use innodb_compression_level for the PAGE_COMPRESSION_LEVEL. FSP_FLAGS_MEM_MASK: A bitmap of the memory-only fil_space_t::flags that are not to be written to FSP_SPACE_FLAGS. Currently, these will include PAGE_COMPRESSION_LEVEL, ATOMIC_WRITES and DATA_DIR. Introduce the macro FSP_FLAGS_PAGE_SSIZE(). We only support one innodb_page_size for the whole instance. When creating a dummy tablespace for the redo log, use fil_space_t::flags=0. The flags are never written to the redo log files. Remove many FSP_FLAGS_SET_ macros. dict_tf_verify_flags(): Remove. This is basically only duplicating the logic of dict_tf_to_fsp_flags(), used in a debug assertion. fil_space_t::mark: Remove. This flag was not used for anything. fil_space_for_table_exists_in_mem(): Remove the unnecessary parameter mark_space, and add a parameter for table flags. Check that fil_space_t::flags match the table flags, and adjust the (memory-only) flags based on the table flags. fil_node_open_file(): Remove some redundant or unreachable conditions, do not use stderr for output, and avoid unnecessary server aborts. fil_user_tablespace_restore_page(): Convert the flags, so that the correct page_size will be used when restoring a page from the doublewrite buffer. fil_space_get_page_compressed(), fsp_flags_is_page_compressed(): Remove. It suffices to have fil_space_is_page_compressed(). FSP_FLAGS_WIDTH_DATA_DIR, FSP_FLAGS_WIDTH_PAGE_COMPRESSION_LEVEL, FSP_FLAGS_WIDTH_ATOMIC_WRITES: Remove, because these flags do not exist in the FSP_SPACE_FLAGS but only in memory. fsp_flags_try_adjust(): New function, to adjust the FSP_SPACE_FLAGS in page 0. Called by fil_open_single_table_tablespace(), fil_space_for_table_exists_in_mem(), innobase_start_or_create_for_mysql() except if --innodb-read-only is active. fsp_flags_is_valid(ulint): Reimplement from the scratch, with accurate comments. Do not display any details of detected inconsistencies, because the output could be confusing when dealing with MariaDB 10.1.x data files. fsp_flags_convert_from_101(ulint): Convert flags from buggy MariaDB 10.1.x format, or return ULINT_UNDEFINED if the flags cannot be in MariaDB 10.1.x format. fsp_flags_match(): Check the flags when probing files. Implemented based on fsp_flags_is_valid() and fsp_flags_convert_from_101(). dict_check_tablespaces_and_store_max_id(): Do not access the page after committing the mini-transaction. IMPORT TABLESPACE fixes: AbstractCallback::init(): Convert the flags. FetchIndexRootPages::operator(): Check that the tablespace flags match the table flags. Do not attempt to convert tablespace flags to table flags, because the conversion would necessarily be lossy. PageConverter::update_header(): Write back the correct flags. This takes care of the flags in IMPORT TABLESPACE.	9 years ago
Marko Mäkelä	384f4d1e36	Post-push fix for MDEV-11556: Make the debug variable UINT. Sometimes innodb_data_file_size_debug was reported as INT UNSIGNED instead of BIGINT UNSIGNED. Make it uint instead of ulong to get a more deterministic result.	9 years ago
Marko Mäkelä	719321e78e	MDEV-11638 Encryption causes race conditions in InnoDB shutdown InnoDB shutdown failed to properly take fil_crypt_thread() into account. The encryption threads were signalled to shut down together with other non-critical tasks. This could be much too early in case of slow shutdown, which could need minutes to complete the purge. Furthermore, InnoDB failed to wait for the fil_crypt_thread() to actually exit before proceeding to the final steps of shutdown, causing the race conditions. Furthermore, the log_scrub_thread() was shut down way too early. Also it should remain until the SRV_SHUTDOWN_FLUSH_PHASE. fil_crypt_threads_end(): Remove. This would cause the threads to be terminated way too early. srv_buf_dump_thread_active, srv_dict_stats_thread_active, lock_sys->timeout_thread_active, log_scrub_thread_active, srv_monitor_active, srv_error_monitor_active: Remove a race condition between startup and shutdown, by setting these in the startup thread that creates threads, not in each created thread. In this way, once the flag is cleared, it will remain cleared during shutdown. srv_n_fil_crypt_threads_started, fil_crypt_threads_event: Declare in global rather than static scope. log_scrub_event, srv_log_scrub_thread_active, log_scrub_thread(): Declare in static rather than global scope. Let these be created by log_init() and freed by log_shutdown(). rotate_thread_t::should_shutdown(): Do not shut down before the SRV_SHUTDOWN_FLUSH_PHASE. srv_any_background_threads_are_active(): Remove. These checks now exist in logs_empty_and_mark_files_at_shutdown(). logs_empty_and_mark_files_at_shutdown(): Shut down the threads in the proper order. Keep fil_crypt_thread() and log_scrub_thread() alive until SRV_SHUTDOWN_FLUSH_PHASE, and check that they actually terminate.	9 years ago
Marko Mäkelä	80d5d1452a	MDEV-11694 InnoDB tries to create unused table SYS_ZIP_DICT MariaDB Server 10.0.28 and 10.1.19 merged code from Percona XtraDB that introduced support for compressed columns. Much but not all of this code was disabled by placing #ifdef HAVE_PERCONA_COMPRESSED_COLUMNS around it. Among the unused but not disabled code is code to access some new system tables related to compressed columns. The creation of these system tables SYS_ZIP_DICT and SYS_ZIP_DICT_COLS would cause a crash in --innodb-read-only mode when upgrading from an earlier version to 10.0.28 or 10.1.19. Let us remove all the dead code related to compressed columns. Users who already upgraded to 10.0.28 and 10.1.19 will have the two above mentioned empty tables in their InnoDB system tablespace. Subsequent versions of MariaDB Server will completely ignore those tables.	9 years ago
Marko Mäkelä	8451e09073	MDEV-11556 InnoDB redo log apply fails to adjust data file sizes fil_space_t::recv_size: New member: recovered tablespace size in pages; 0 if no size change was read from the redo log, or if the size change was implemented. fil_space_set_recv_size(): New function for setting space->recv_size. innodb_data_file_size_debug: A debug parameter for setting the system tablespace size in recovery even when the redo log does not contain any size changes. It is hard to write a small test case that would cause the system tablespace to be extended at the critical moment. recv_parse_log_rec(): Note those tablespaces whose size is being changed by the redo log, by invoking fil_space_set_recv_size(). innobase_init(): Correct an error message, and do not require a larger innodb_buffer_pool_size when starting up with a smaller innodb_page_size. innobase_start_or_create_for_mysql(): Allow startup with any initial size of the ibdata1 file if the autoextend attribute is set. Require the minimum size of fixed-size system tablespaces to be 640 pages, not 10 megabytes. Implement innodb_data_file_size_debug. open_or_create_data_files(): Round the system tablespace size down to pages, not to full megabytes, (Our test truncates the system tablespace to more than 800 pages with innodb_page_size=4k. InnoDB should not imagine that it was truncated to 768 pages and then overwrite good pages in the tablespace.) fil_flush_low(): Refactored from fil_flush(). fil_space_extend_must_retry(): Refactored from fil_extend_space_to_desired_size(). fil_mutex_enter_and_prepare_for_io(): Extend the tablespace if fil_space_set_recv_size() was called. The test case has been successfully run with all the innodb_page_size values 4k, 8k, 16k, 32k, 64k.	9 years ago
Marko Mäkelä	d50cf42bc0	MDEV-9282 Debian: the Lintian complains about "shlib-calls-exit" in ha_innodb.so Replace all exit() calls in InnoDB with abort() [possibly via ut_a()]. Calling exit() in a multi-threaded program is problematic also for the reason that other threads could see corrupted data structures while some data structures are being cleaned up by atexit() handlers or similar. In the long term, all these calls should be replaced with something that returns an error all the way up the call stack.	9 years ago
Marko Mäkelä	7e02fd1f71	MDEV-11630 Call mutex_free() before freeing the mutex list Make some global fil_crypt_ variables static. fil_close(): Call mutex_free(&fil_system->mutex) also in InnoDB, not only in XtraDB. In InnoDB, sync_close() was called before fil_close(). innobase_shutdown_for_mysql(): Call fil_close() before sync_close(), similar to XtraDB shutdown. fil_space_crypt_cleanup(): Call mutex_free() to pair with fil_space_crypt_init(). fil_crypt_threads_cleanup(): Call mutex_free() to pair with fil_crypt_threads_init().	9 years ago
Sergei Golubchik	d7dc03a267	5.6.33-79.0	9 years ago
Vladislav Vaintroub	ee1d08c115	Revert "Prepare XtraDB to be used with xtrabackup." This reverts commit `de5646f1a9`.	9 years ago
Vladislav Vaintroub	de5646f1a9	Prepare XtraDB to be used with xtrabackup. The changes are deliberately kept minimal - some functions are made global instead of static (they will be used in xtrabackup later on) - functions got additional parameter, deliberately unused for now : fil_load_single_tablespaces srv_undo_tablespaces_init - Global variables added, also unused for now : srv_archive_recovery srv_archive_recovery_limit_lsn srv_apply_log_only srv_backup_mode srv_close_files - To make xtrabackup link with sql.lib on Windows, added some missing source files to sql.lib - Fixed os_thread_ret_t to be DWORD on Windows	9 years ago
Jan Lindström	2bedc3978b	MDEV-9931: InnoDB reads first page of every .ibd file at startup Analysis: By design InnoDB was reading first page of every .ibd file at startup to find out is tablespace encrypted or not. This is because tablespace could have been encrypted always, not encrypted newer or encrypted based on configuration and this information can be find realible only from first page of .ibd file. Fix: Do not read first page of every .ibd file at startup. Instead whenever tablespace is first time accedded we will read the first page to find necessary information about tablespace encryption status. TODO: Add support for SYS_TABLEOPTIONS where all table options encryption information included will be stored.	9 years ago
Sergei Golubchik	64752acf72	5.6.31-77.0	9 years ago
Sergei Golubchik	b42664e85e	5.6.30-76.3	10 years ago
Sergei Golubchik	5b8ac23030	5.6.29-76.2	10 years ago
Sergei Golubchik	d76eba6a6b	5.6.28-76.1	10 years ago
Jan Lindström	111acb721f	MDEV-9359: encryption.create_or_replace fails sporadically in buildbot: failing assertion: mutex->magic_n == MUTEX_MAGIC_N Make sure that encryption threads mutex is initialized before starting encryption threads.	10 years ago
Sergei Golubchik	dfb58a3782	innodb/xtradb: init scrub mutex even in read-only mode because it's locked for innodb part of SHOW STATUS, which can happen in read-only mode too.	10 years ago
Daniel Axtens	2538c7cf89	Use POWER8 accelerated crc32 - Make accelerated checksum available to InnoDB and XtraDB. - Fall back to slice-by-eight if not available. The mode used is printed on startup. - Will only build on POWER systems at the moment until CMakeLists are modified to only add the crc32_power8/ files when building on POWER. running MySQL-5.7 unittest/gunit/innodb/ut0crc32-t Before: 1..2 Using software crc32 implementation, CPU is little-endian ok 1 Using software crc32 implementation, CPU is little-endian normal CRC32: real 0.148006 sec normal CRC32: user 0.148000 sec normal CRC32: sys 0.000000 sec big endian CRC32: real 0.144293 sec big endian CRC32: user 0.144000 sec big endian CRC32: sys 0.000000 sec ok 2 After: 1..2 Using POWER8 crc32 implementation, CPU is little-endian ok 1 Using POWER8 crc32 implementation, CPU is little-endian normal CRC32: real 0.008097 sec normal CRC32: user 0.008000 sec normal CRC32: sys 0.000000 sec big endian CRC32: real 0.147043 sec big endian CRC32: user 0.144000 sec big endian CRC32: sys 0.000000 sec ok 2 Author CRC32 ASM code: Anton Blanchard <anton@au.ibm.com> ref: https://github.com/antonblanchard/crc32-vpmsum Signed-off-by: Daniel Black <daniel.black@au.ibm.com>	10 years ago
Sergei Golubchik	1e270d504d	5.6.27-76.0	10 years ago
Sergei Golubchik	6a821d78a6	5.6.26-74.0	10 years ago
Sergei Golubchik	0b3eb4559b	5.6.25-73.1	10 years ago
Nirbhay Choubey	dced5146bd	Merge branch '10.0-galera' into 10.1	10 years ago
Jan Lindström	6efdc398ba	MDEV-8463: 10.0 tree does not build Merge error on srv0start.cc	10 years ago
josh4trunks	ee5633a39e	Apply fix for raw devices to XtraDB	10 years ago
Sergei Golubchik	fe7e334f3e	cleanup: remove unused function argument	11 years ago
Jan Lindström	27f0bd7d95	Fix test case innodb.xa_recovery crash on xtradb.	11 years ago
Jan Lindström	59815a268b	MDEV-7484: Log Time not consistent with InnoDB errors nor with MySQL error log time format	11 years ago
Jan Lindström	f8cacd03a7	MDEV-8143: InnoDB: Database page corruption on disk or a failed file read Analysis: Problem was that we did create crypt data for encrypted table but this new crypt data was not written to page 0. Instead a default crypt data was written to page 0 at table creation. Fixed by explicitly writing new crypt data to page 0 after successfull table creation.	11 years ago

1 2 3

129 Commits (8c4df595b8bdd186ee68aee8a7953958fa503d3c)