mariadb

Commit Graph

Author	SHA1	Message	Date
Marko Mäkelä	644ffdeb92	Fix integer type mismatch in WSREP debug output	8 years ago
Marko Mäkelä	dfd010ef90	MDEV-14086 Setting innodb_buffer_pool_load_now or innodb_buffer_load_abort will crash if innodb_read_only When MySQL 5.6.10 introduced innodb_read_only mode, it skipped the creation of the InnoDB buffer pool dump/restore subsystem in that mode. Attempts to set the variable innodb_buf_pool_dump_now would have no effect in innodb_read_only mode, but the corresponding condition was forgotten in from the other two update functions. MySQL 5.7.20 would fix the innodb_buffer_pool_load_now, but not innodb_buffer_pool_load_abort. Let us fix both in MariaDB.	8 years ago
Sergei Golubchik	d76f5774fe	MDEV-13459 Warnings, when compiling with gcc-7.x mostly caused by -Wimplicit-fallthrough	8 years ago
Jan Lindström	fc9ff69578	MDEV-13838: Wrong result after altering a partitioned table Reverted incorrect changes done on MDEV-7367 and MDEV-9469. Fixes properly also related bugs: MDEV-13668: InnoDB unnecessarily rebuilds table when renaming a column and adding index MDEV-9469: 'Incorrect key file' on ALTER TABLE MDEV-9548: Alter table (renaming and adding index) fails with "Incorrect key file for table" MDEV-10535: ALTER TABLE causes standalone/wsrep cluster crash MDEV-13640: ALTER TABLE CHANGE and ADD INDEX on auto_increment column fails with "Incorrect key file for table..." Root cause for all these bugs is the fact that MariaDB .frm file can contain virtual columns but InnoDB dictionary does not and previous fixes were incorrect or unnecessarily forced table rebuilt. In index creation key_part->fieldnr can be bigger than number of columns in InnoDB data dictionary. We need to skip not stored fields when calculating correct column number for InnoDB data dictionary. dict_table_get_col_name_for_mysql Remove innobase_match_index_columns Revert incorrect change done on MDEV-7367 innobase_need_rebuild Remove unnecessary rebuild force when column is renamed. innobase_create_index_field_def Calculate InnoDB column number correctly and remove unnecessary column name set. innobase_create_index_def, innobase_create_key_defs Remove unneeded fields parameter. Revert unneeded memset. prepare_inplace_alter_table_dict Remove unneeded col_names parameter index_field_t Remove unneeded col_name member. row_merge_create_index Remove unneeded col_names parameter and resolution. Effected tests: innodb-alter-table : Add test case for MDEV-13668 innodb-alter : Remove MDEV-13668, MDEV-9469 FIXMEs and restore original tests innodb-wl5980-alter : Remove MDEV-13668, MDEV-9469 FIXMEs and restore original tests	8 years ago
Sachin Setiya	0627929f62	MDEV-13787 Crash in persistent stats wsrep_on (thd=0x0) Problem:- This crash happens because of thd = NULL , and while checking for wsrep_on , we no longer check for thd != NULL (MDEV-7955). So this problem is regression of MDEV-7955. However this patch not only solves this regression , It solves all regression caused by MDEV-7955 patch. To get all possible cases when thd can be null , assert(thd)/ assert(trx->mysql_thd) is place just before all wsrep_on and innodb test suite is run. And the assert which caused failure are removed with a physical check for thd != NULL. Rest assert are removed. Hopefully this method will remove all current/potential regression of MDEV-7955.	8 years ago
Vicențiu Ciorbaru	b4606367d7	5.6.37-82.2	8 years ago
Vladislav Vaintroub	ad17e8e518	MDEV-13821 : mariabackup sometimes could lose ib_logf(FATAL) messages, The messages are getting lost because they are written with fprintf() but without fflush(), so abort() would lose buffered text. Applied fix from 10.2, which makes in_logf() use sql_print_information(), which in turn does proper flush after each message	8 years ago
Jan Lindström	fa2701c6f7	MDEV-12634: Uninitialised ROW_MERGE_RESERVE_SIZE bytes written to tem… …porary file Fixed by removing writing key version to start of every block that was encrypted. Instead we will use single key version from log_sys crypt info. After this MDEV also blocks writen to row log are encrypted and blocks read from row log aren decrypted if encryption is configured for the table. innodb_status_variables[], struct srv_stats_t Added status variables for merge block and row log block encryption and decryption amounts. Removed ROW_MERGE_RESERVE_SIZE define. row_merge_fts_doc_tokenize Remove ROW_MERGE_RESERVE_SIZE row_log_t Add index, crypt_tail, crypt_head to be used in case of encryption. row_log_online_op, row_log_table_close_func Before writing a block encrypt it if encryption is enabled row_log_table_apply_ops, row_log_apply_ops After reading a block decrypt it if encryption is enabled row_log_allocate Allocate temporary buffers crypt_head and crypt_tail if needed. row_log_free Free temporary buffers crypt_head and crypt_tail if they exist. row_merge_encrypt_buf, row_merge_decrypt_buf Removed. row_merge_buf_create, row_merge_buf_write Remove ROW_MERGE_RESERVE_SIZE row_merge_build_indexes Allocate temporary buffer used in decryption and encryption if needed. log_tmp_blocks_crypt, log_tmp_block_encrypt, log_temp_block_decrypt New functions used in block encryption and decryption log_tmp_is_encrypted New function to check is encryption enabled. Added test case innodb-rowlog to force creating a row log and verify that operations are done using introduced status variables.	8 years ago
Andrei Elkin	888a8b69bd	MDEV-13437 InnoDB fails to return error for XA COMMIT or XA ROLLBACK in read-only mode Assertions failed due to incorrect handling of the --tc-heuristic-recover option when InnoDB is in read-only mode either due to innodb_read_only=1 or innodb_force_recovery>3. InnoDB failed to refuse a XA COMMIT or XA ROLLBACK operation, and there were errors in the error handling in the upper layer. This was fixed by making InnoDB XA operations respect the high_level_read_only flag. The InnoDB part of the fix and parts of the test main.tc_heuristic_recover were provided by Marko Mäkelä. LOCK_log mutex lock/unlock had to be added to fix MDEV-13438. The measure is confirmed by mysql sources as well. For testing of the conflicting option combination, mysql-test-run is made to export a new $MYSQLD_LAST_CMD. It holds the very last value generated by mtr.mysqld_start(). Even though the options have been also always stored in $mysqld->{'started_opts'} there were no access to them beyond the automatic server restart by mtr through the expect file interface. Effectively therefore $MYSQLD_LAST_CMD represents a more general interface to $mysqld->{'started_opts'} which can be used in wider scopes including server launch with incompatible options. Notice another existing method to restart the server with incompatible options relying on $MYSQLD_CMD is is aware of $mysqld->{'started_opts'} (the actual options that the server is launched by mtr). In order to use this method they would have to be provided manually. NOTE: When merging to 10.2, the file search_pattern_in_file++.inc should be replaced with the pre-existing search_pattern_in_file.inc.	8 years ago
sjaakola	0ddf32c840	refs: MW-322 * generating fake trx id for CTAS, requires trx_sys mutex protection to be safe for concurrent CTAS processors	9 years ago
Jan Lindström	9284e8b2c6	MDEV-11828: innodb_page_size=64k must reject ROW_FORMAT=REDUNDANT records longer than 16383 bytes In all InnoDB row formats, the pointers or lengths stored in the record header can be at most 14 bits, that is, count up to 16383. In ROW_FORMAT=REDUNDANT, this limits the maximum possible record length to 16383 bytes. In other ROW_FORMAT, it could merely limit the maximum length of variable-length fields. When MySQL 5.7 introduced innodb_page_size=32k and 64k, the maximum record length was limited to 16383 bytes (I hope 16383, not 16384, to be able to distinguish from a record whose length is 0 bytes). This change is present in MariaDB Server 10.2. btr_cur_optimistic_update(): Restrict maximum record size to 16K-1 for REDUNDANT and 64K page size. dict_index_too_big_for_tree(): The maximum allowed record size is half a B-tree page or 16K(-1 for REDUNDANT) for 64K page size. convert_error_code_to_mysql(): Fix error message to print correct limits. my_error_innodb(): Fix error message to print correct limits. page_zip_rec_needs_ext() : record size was already restricted to 16K. Restrict REDUNDANT to 16K-1. rem0rec.h: Introduce REDUNDANT_REC_MAX_DATA_SIZE (16K-1) and COMPRESSED_REC_MAX_DATA_SIZE (16K).	8 years ago
Marko Mäkelä	2b5c9bc2c8	MDEV-13247 innodb_log_compressed_pages=OFF breaks crash recovery of ROW_FORMAT=COMPRESSED tables The option innodb_log_compressed_pages was contributed by Facebook to MySQL 5.6. It was disabled in the 5.6.10 GA release due to problems that were fixed in 5.6.11, which is when the option was enabled. The option was set to innodb_log_compressed_pages=ON by default (disabling the feature), because safety was considered more important than speed. The option innodb_log_compressed_pages=OFF can CORRUPT ROW_FORMAT=COMPRESSED tables on crash recovery if the zlib deflate function is behaving differently (producing a different amount of compressed data) from how it behaved when the redo log records were written (prior to the crash recovery). In MDEV-6935, the default value was changed to innodb_log_compressed_pages=OFF. This is inherently unsafe, because there are very many different environments where MariaDB can be running, using different zlib versions. While zlib can decompress data just fine, there are no guarantees that different versions will always compress the same data to the exactly same size. To avoid problems related to zlib upgrades or version mismatch, we must use a safe default setting. This will reduce the write performance for users of ROW_FORMAT=COMPRESSED tables. If you configure innodb_log_compressed_pages=ON, please make sure that you will always cleanly shut down InnoDB before upgrading the server or zlib.	8 years ago
Marko Mäkelä	e5980bf1b1	Remove the unnecessary method handlerton::release_temporary_latches() The sole purpose of handlerton::release_temporary_latches and its wrapper function was to release the InnoDB adaptive hash index latch (btr_search_latch). When the btr_search_latch was split into an array of latches in MySQL 5.7.8 as part of the Oracle Bug#20985298 fix, the "caching" of the latch across storage engine API calls was removed. As part of that, the function trx_search_latch_release_if_reserved() was changed to an assertion and the function trx_reserve_search_latch_if_not_reserved() was removed, and handlerton::release_temporary_latches() practically became a no-op. Note: MDEV-12121 replaced the function trx_search_latch_release_if_reserved() with the more appropriately named macro trx_assert_no_search_latch().	9 years ago
Marko Mäkelä	58f87a41bd	Remove some fields from dict_table_t dict_table_t::thd: Remove. This was only used by btr_root_block_get() for reporting decryption failures, and it was only assigned by ha_innobase::open(), and never cleared. This could mean that if a connection is closed, the pointer would become stale, and the server could crash while trying to report the error. It could also mean that an error is being reported to the wrong client. It is better to use current_thd in this case, even though it could mean that if the code is invoked from an InnoDB background operation, there would be no connection to which to send the error message. Remove dict_table_t::crypt_data and dict_table_t::page_0_read. These fields were never read. fil_open_single_table_tablespace(): Remove the parameter "table".	9 years ago
Marko Mäkelä	4325041df6	MDEV-13057 innodb_read_only=1 should avoid creating buf_flush_page_cleaner_thread When the server is started in innodb_read_only mode, there cannot be any writes to persistent InnoDB/XtraDB files. Just like the creation of buf_flush_page_cleaner_thread is skipped in this case, also the creation of the XtraDB-specific buf_flush_lru_manager_thread should be skipped.	9 years ago
Marko Mäkelä	417434f12d	MDEV-13039 innodb_fast_shutdown=0 may fail to purge all undo log When a slow shutdown is performed soon after spawning some work for background threads that can create or commit transactions, it is possible that new transactions are started or committed after the purge has finished. This is violating the specification of innodb_fast_shutdown=0, namely that the purge must be completed. (None of the history of the recent transactions would be purged.) Also, it is possible that the purge threads would exit in slow shutdown while there exist active transactions, such as recovered incomplete transactions that are being rolled back. Thus, the slow shutdown could fail to purge some undo log that becomes purgeable after the transaction commit or rollback. srv_undo_sources: A flag that indicates if undo log can be generated or the persistent, whether by background threads or by user SQL. Even when this flag is clear, active transactions that already exist in the system may be committed or rolled back. innodb_shutdown(): Renamed from innobase_shutdown_for_mysql(). Do not return an error code; the operation never fails. Clear the srv_undo_sources flag, and also ensure that the background DROP TABLE queue is empty. srv_purge_should_exit(): Do not allow the purge to exit if srv_undo_sources are active or the background DROP TABLE queue is not empty, or in slow shutdown, if any active transactions exist (and are being rolled back). srv_purge_coordinator_thread(): Remove some previous workarounds for this bug. innobase_start_or_create_for_mysql(): Set buf_page_cleaner_is_active and srv_dict_stats_thread_active directly. Set srv_undo_sources before starting the purge subsystem, to prevent immediate shutdown of the purge. Create dict_stats_thread and fts_optimize_thread immediately after setting srv_undo_sources, so that shutdown can use this flag to determine if these subsystems were started. dict_stats_shutdown(): Shut down dict_stats_thread. Backported from 10.2. srv_shutdown_table_bg_threads(): Remove (unused).	9 years ago
Marko Mäkelä	fbeb9489cd	Cleanup of MDEV-12600: crash during install_db with innodb_page_size=32K and ibdata1=3M The doublewrite buffer pages must fit in the first InnoDB system tablespace data file. The checks that were added in the initial patch (commit `112b21da37`) were at too high level and did not cover all cases. innodb.log_data_file_size: Test all innodb_page_size combinations. fsp_header_init(): Never return an error. Move the change buffer creation to the only caller that needs to do it. btr_create(): Clean up the logic. Remove the error log messages. buf_dblwr_create(): Try to return an error on non-fatal failure. Check that the first data file is big enough for creating the doublewrite buffers. buf_dblwr_process(): Check if the doublewrite buffer is available. Display the message only if it is available. recv_recovery_from_checkpoint_start_func(): Remove a redundant message about FIL_PAGE_FILE_FLUSH_LSN mismatch when crash recovery has already been initiated. fil_report_invalid_page_access(): Simplify the message. fseg_create_general(): Do not emit messages to the error log. innobase_init(): Revert the changes. trx_rseg_create(): Refactor (no functional change).	9 years ago
Jan Lindström	112b21da37	MDEV-12600: crash during install_db with innodb_page_size=32K and ibdata1=3M; Problem was that all doublewrite buffer pages must fit to first system datafile. Ported commit `27a34df788` Author: Shaohua Wang <shaohua.wang@oracle.com> Date: Wed Aug 12 15:55:19 2015 +0800 BUG#21551464 - SEGFAULT WHILE INITIALIZING DATABASE WHEN INNODB_DATA_FILE SIZE IS SMALL To 10.1 (with extended error printout). btr_create(): If ibuf header page allocation fails report error and return FIL_NULL. Similarly if root page allocation fails return a error. dict_build_table_def_step: If fsp_header_init fails return error code. fsp_header_init: returns true if header initialization succeeds and false if not. fseg_create_general: report error if segment or page allocation fails. innobase_init: If first datafile is smaller than 3M and could not contain all doublewrite buffer pages report error and fail to initialize InnoDB plugin. row_truncate_table_for_mysql: report error if fsp header init fails. srv_init_abort: New function to report database initialization errors. srv_undo_tablespaces_init, innobase_start_or_create_for_mysql: If database initialization fails report error and abort. trx_rseg_create: If segment header creation fails return.	9 years ago
Jan Lindström	1af8bf39ca	MDEV-12113: install_db shows corruption for rest encryption with innodb_data_file_path=ibdata1:3M; Problem was that FIL_PAGE_FLUSH_LSN_OR_KEY_VERSION field that for encrypted pages even in system datafiles should contain key_version except very first page (0:0) is after encryption overwritten with flush lsn. Ported WL#7990 Repurpose FIL_PAGE_FLUSH_LSN to 10.1 The field FIL_PAGE_FLUSH_LSN_OR_KEY_VERSION is consulted during InnoDB startup. At startup, InnoDB reads the FIL_PAGE_FLUSH_LSN_OR_KEY_VERSION from the first page of each file in the InnoDB system tablespace. If there are multiple files, the minimum and maximum LSN can differ. These numbers are passed to InnoDB startup. Having the number in other files than the first file of the InnoDB system tablespace is not providing much additional value. It is conflicting with other use of the field, such as on InnoDB R-tree index pages and encryption key_version. This worklog will stop writing FIL_PAGE_FLUSH_LSN_OR_KEY_VERSION to other files than the first file of the InnoDB system tablespace (page number 0:0) when system tablespace is encrypted. If tablespace is not encrypted we continue writing FIL_PAGE_FLUSH_LSN_OR_KEY_VERSION to all first pages of system tablespace to avoid unnecessary warnings on downgrade. open_or_create_data_files(): pass only one flushed_lsn parameter xb_load_tablespaces(): pass only one flushed_lsn parameter. buf_page_create(): Improve comment about where FIL_PAGE_FIL_FLUSH_LSN_OR_KEY_VERSION is set. fil_write_flushed_lsn(): A new function, merged from fil_write_lsn_and_arch_no_to_file() and fil_write_flushed_lsn_to_data_files(). Only write to the first page of the system tablespace (page 0:0) if tablespace is encrypted, or write all first pages of system tablespace and invoke fil_flush_file_spaces(FIL_TYPE_TABLESPACE) afterwards. fil_read_first_page(): read flush_lsn and crypt_data only from first datafile. fil_open_single_table_tablespace(): Remove output of LSN, because it was only valid for the system tablespace and the undo tablespaces, not user tablespaces. fil_validate_single_table_tablespace(): Remove output of LSN. checkpoint_now_set(): Use fil_write_flushed_lsn and output a error if operation fails. Remove lsn variable from fsp_open_info. recv_recovery_from_checkpoint_start(): Remove unnecessary second flush_lsn parameter. log_empty_and_mark_files_at_shutdown(): Use fil_writte_flushed_lsn and output error if it fails. open_or_create_data_files(): Pass only one flushed_lsn variable.	9 years ago
Sergei Golubchik	7c03edf2fe	MDEV-6262 analyze the coverity report on mariadb uploaded 10.0, analyzed everything with the Impact=High (and a couple of Medium)	9 years ago
Vicențiu Ciorbaru	3c7af6c490	Fix xtradb handler compilation post merge	9 years ago
Marko Mäkelä	54bb04f7ef	Fix some __attribute__((nonnull)) misuse This fixes warnings that were emitted when running InnoDB test suites on a debug server that was compiled with GCC 7.1.0 using the flags -O3 -fsanitize=undefined. thd_requested_durability(): XtraDB can call this with trx->mysql_thd=NULL. Remove the function in InnoDB, because it is not used there. calc_row_difference(): Do not call memcmp(o_ptr, NULL, 0). innobase_index_name_is_reserved(): This can be called with key_info=NULL, num_of_keys=0. innobase_dropping_foreign(), innobase_check_foreigns_low(), innobase_check_foreigns(): This can be called with drop_fk=NULL, n_drop_fk=0. rec_convert_dtuple_to_rec_comp(): Do not invoke memcpy(end, NULL, 0).	9 years ago
Marko Mäkelä	9f89b94ba6	MDEV-12358 Work around what looks like a bug in GCC 7.1.0 The parameter thr of the function btr_cur_optimistic_insert() is not declared as nonnull, but GCC 7.1.0 with -O3 is wrongly optimizing away the first part of the condition UNIV_UNLIKELY(thr && thr_get_trx(thr)->fake_changes) when the function is being called by row_merge_insert_index_tuples() with thr==NULL. The fake_changes is an XtraDB addition. This GCC bug only appears to have an impact on XtraDB, not InnoDB. We work around the problem by not attempting to dereference thr when both BTR_NO_LOCKING_FLAG and BTR_NO_UNDO_LOG_FLAG are set in the flags. Probably BTR_NO_LOCKING_FLAG alone should suffice. btr_cur_optimistic_insert(), btr_cur_pessimistic_insert(), btr_cur_pessimistic_update(): Correct comments that disagree with usage and with nonnull attributes. No other parameter than thr can actually be NULL. row_ins_duplicate_error_in_clust(): Remove an unused parameter. innobase_is_fake_change(): Unused function; remove. ibuf_insert_low(), row_log_table_apply(), row_log_apply(), row_undo_mod_clust_low(): Because we will be passing BTR_NO_LOCKING_FLAG \| BTR_NO_UNDO_LOG_FLAG in the flags, the trx->fake_changes flag will be treated as false, which is the right thing to do at these low-level operations (change buffer merge, ALTER TABLE…LOCK=NONE, or ROLLBACK). This might be fixing actual XtraDB bugs. Other callers that pass these two flags are also passing thr=NULL, implying fake_changes=false. (Some callers in ROLLBACK are passing BTR_NO_LOCKING_FLAG and a nonnull thr. In these callers, fake_changes better be false, to avoid corruption.)	9 years ago
Marko Mäkelä	408ef65f84	Never pass NULL to innobase_get_stmt()	9 years ago
Marko Mäkelä	71cd205956	Silence bogus GCC 7 warnings -Wimplicit-fallthrough Do not silence uncertain cases, or fix any bugs. The only functional change should be that ha_federated::extra() is not calling DBUG_PRINT to report an unhandled case for HA_EXTRA_PREPARE_FOR_DROP.	9 years ago
Marko Mäkelä	4754f88cff	Never pass NULL to innobase_get_stmt()	9 years ago
Marko Mäkelä	7972da8aa1	Silence bogus GCC 7 warnings -Wimplicit-fallthrough Do not silence uncertain cases, or fix any bugs. The only functional change should be that ha_federated::extra() is not calling DBUG_PRINT to report an unhandled case for HA_EXTRA_PREPARE_FOR_DROP.	9 years ago
Marko Mäkelä	b8187fd005	MDEV-12071 storage/xtradb/handler/ha_innodb.cc: Cannot assign const_iterator to iterator	9 years ago
Vicențiu Ciorbaru	360a4a0372	5.6.36-82.0	9 years ago
Sergei Golubchik	71b4503242	MDEV-9998 Fix issues caught by Clang's -Wpointer-bool-conversion warning remove useless checks and a couple of others	9 years ago
Sergei Golubchik	c0e24cd0e8	compiler warnings	9 years ago
Vladislav Vaintroub	9c4b7cad27	MDEV-9566 Prepare xtradb for xtrabackup These changes are comparable to Percona's modifications in innodb in the Percona Xtrabackup repository. - If functions are used in backup as well as in innodb, make them non-static. - Define IS_XTRABACKUP() macro for special handling of innodb running inside backup. - Extend some functions for backup. fil_space_for_table_exists_in_mem() gets additional parameter 'remove_from_data_dict_if_does_not_exist', for partial backups fil_load_single_table_tablespaces() gets an optional parameter predicate which tells whether to load tablespace based on database or table name, also for partial backups. srv_undo_tablespaces_init() gets an optional parameter 'backup_mode' - Allow single redo log file (for backup "prepare") - Do not read doublewrite buffer pages in backup, they are outdated - Add function fil_remove_invalid_table_from_data_dict(), to remove non-existing tables from data dictionary in case of partial backups. - On Windows, fix file share modes when opening tablespaces, to allow mariabackup to read tablespaces while server is online. - Avoid access to THDVARs in backup, because innodb plugin is not loaded, and THDVAR would crash in this case.	9 years ago
Jan Lindström	765a43605a	MDEV-12253: Buffer pool blocks are accessed after they have been freed Problem was that bpage was referenced after it was already freed from LRU. Fixed by adding a new variable encrypted that is passed down to buf_page_check_corrupt() and used in buf_page_get_gen() to stop processing page read. This patch should also address following test failures and bugs: MDEV-12419: IMPORT should not look up tablespace in PageConverter::validate(). This is now removed. MDEV-10099: encryption.innodb_onlinealter_encryption fails sporadically in buildbot MDEV-11420: encryption.innodb_encryption-page-compression failed in buildbot MDEV-11222: encryption.encrypt_and_grep failed in buildbot on P8 Removed dict_table_t::is_encrypted and dict_table_t::ibd_file_missing and replaced these with dict_table_t::file_unreadable. Table ibd file is missing if fil_get_space(space_id) returns NULL and encrypted if not. Removed dict_table_t::is_corrupted field. Ported FilSpace class from 10.2 and using that on buf_page_check_corrupt(), buf_page_decrypt_after_read(), buf_page_encrypt_before_write(), buf_dblwr_process(), buf_read_page(), dict_stats_save_defrag_stats(). Added test cases when enrypted page could be read while doing redo log crash recovery. Also added test case for row compressed blobs. btr_cur_open_at_index_side_func(), btr_cur_open_at_rnd_pos_func(): Avoid referencing block that is NULL. buf_page_get_zip(): Issue error if page read fails. buf_page_get_gen(): Use dberr_t for error detection and do not reference bpage after we hare freed it. buf_mark_space_corrupt(): remove bpage from LRU also when it is encrypted. buf_page_check_corrupt(): @return DB_SUCCESS if page has been read and is not corrupted, DB_PAGE_CORRUPTED if page based on checksum check is corrupted, DB_DECRYPTION_FAILED if page post encryption checksum matches but after decryption normal page checksum does not match. In read case only DB_SUCCESS is possible. buf_page_io_complete(): use dberr_t for error handling. buf_flush_write_block_low(), buf_read_ahead_random(), buf_read_page_async(), buf_read_ahead_linear(), buf_read_ibuf_merge_pages(), buf_read_recv_pages(), fil_aio_wait(): Issue error if page read fails. btr_pcur_move_to_next_page(): Do not reference page if it is NULL. Introduced dict_table_t::is_readable() and dict_index_t::is_readable() that will return true if tablespace exists and pages read from tablespace are not corrupted or page decryption failed. Removed buf_page_t::key_version. After page decryption the key version is not removed from page frame. For unencrypted pages, old key_version is removed at buf_page_encrypt_before_write() dict_stats_update_transient_for_index(), dict_stats_update_transient() Do not continue if table decryption failed or table is corrupted. dict0stats.cc: Introduced a dict_stats_report_error function to avoid code duplication. fil_parse_write_crypt_data(): Check that key read from redo log entry is found from encryption plugin and if it is not, refuse to start. PageConverter::validate(): Removed access to fil_space_t as tablespace is not available during import. Fixed error code on innodb.innodb test. Merged test cased innodb-bad-key-change5 and innodb-bad-key-shutdown to innodb-bad-key-change2. Removed innodb-bad-key-change5 test. Decreased unnecessary complexity on some long lasting tests. Removed fil_inc_pending_ops(), fil_decr_pending_ops(), fil_get_first_space(), fil_get_next_space(), fil_get_first_space_safe(), fil_get_next_space_safe() functions. fil_space_verify_crypt_checksum(): Fixed bug found using ASAN where FIL_PAGE_END_LSN_OLD_CHECKSUM field was incorrectly accessed from row compressed tables. Fixed out of page frame bug for row compressed tables in fil_space_verify_crypt_checksum() found using ASAN. Incorrect function was called for compressed table. Added new tests for discard, rename table and drop (we should allow them even when page decryption fails). Alter table rename is not allowed. Added test for restart with innodb-force-recovery=1 when page read on redo-recovery cant be decrypted. Added test for corrupted table where both page data and FIL_PAGE_FILE_FLUSH_LSN_OR_KEY_VERSION is corrupted. Adjusted the test case innodb_bug14147491 so that it does not anymore expect crash. Instead table is just mostly not usable. fil0fil.h: fil_space_acquire_low is not visible function and fil_space_acquire and fil_space_acquire_silent are inline functions. FilSpace class uses fil_space_acquire_low directly. recv_apply_hashed_log_recs() does not return anything.	9 years ago
Marko Mäkelä	25d69ea012	MDEV-12198 innodb_defragment=1 crashes server on OPTIMIZE TABLE when FULLTEXT index exists ha_innobase::defragment_table(): Skip corrupted indexes and FULLTEXT INDEX. In InnoDB, FULLTEXT INDEX is implemented with auxiliary tables. We will not defragment them on OPTIMIZE TABLE.	9 years ago
Vladislav Vaintroub	d7c35a9992	Fix compiler error	9 years ago
Vladislav Vaintroub	e5b67a46bc	MDEV-12345 Performance : replace calls to clock() inside trx_start_low() by THD::start_utime	9 years ago
Jan Lindström	50eb40a2a8	MDEV-11738: Mariadb uses 100% of several of my 8 cpus doing nothing MDEV-11581: Mariadb starts InnoDB encryption threads when key has not changed or data scrubbing turned off Background: Key rotation is based on background threads (innodb-encryption-threads) periodically going through all tablespaces on fil_system. For each tablespace current used key version is compared to max key age (innodb-encryption-rotate-key-age). This process naturally takes CPU. Similarly, in same time need for scrubbing is investigated. Currently, key rotation is fully supported on Amazon AWS key management plugin only but InnoDB does not have knowledge what key management plugin is used. This patch re-purposes innodb-encryption-rotate-key-age=0 to disable key rotation and background data scrubbing. All new tables are added to special list for key rotation and key rotation is based on sending a event to background encryption threads instead of using periodic checking (i.e. timeout). fil0fil.cc: Added functions fil_space_acquire_low() to acquire a tablespace when it could be dropped concurrently. This function is used from fil_space_acquire() or fil_space_acquire_silent() that will not print any messages if we try to acquire space that does not exist. fil_space_release() to release a acquired tablespace. fil_space_next() to iterate tablespaces in fil_system using fil_space_acquire() and fil_space_release(). Similarly, fil_space_keyrotation_next() to iterate new list fil_system->rotation_list where new tables. are added if key rotation is disabled. Removed unnecessary functions fil_get_first_space_safe() fil_get_next_space_safe() fil_node_open_file(): After page 0 is read read also crypt_info if it is not yet read. btr_scrub_lock_dict_func() buf_page_check_corrupt() buf_page_encrypt_before_write() buf_merge_or_delete_for_page() lock_print_info_all_transactions() row_fts_psort_info_init() row_truncate_table_for_mysql() row_drop_table_for_mysql() Use fil_space_acquire()/release() to access fil_space_t. buf_page_decrypt_after_read(): Use fil_space_get_crypt_data() because at this point we might not yet have read page 0. fil0crypt.cc/fil0fil.h: Lot of changes. Pass fil_space_t* directly to functions needing it and store fil_space_t* to rotation state. Use fil_space_acquire()/release() when iterating tablespaces and removed unnecessary is_closing from fil_crypt_t. Use fil_space_t::is_stopping() to detect when access to tablespace should be stopped. Removed unnecessary fil_space_get_crypt_data(). fil_space_create(): Inform key rotation that there could be something to do if key rotation is disabled and new table with encryption enabled is created. Remove unnecessary functions fil_get_first_space_safe() and fil_get_next_space_safe(). fil_space_acquire() and fil_space_release() are used instead. Moved fil_space_get_crypt_data() and fil_space_set_crypt_data() to fil0crypt.cc. fsp_header_init(): Acquire fil_space_t, write crypt_data and release space. check_table_options() Renamed FIL_SPACE_ENCRYPTION_ TO FIL_ENCRYPTION_* i_s.cc: Added ROTATING_OR_FLUSHING field to information_schema.innodb_tablespace_encryption to show current status of key rotation.	9 years ago
Marko Mäkelä	5aacb861f2	WSREP: Use TRX_ID_FMT for trx_id_t in messages.	9 years ago
Vicențiu Ciorbaru	8d69ce7b82	5.6.35-80.0	9 years ago
Nirbhay Choubey	3435e8a515	MDEV-7635: Part 1 innodb_autoinc_lock_mode = 2 innodb_buffer_pool_dump_at_shutdown = ON innodb_buffer_pool_dump_pct = 25 innodb_buffer_pool_load_at_startup = ON innodb_checksum_algorithm = CRC32 innodb_file_format = Barracuda innodb_large_prefix = ON innodb_log_compressed_pages = ON innodb_purge_threads = 4 innodb_strict_mode = ON binlog_annotate_row_events = ON binlog_format = MIXED binlog-row-event-max-size = 8192 group_concat_max_len = 1M lock_wait_timeout = 86400 log_slow_admin_statements = ON log_slow_slave_statements = ON log_warnings = 2 max_allowed_packet = 16M replicate_annotate_row_events = ON slave_net_timeout = 60 sync_binlog = 1 aria_recover = BACKUP,QUICK myisam_recover_options = BACKUP,QUICK	9 years ago
Marko Mäkelä	f162704570	Rewrite the innodb.log_file_size test with DBUG_EXECUTE_IF. Remove the debug parameter innodb_force_recovery_crash that was introduced into MySQL 5.6 by me in WL#6494 which allowed InnoDB to resize the redo log on startup. Let innodb.log_file_size actually start up the server, but ensure that the InnoDB storage engine refuses to start up in each of the scenarios.	9 years ago
Jan Lindström	17cc619847	MDEV-11671: Duplicated [NOTE] output for changed innodb_page_size Remove duplicated output and change output level to info.	9 years ago
Marko Mäkelä	732672c304	MDEV-11233 CREATE FULLTEXT INDEX with a token longer than 127 bytes crashes server This bug is the result of merging the Oracle MySQL follow-up fix BUG#22963169 MYSQL CRASHES ON CREATE FULLTEXT INDEX without merging the base bug fix: Bug#79475 Insert a token of 84 4-bytes chars into fts index causes server crash. Unlike the above mentioned fixes in MySQL, our fix will not change the storage format of fulltext indexes in InnoDB or XtraDB when a character encoding with mbmaxlen=2 or mbmaxlen=3 and the length of a word is between 128 and 84mbmaxlen bytes. The Oracle fix would allocate 2 length bytes for these cases. Compatibility with other MySQL and MariaDB releases is ensured by persisting the used maximum length in the SYS_COLUMNS table in the InnoDB data dictionary. This fix also removes some unnecessary strcmp() calls when checking for the legacy default collation my_charset_latin1 (my_charset_latin1.name=="latin1_swedish_ci"). fts_create_one_index_table(): Store the actual length in bytes. This metadata will be written to the SYS_COLUMNS table. fts_zip_initialize(): Initialize only the first byte of the buffer. Actually the code should not even care about this first byte, because the length is set as 0. FTX_MAX_WORD_LEN: Define as HA_FT_MAXCHARLEN 4 aka 336 bytes, not as 254 bytes. row_merge_create_fts_sort_index(): Set the actual maximum length of the column in bytes, similar to fts_create_one_index_table(). row_merge_fts_doc_tokenize(): Remove the redundant parameter word_dtype. Use the actual maximum length of the column. Calculate the extra_size in the same way as row_merge_buf_encode() does.	9 years ago
Jan Lindström	8a4d605500	MDEV-11838: Innodb-encryption-algorithm default should be != none Change default to zlib, this has effect only if user has explicitly requested page compression and then user naturally expects that pages are really compressed if they can be compressed.	9 years ago
Nirbhay Choubey	5fc1ba604e	Fix for post-merge build failure.	9 years ago
Marko Mäkelä	384f4d1e36	Post-push fix for MDEV-11556: Make the debug variable UINT. Sometimes innodb_data_file_size_debug was reported as INT UNSIGNED instead of BIGINT UNSIGNED. Make it uint instead of ulong to get a more deterministic result.	9 years ago
Marko Mäkelä	719321e78e	MDEV-11638 Encryption causes race conditions in InnoDB shutdown InnoDB shutdown failed to properly take fil_crypt_thread() into account. The encryption threads were signalled to shut down together with other non-critical tasks. This could be much too early in case of slow shutdown, which could need minutes to complete the purge. Furthermore, InnoDB failed to wait for the fil_crypt_thread() to actually exit before proceeding to the final steps of shutdown, causing the race conditions. Furthermore, the log_scrub_thread() was shut down way too early. Also it should remain until the SRV_SHUTDOWN_FLUSH_PHASE. fil_crypt_threads_end(): Remove. This would cause the threads to be terminated way too early. srv_buf_dump_thread_active, srv_dict_stats_thread_active, lock_sys->timeout_thread_active, log_scrub_thread_active, srv_monitor_active, srv_error_monitor_active: Remove a race condition between startup and shutdown, by setting these in the startup thread that creates threads, not in each created thread. In this way, once the flag is cleared, it will remain cleared during shutdown. srv_n_fil_crypt_threads_started, fil_crypt_threads_event: Declare in global rather than static scope. log_scrub_event, srv_log_scrub_thread_active, log_scrub_thread(): Declare in static rather than global scope. Let these be created by log_init() and freed by log_shutdown(). rotate_thread_t::should_shutdown(): Do not shut down before the SRV_SHUTDOWN_FLUSH_PHASE. srv_any_background_threads_are_active(): Remove. These checks now exist in logs_empty_and_mark_files_at_shutdown(). logs_empty_and_mark_files_at_shutdown(): Shut down the threads in the proper order. Keep fil_crypt_thread() and log_scrub_thread() alive until SRV_SHUTDOWN_FLUSH_PHASE, and check that they actually terminate.	9 years ago
Marko Mäkelä	80d5d1452a	MDEV-11694 InnoDB tries to create unused table SYS_ZIP_DICT MariaDB Server 10.0.28 and 10.1.19 merged code from Percona XtraDB that introduced support for compressed columns. Much but not all of this code was disabled by placing #ifdef HAVE_PERCONA_COMPRESSED_COLUMNS around it. Among the unused but not disabled code is code to access some new system tables related to compressed columns. The creation of these system tables SYS_ZIP_DICT and SYS_ZIP_DICT_COLS would cause a crash in --innodb-read-only mode when upgrading from an earlier version to 10.0.28 or 10.1.19. Let us remove all the dead code related to compressed columns. Users who already upgraded to 10.0.28 and 10.1.19 will have the two above mentioned empty tables in their InnoDB system tablespace. Subsequent versions of MariaDB Server will completely ignore those tables.	9 years ago
Sachin Setiya	b4616c40be	MDEV-7955 WSREP() appears on radar in OLTP RO This commit is for optimizing WSREP(thd) macro. #define WSREP(thd) \ (WSREP_ON && wsrep && (thd && thd->variables.wsrep_on)) In this we can safely remove wsrep and thd. We are not removing WSREP_ON because this will change WSREP(thd) behaviour. Patch Credit:- Nirbhay Choubay, Sergey Vojtovich	9 years ago
Marko Mäkelä	8451e09073	MDEV-11556 InnoDB redo log apply fails to adjust data file sizes fil_space_t::recv_size: New member: recovered tablespace size in pages; 0 if no size change was read from the redo log, or if the size change was implemented. fil_space_set_recv_size(): New function for setting space->recv_size. innodb_data_file_size_debug: A debug parameter for setting the system tablespace size in recovery even when the redo log does not contain any size changes. It is hard to write a small test case that would cause the system tablespace to be extended at the critical moment. recv_parse_log_rec(): Note those tablespaces whose size is being changed by the redo log, by invoking fil_space_set_recv_size(). innobase_init(): Correct an error message, and do not require a larger innodb_buffer_pool_size when starting up with a smaller innodb_page_size. innobase_start_or_create_for_mysql(): Allow startup with any initial size of the ibdata1 file if the autoextend attribute is set. Require the minimum size of fixed-size system tablespaces to be 640 pages, not 10 megabytes. Implement innodb_data_file_size_debug. open_or_create_data_files(): Round the system tablespace size down to pages, not to full megabytes, (Our test truncates the system tablespace to more than 800 pages with innodb_page_size=4k. InnoDB should not imagine that it was truncated to 768 pages and then overwrite good pages in the tablespace.) fil_flush_low(): Refactored from fil_flush(). fil_space_extend_must_retry(): Refactored from fil_extend_space_to_desired_size(). fil_mutex_enter_and_prepare_for_io(): Extend the tablespace if fil_space_set_recv_size() was called. The test case has been successfully run with all the innodb_page_size values 4k, 8k, 16k, 32k, 64k.	9 years ago

1 2 3 4 5 ...

684 Commits (7c85a8d936b1a8bb82ff069f82f5ecf3ff2ab654)