mariadb

Commit Graph

Author	SHA1	Message	Date
Eugene Kosov	14be814380	MDEV-17491 micro optimize page_id_t page_id_t: remove m_fold member various places: pass page_id_t by value instead of by reference	7 years ago
Marko Mäkelä	2ca904f0ca	MDEV-13103 Deal with page_compressed page corruption fil_page_decompress(): Replaces fil_decompress_page(). Allow the caller detect errors. Remove duplicated code. Use the "safe" instead of "fast" variants of decompression routines. fil_page_compress(): Replaces fil_compress_page(). The length of the input buffer always was srv_page_size (innodb_page_size). Remove printouts, and remove the fil_space_t* parameter. buf_tmp_buffer_t::reserved: Make private; the accessors acquire() and release() will use atomic memory access. buf_pool_reserve_tmp_slot(): Make static. Remove the second parameter. Do not acquire any mutex. Remove the allocation of the buffers. buf_tmp_reserve_crypt_buf(), buf_tmp_reserve_compression_buf(): Refactored away from buf_pool_reserve_tmp_slot(). buf_page_decrypt_after_read(): Make static, and simplify the logic. Use the encryption buffer also for decompressing. buf_page_io_complete(), buf_dblwr_process(): Check more failures. fil_space_encrypt(): Simplify the debug checks. fil_space_t::printed_compression_failure: Remove. fil_get_compression_alg_name(): Remove. fil_iterate(): Allocate a buffer for compression and decompression only once, instead of allocating and freeing it for every page that uses compression, during IMPORT TABLESPACE. Also, validate the page checksum before decryption, and reduce the scope of some variables. fil_page_is_index_page(), fil_page_is_lzo_compressed(): Remove (unused). AbstractCallback::operator()(): Remove the parameter 'offset'. The check for it in FetchIndexRootPages::operator() was basically redundant and dead code since the previous refactoring.	7 years ago
Marko Mäkelä	f5eb37129f	MDEV-13103 Deal with page_compressed page corruption fil_page_decompress(): Replaces fil_decompress_page(). Allow the caller detect errors. Remove duplicated code. Use the "safe" instead of "fast" variants of decompression routines. fil_page_compress(): Replaces fil_compress_page(). The length of the input buffer always was srv_page_size (innodb_page_size). Remove printouts, and remove the fil_space_t* parameter. buf_tmp_buffer_t::reserved: Make private; the accessors acquire() and release() will use atomic memory access. buf_pool_reserve_tmp_slot(): Make static. Remove the second parameter. Do not acquire any mutex. Remove the allocation of the buffers. buf_tmp_reserve_crypt_buf(), buf_tmp_reserve_compression_buf(): Refactored away from buf_pool_reserve_tmp_slot(). buf_page_decrypt_after_read(): Make static, and simplify the logic. Use the encryption buffer also for decompressing. buf_page_io_complete(), buf_dblwr_process(): Check more failures. fil_space_encrypt(): Simplify the debug checks. fil_space_t::printed_compression_failure: Remove. fil_get_compression_alg_name(): Remove. fil_iterate(): Allocate a buffer for compression and decompression only once, instead of allocating and freeing it for every page that uses compression, during IMPORT TABLESPACE. fil_node_get_space_id(), fil_page_is_index_page(), fil_page_is_lzo_compressed(): Remove (unused code).	7 years ago
Marko Mäkelä	112df06996	MDEV-15529 IMPORT TABLESPACE unnecessarily uses the doublewrite buffer fil_space_t::atomic_write_supported: Always set this flag for TEMPORARY TABLESPACE and during IMPORT TABLESPACE. The page writes during these operations are by definition not crash-safe because they are not written to the redo log. fil_space_t::use_doublewrite(): Determine if doublewrite should be used. buf_dblwr_update(): Add assertions, and let the caller check whether doublewrite buffering is desired. buf_flush_write_block_low(): Disable the doublewrite buffer for the temporary tablespace and for IMPORT TABLESPACE. fil_space_set_imported(), fil_node_open_file(), fil_space_create(): Initialize or revise the space->atomic_write_supported flag. buf_page_io_complete(), buf_flush_write_complete(): Add the parameter dblwr, to indicate whether doublewrite was used for writes. buf_dblwr_sync_datafiles(): Remove an unnecessary flush of persistent tablespaces when flushing temporary tablespaces. (Move the call to buf_dblwr_flush_buffered_writes().)	8 years ago
Vladislav Vaintroub	ac8e3c85a4	MDEV-15295 Type mismatch for srv_fatal_semaphore_wait_threshold	8 years ago
Marko Mäkelä	51b4366bfb	MDEV-13328 ALTER TABLE…DISCARD TABLESPACE takes a lot of time With a big buffer pool that contains many data pages, DISCARD TABLESPACE took a long time, because it would scan the entire buffer pool to remove any pages that belong to the tablespace. With a large buffer pool, this would take a lot of time, especially when the table-to-discard is empty. The minimum amount of work that DISCARD TABLESPACE must do is to remove the pages of the to-be-discarded table from the buf_pool->flush_list because any writes to the data file must be prevented before the file is deleted. If DISCARD TABLESPACE does not evict the pages from the buffer pool, then IMPORT TABLESPACE must do it, because we must prevent pre-DISCARD, not-yet-evicted pages from being mistaken for pages of the imported tablespace. It would not be a useful fix to simply move the buffer pool scan to the IMPORT TABLESPACE step. What we can do is to actively evict those pages that could be mistaken for imported pages. In this way, when importing a small table into a big buffer pool, the import should still run relatively fast. Import is bypassing the buffer pool when reading pages for the adjustment phase. In the adjustment phase, if a page exists in the buffer pool, we could replace it with the page from the imported file. Unfortunately I did not get this to work properly, so instead we will simply evict any matching page from the buffer pool. buf_page_get_gen(): Implement BUF_EVICT_IF_IN_POOL, a new mode where the requested page will be evicted if it is found. There must be no unwritten changes for the page. buf_remove_t: Remove. Instead, use trx!=NULL to signify that a write to file is desired, and use a separate parameter bool drop_ahi. buf_LRU_flush_or_remove_pages(), fil_delete_tablespace(): Replace buf_remove_t. buf_LRU_remove_pages(), buf_LRU_remove_all_pages(): Remove. PageConverter::m_mtr: A dummy mini-transaction buffer PageConverter::PageConverter(): Complete the member initialization list. PageConverter::operator()(): Evict any 'shadow' pages from the buffer pool so that pre-existing (garbage) pages cannot be mistaken for pages that exist in the being-imported file. row_discard_tablespace(): Remove a bogus comment that seems to refer to IMPORT TABLESPACE, not DISCARD TABLESPACE.	8 years ago
Marko Mäkelä	6b45355e6b	MDEV-13103 Assertion `flags & BUF_PAGE_PRINT_NO_CRASH' failed in buf_page_print buf_page_print(): Remove the parameter 'flags', and when a server abort is intended, perform that in the caller. In this way, page corruption reports due to different reasons can be distinguished better. This is non-functional code refactoring that does not fix any page corruption issues. The change is only made to avoid falsely grouping together unrelated causes of page corruption.	8 years ago
Jan Lindström	34eef269eb	MDEV-11939: innochecksum mistakes a file for an encrypted one (page 0 invalid) Always read full page 0 to determine does tablespace contain encryption metadata. Tablespaces that are page compressed or page compressed and encrypted do not compare checksum as it does not exists. For encrypted tables use checksum verification written for encrypted tables and normal tables use normal method. buf_page_is_checksum_valid_crc32 buf_page_is_checksum_valid_innodb buf_page_is_checksum_valid_none Modify Innochecksum logging to file to avoid compilation warnings. fil0crypt.cc fil0crypt.h Modify to be able to use in innochecksum compilation and move fil_space_verify_crypt_checksum to end of the file. Add innochecksum logging to file. univ.i Add innochecksum strict_verify, log_file and cur_page_num variables as extern. page_zip_verify_checksum Add innochecksum logging to file and remove unnecessary code. innochecksum.cc Lot of changes most notable able to read encryption metadata from page 0 of the tablespace. Added test case where we corrupt intentionally FIL_PAGE_FILE_FLUSH_LSN_OR_KEY_VERSION (encryption key version) FIL_PAGE_FILE_FLUSH_LSN_OR_KEY_VERSION+4 (post encryption checksum) FIL_DATA+10 (data)	8 years ago
Jan Lindström	8b019f87dd	MDEV-11939: innochecksum mistakes a file for an encrypted one (page 0 invalid) Always read full page 0 to determine does tablespace contain encryption metadata. Tablespaces that are page compressed or page compressed and encrypted do not compare checksum as it does not exists. For encrypted tables use checksum verification written for encrypted tables and normal tables use normal method. buf_page_is_checksum_valid_crc32 buf_page_is_checksum_valid_innodb buf_page_is_checksum_valid_none Add Innochecksum logging to file buf_page_is_corrupted Remove ib_logf and page_warn_strict_checksum calls in innochecksum compilation. Add innochecksum logging to file. fil0crypt.cc fil0crypt.h Modify to be able to use in innochecksum compilation and move fil_space_verify_crypt_checksum to end of the file. Add innochecksum logging to file. univ.i Add innochecksum strict_verify, log_file and cur_page_num variables as extern. page_zip_verify_checksum Add innochecksum logging to file. innochecksum.cc Lot of changes most notable able to read encryption metadata from page 0 of the tablespace. Added test case where we corrupt intentionally FIL_PAGE_FILE_FLUSH_LSN_OR_KEY_VERSION (encryption key version) FIL_PAGE_FILE_FLUSH_LSN_OR_KEY_VERSION+4 (post encryption checksum) FIL_DATA+10 (data)	8 years ago
Jan Lindström	90c52e5291	MDEV-12615: InnoDB page compression method snappy mostly does not compress pages Snappy compression method require that output buffer used for compression is bigger than input buffer. Similarly lzo require additional work memory buffer. Increase the allocated buffer accordingly. buf_tmp_buffer_t: removed unnecessary lzo_mem, crypt_buf_free and comp_buf_free. buf_pool_reserve_tmp_slot: use alligned_alloc and if snappy available allocate size based on snappy_max_compressed_length and if lzo is available increase buffer by LZO1X_1_15_MEM_COMPRESS. fil_compress_page: Remove unneeded lzo mem (we use same buffer) and if output buffer is not yet allocated allocate based similarly as above. Decompression does not require additional work area. Modify test to use same test as other compression method tests.	9 years ago
Sergei Golubchik	0072d2e9a1	InnoDB cleanup: remove a bunch of #ifdef UNIV_INNOCHECKSUM innochecksum uses global variables. great, let's use them all the way down, instead of passing them as arguments to innodb internals, conditionally modifying function prototypes with #ifdefs	9 years ago
Marko Mäkelä	b82c602db5	MDEV-12602 InnoDB: Failing assertion: space->n_pending_ops == 0 This fixes a regression caused by MDEV-12428. When we introduced a variant of fil_space_acquire() that could increment space->n_pending_ops after space->stop_new_ops was set, the logic of fil_check_pending_operations() was broken. fil_space_t::n_pending_ios: A new field to track read or write access from the buffer pool routines immediately before a block write or after a block read in the file system. fil_space_acquire_for_io(), fil_space_release_for_io(): Similar to fil_space_acquire_silent() and fil_space_release(), but modify fil_space_t::n_pending_ios instead of fil_space_t::n_pending_ops. Adjust a number of places accordingly, and remove some redundant tablespace lookups. The following parts of this fix differ from the 10.2 version of this fix: buf_page_get_corrupt(): Add a tablespace parameter. In 10.2, we already had a two-phase process of freeing fil_space objects (first, fil_space_detach(), then release fil_system->mutex, and finally free the fil_space and fil_node objects). fil_space_free_and_mutex_exit(): Renamed from fil_space_free(). Detach the tablespace from the fil_system cache, release the fil_system->mutex, and then wait for space->n_pending_ios to reach 0, to avoid accessing freed data in a concurrent thread. During the wait, future calls to fil_space_acquire_for_io() will not find this tablespace, and the count can only be decremented to 0, at which point it is safe to free the objects. fil_node_free_part1(), fil_node_free_part2(): Refactored from fil_node_free().	9 years ago
Marko Mäkelä	4b24467ff3	MDEV-12602 InnoDB: Failing assertion: space->n_pending_ops == 0 This fixes a regression caused by MDEV-12428. When we introduced a variant of fil_space_acquire() that could increment space->n_pending_ops after space->stop_new_ops was set, the logic of fil_check_pending_operations() was broken. fil_space_t::n_pending_ios: A new field to track read or write access from the buffer pool routines immediately before a block write or after a block read in the file system. fil_space_acquire_for_io(), fil_space_release_for_io(): Similar to fil_space_acquire_silent() and fil_space_release(), but modify fil_space_t::n_pending_ios instead of fil_space_t::n_pending_ops. fil_space_free_low(): Wait for space->n_pending_ios to reach 0, to avoid accessing freed data in a concurrent thread. Future calls to fil_space_acquire_for_io() will not find this tablespace, because it will already have been detached from fil_system. Adjust a number of places accordingly, and remove some redundant tablespace lookups. FIXME: buf_page_check_corrupt() should take a tablespace from fil_space_acquire_for_io() as a parameter. This will be done in the 10.1 version of this patch and merged from there. That depends on MDEV-12253, which has not been merged from 10.1 yet.	9 years ago
Marko Mäkelä	472b5f0d1f	Follow-up to Bug#24346574 PAGE CLEANER THREAD, ASSERT BLOCK->N_POINTERS == 0 Silence the Valgrind warnings on instrumented builds (-DWITH_VALGRIND). assert_block_ahi_empty_on_init(): A variant of assert_block_ahi_empty() that declares n_pointers initialized and then asserts that n_pointers==0. In Valgrind-instrumented builds, InnoDB declares allocated memory uninitialized.	9 years ago
Marko Mäkelä	e63ead68bf	Bug#24346574 PAGE CLEANER THREAD, ASSERT BLOCK->N_POINTERS == 0 btr_search_drop_page_hash_index(): Do not return before ensuring that block->index=NULL, even if !btr_search_enabled. We would typically still skip acquiring the AHI latch when the AHI is disabled, because block->index would already be NULL. Only if the AHI is in the process of being disabled, we would wait for the AHI latch and then notice that block->index=NULL and return. The above bug was a regression caused in MySQL 5.7.9 by the fix of Bug#21407023: DISABLING AHI SHOULD AVOID TAKING AHI LATCH The rest of this patch improves diagnostics by adding assertions. assert_block_ahi_valid(): A debug predicate for checking that block->n_pointers!=0 implies block->index!=NULL. assert_block_ahi_empty(): A debug predicate for checking that block->n_pointers==0. buf_block_init(): Instead of assigning block->n_pointers=0, assert_block_ahi_empty(block). buf_pool_clear_hash_index(): Clarify comments, and assign block->n_pointers=0 before assigning block->index=NULL. The wrong ordering could make block->n_pointers appear incorrect in debug assertions. This bug was introduced in MySQL 5.1.52 by Bug#13006367 62487: INNODB TAKES 3 MINUTES TO CLEAN UP THE ADAPTIVE HASH INDEX AT SHUTDOWN i_s_innodb_buffer_page_get_info(): Add a comment that the IS_HASHED column in the INFORMATION_SCHEMA views INNODB_BUFFER_POOL_PAGE and INNODB_BUFFER_PAGE_LRU may show false positives (there may be no pointers after all.) ha_insert_for_fold_func(), ha_delete_hash_node(), ha_search_and_update_if_found_func(): Use atomics for updating buf_block_t::n_pointers. While buf_block_t::index is always protected by btr_search_x_lock(index), in ha_insert_for_fold_func() the n_pointers-- may belong to another dict_index_t whose btr_search_latches[] we are not holding. RB: 13879 Reviewed-by: Jimmy Yang <jimmy.yang@oracle.com>	9 years ago
Jan Lindström	765a43605a	MDEV-12253: Buffer pool blocks are accessed after they have been freed Problem was that bpage was referenced after it was already freed from LRU. Fixed by adding a new variable encrypted that is passed down to buf_page_check_corrupt() and used in buf_page_get_gen() to stop processing page read. This patch should also address following test failures and bugs: MDEV-12419: IMPORT should not look up tablespace in PageConverter::validate(). This is now removed. MDEV-10099: encryption.innodb_onlinealter_encryption fails sporadically in buildbot MDEV-11420: encryption.innodb_encryption-page-compression failed in buildbot MDEV-11222: encryption.encrypt_and_grep failed in buildbot on P8 Removed dict_table_t::is_encrypted and dict_table_t::ibd_file_missing and replaced these with dict_table_t::file_unreadable. Table ibd file is missing if fil_get_space(space_id) returns NULL and encrypted if not. Removed dict_table_t::is_corrupted field. Ported FilSpace class from 10.2 and using that on buf_page_check_corrupt(), buf_page_decrypt_after_read(), buf_page_encrypt_before_write(), buf_dblwr_process(), buf_read_page(), dict_stats_save_defrag_stats(). Added test cases when enrypted page could be read while doing redo log crash recovery. Also added test case for row compressed blobs. btr_cur_open_at_index_side_func(), btr_cur_open_at_rnd_pos_func(): Avoid referencing block that is NULL. buf_page_get_zip(): Issue error if page read fails. buf_page_get_gen(): Use dberr_t for error detection and do not reference bpage after we hare freed it. buf_mark_space_corrupt(): remove bpage from LRU also when it is encrypted. buf_page_check_corrupt(): @return DB_SUCCESS if page has been read and is not corrupted, DB_PAGE_CORRUPTED if page based on checksum check is corrupted, DB_DECRYPTION_FAILED if page post encryption checksum matches but after decryption normal page checksum does not match. In read case only DB_SUCCESS is possible. buf_page_io_complete(): use dberr_t for error handling. buf_flush_write_block_low(), buf_read_ahead_random(), buf_read_page_async(), buf_read_ahead_linear(), buf_read_ibuf_merge_pages(), buf_read_recv_pages(), fil_aio_wait(): Issue error if page read fails. btr_pcur_move_to_next_page(): Do not reference page if it is NULL. Introduced dict_table_t::is_readable() and dict_index_t::is_readable() that will return true if tablespace exists and pages read from tablespace are not corrupted or page decryption failed. Removed buf_page_t::key_version. After page decryption the key version is not removed from page frame. For unencrypted pages, old key_version is removed at buf_page_encrypt_before_write() dict_stats_update_transient_for_index(), dict_stats_update_transient() Do not continue if table decryption failed or table is corrupted. dict0stats.cc: Introduced a dict_stats_report_error function to avoid code duplication. fil_parse_write_crypt_data(): Check that key read from redo log entry is found from encryption plugin and if it is not, refuse to start. PageConverter::validate(): Removed access to fil_space_t as tablespace is not available during import. Fixed error code on innodb.innodb test. Merged test cased innodb-bad-key-change5 and innodb-bad-key-shutdown to innodb-bad-key-change2. Removed innodb-bad-key-change5 test. Decreased unnecessary complexity on some long lasting tests. Removed fil_inc_pending_ops(), fil_decr_pending_ops(), fil_get_first_space(), fil_get_next_space(), fil_get_first_space_safe(), fil_get_next_space_safe() functions. fil_space_verify_crypt_checksum(): Fixed bug found using ASAN where FIL_PAGE_END_LSN_OLD_CHECKSUM field was incorrectly accessed from row compressed tables. Fixed out of page frame bug for row compressed tables in fil_space_verify_crypt_checksum() found using ASAN. Incorrect function was called for compressed table. Added new tests for discard, rename table and drop (we should allow them even when page decryption fails). Alter table rename is not allowed. Added test for restart with innodb-force-recovery=1 when page read on redo-recovery cant be decrypted. Added test for corrupted table where both page data and FIL_PAGE_FILE_FLUSH_LSN_OR_KEY_VERSION is corrupted. Adjusted the test case innodb_bug14147491 so that it does not anymore expect crash. Instead table is just mostly not usable. fil0fil.h: fil_space_acquire_low is not visible function and fil_space_acquire and fil_space_acquire_silent are inline functions. FilSpace class uses fil_space_acquire_low directly. recv_apply_hashed_log_recs() does not return anything.	9 years ago
Marko Mäkelä	0871a00a62	MDEV-12545 Reduce the amount of fil_space_t lookups buf_flush_write_block_low(): Acquire the tablespace reference once, and pass it to lower-level functions. This is only a start; further calls may be removed. fil_decompress_page(): Remove unsafe use of fil_space_get_by_id().	9 years ago
Marko Mäkelä	996c7d5cb5	MDEV-12545 Reduce the amount of fil_space_t lookups buf_flush_write_block_low(): Acquire the tablespace reference once, and pass it to lower-level functions. This is only a start; further calls may be removed later.	9 years ago
Marko Mäkelä	97acc4a1c3	MDEV-12270 Port MySQL 8.0 Bug#21141390 REMOVE UNUSED FUNCTIONS AND CONVERT GLOBAL SYMBOLS TO STATIC InnoDB defines some functions that are not called at all. Other functions are called, but only from the same compilation unit. Remove some function declarations and definitions, and add 'static' keywords. Some symbols must be kept for separately compiled tools, such as innochecksum.	11 years ago
Marko Mäkelä	4e1116b2c6	MDEV-12271 Port MySQL 8.0 Bug#23150562 REMOVE UNIV_MUST_NOT_INLINE AND UNIV_NONINL Also, remove empty .ic files that were not removed by my MySQL commit. Problem: InnoDB used to support a compilation mode that allowed to choose whether the function definitions in .ic files are to be inlined or not. This stopped making sense when InnoDB moved to C++ in MySQL 5.6 (and ha_innodb.cc started to #include .ic files), and more so in MySQL 5.7 when inline methods and functions were introduced in .h files. Solution: Remove all references to UNIV_NONINL and UNIV_MUST_NOT_INLINE from all files, assuming that the symbols are never defined. Remove the files fut0fut.cc and ut0byte.cc which only mattered when UNIV_NONINL was defined.	10 years ago
Marko Mäkelä	89d80c1b0b	Fix many -Wconversion warnings. Define my_thread_id as an unsigned type, to avoid mismatch with ulonglong. Change some parameters to this type. Use size_t in a few more places. Declare many flag constants as unsigned to avoid sign mismatch when shifting bits or applying the unary ~ operator. When applying the unary ~ operator to enum constants, explictly cast the result to an unsigned type, because enum constants can be treated as signed. In InnoDB, change the source code line number parameters from ulint to unsigned type. Also, make some InnoDB functions return a narrower type (unsigned or uint32_t instead of ulint; bool instead of ibool).	9 years ago
Marko Mäkelä	27b9989d31	MDEV-12121 Introduce build option WITH_INNODB_AHI to disable innodb_adaptive_hash_index The InnoDB adaptive hash index is sometimes degrading the performance of InnoDB, and it is sometimes disabled to get more consistent performance. We should have a compile-time option to disable the adaptive hash index. Let us introduce two options: OPTION(WITH_INNODB_AHI "Include innodb_adaptive_hash_index" ON) OPTION(WITH_INNODB_ROOT_GUESS "Cache index root block descriptors" ON) where WITH_INNODB_AHI always implies WITH_INNODB_ROOT_GUESS. As part of this change, the misleadingly named function trx_search_latch_release_if_reserved(trx) will be replaced with the macro trx_assert_no_search_latch(trx) that will be empty unless BTR_CUR_HASH_ADAPT is defined (cmake -DWITH_INNODB_AHI=ON). We will also remove the unused column INFORMATION_SCHEMA.INNODB_TRX.TRX_ADAPTIVE_HASH_TIMEOUT. In MariaDB Server 10.1, it used to reflect the value of trx_t::search_latch_timeout which could be adjusted during row_search_for_mysql(). In 10.2, there is no such field. Other than the removal of the unused column TRX_ADAPTIVE_HASH_TIMEOUT, this is an almost non-functional change to the server when using the default build options. Some tests are adjusted so that they will work with both -DWITH_INNODB_AHI=ON and -DWITH_INNODB_AHI=OFF. The test innodb.innodb_monitor has been renamed to innodb.monitor in order to track MySQL 5.7, and the duplicate tests sys_vars.innodb_monitor_* are removed.	9 years ago
Marko Mäkelä	a13a636c74	MDEV-11802 innodb.innodb_bug14676111 fails The function trx_purge_stop() was calling os_event_reset(purge_sys->event) before calling rw_lock_x_lock(&purge_sys->latch). The os_event_set() call in srv_purge_coordinator_suspend() is protected by that X-latch. It would seem a good idea to consistently protect both os_event_set() and os_event_reset() calls with a common mutex or rw-lock in those cases where os_event_set() and os_event_reset() are used like condition variables, tied to changes of shared state. For each os_event_t, we try to document the mutex or rw-lock that is being used. For some events, frequent calls to os_event_set() seem to try to avoid hangs. Some events are never waited for infinitely, only timed waits, and os_event_set() is used for early termination of these waits. os_aio_simulated_put_read_threads_to_sleep(): Define as a null macro on other systems than Windows. TODO: remove this altogether and disable innodb_use_native_aio on Windows. os_aio_segment_wait_events[]: Initialize only if innodb_use_native_aio=0. log_write_flush_to_disk_low(): Invoke log_mutex_enter() at the end, to avoid race conditions when changing the system state. (No potential race condition existed before MySQL 5.7.)	9 years ago
Marko Mäkelä	13493078e9	MDEV-11802 innodb.innodb_bug14676111 fails The function trx_purge_stop() was calling os_event_reset(purge_sys->event) before calling rw_lock_x_lock(&purge_sys->latch). The os_event_set() call in srv_purge_coordinator_suspend() is protected by that X-latch. It would seem a good idea to consistently protect both os_event_set() and os_event_reset() calls with a common mutex or rw-lock in those cases where os_event_set() and os_event_reset() are used like condition variables, tied to changes of shared state. For each os_event_t, we try to document the mutex or rw-lock that is being used. For some events, frequent calls to os_event_set() seem to try to avoid hangs. Some events are never waited for infinitely, only timed waits, and os_event_set() is used for early termination of these waits. os_aio_simulated_put_read_threads_to_sleep(): Define as a null macro on other systems than Windows. TODO: remove this altogether and disable innodb_use_native_aio on Windows. os_aio_segment_wait_events[]: Initialize only if innodb_use_native_aio=0.	9 years ago
Jan Lindström	41cd80fe06	After review fixes.	9 years ago
Jan Lindström	0340067608	After review fixes for MDEV-11759. buf_page_is_checksum_valid_crc32() buf_page_is_checksum_valid_innodb() buf_page_is_checksum_valid_none(): Use ULINTPF instead of %lu and %u for ib_uint32_t fil_space_verify_crypt_checksum(): Check that page is really empty if checksum and LSN are zero. fil_space_verify_crypt_checksum(): Correct the comment to be more agurate. buf0buf.h: Remove unnecessary is_corrupt variable from buf_page_t structure.	9 years ago
Jan Lindström	ddf2fac733	MDEV-11759: Encryption code in MariaDB 10.1/10.2 causes compatibility problems Pages that are encrypted contain post encryption checksum on different location that normal checksum fields. Therefore, we should before decryption check this checksum to avoid unencrypting corrupted pages. After decryption we can use traditional checksum check to detect if page is corrupted or unencryption was done using incorrect key. Pages that are page compressed do not contain any checksum, here we need to fist unencrypt, decompress and finally use tradional checksum check to detect page corruption or that we used incorrect key in unencryption. buf0buf.cc: buf_page_is_corrupted() mofified so that compressed pages are skipped. buf0buf.h, buf_block_init(), buf_page_init_low(): removed unnecessary page_encrypted, page_compressed, stored_checksum, valculated_checksum fields from buf_page_t buf_page_get_gen(): use new buf_page_check_corrupt() function to detect corrupted pages. buf_page_check_corrupt(): If page was not yet decrypted check if post encryption checksum still matches. If page is not anymore encrypted, use buf_page_is_corrupted() traditional checksum method. If page is detected as corrupted and it is not encrypted we print corruption message to error log. If page is still encrypted or it was encrypted and now corrupted, we will print message that page is encrypted to error log. buf_page_io_complete(): use new buf_page_check_corrupt() function to detect corrupted pages. buf_page_decrypt_after_read(): Verify post encryption checksum before tring to decrypt. fil0crypt.cc: fil_encrypt_buf() verify post encryption checksum and ind fil_space_decrypt() return true if we really decrypted the page. fil_space_verify_crypt_checksum(): rewrite to use the method used when calculating post encryption checksum. We also check if post encryption checksum matches that traditional checksum check does not match. fil0fil.ic: Add missed page type encrypted and page compressed to fil_get_page_type_name() Note that this change does not yet fix innochecksum tool, that will be done in separate MDEV. Fix test failures caused by buf page corruption injection.	9 years ago
Marko Mäkelä	1eabad5dbe	Remove MYSQL_COMPRESSION. The MariaDB 10.1 page_compression is incompatible with the Oracle implementation that was introduced in MySQL 5.7 later. Remove the Oracle implementation. Also remove the remaining traces of MYSQL_ENCRYPTION. This will also remove traces of PUNCH_HOLE until it is implemented better. The only effective call to os_file_punch_hole() was in fil_node_create_low() to test if the operation is supported for the file. In other words, it looks like page_compression is not working in MariaDB 10.2, because no code equivalent to the 10.1 os_file_trim() is enabled.	9 years ago
Marko Mäkelä	63574f1275	MDEV-11690 Remove UNIV_HOTBACKUP The InnoDB source code contains quite a few references to a closed-source hot backup tool which was originally called InnoDB Hot Backup (ibbackup) and later incorporated in MySQL Enterprise Backup. The open source backup tool XtraBackup uses the full database for recovery. So, the references to UNIV_HOTBACKUP are only cluttering the source code.	9 years ago
Marko Mäkelä	c868acdf65	MDEV-11487 Revert InnoDB internal temporary tables from WL#7682 WL#7682 in MySQL 5.7 introduced the possibility to create light-weight temporary tables in InnoDB. These are called 'intrinsic temporary tables' in InnoDB, and in MySQL 5.7, they can be created by the optimizer for sorting or buffering data in query processing. In MariaDB 10.2, the optimizer temporary tables cannot be created in InnoDB, so we should remove the dead code and related data structures.	9 years ago
Marko Mäkelä	8da33e3a86	MDEV-11349 (1/2) Fix some clang 4.0 warnings In InnoDB and XtraDB functions that declare pointer parameters as nonnull, remove nullness checks, because GCC would optimize them away anyway. Use #ifdef instead of #if when checking for a configuration flag. Clang says that left shifts of negative values are undefined. So, use ~0U instead of ~0 in a number of macros. Some functions that were defined as UNIV_INLINE were declared as UNIV_INTERN. Consistently use the same type of linkage. ibuf_merge_or_delete_for_page() could pass bitmap_page=NULL to buf_page_print(), conflicting with the __attribute__((nonnull)).	9 years ago
Jan Lindström	fec844aca8	Merge InnoDB 5.7 from mysql-5.7.14. Contains also: MDEV-10549 mysqld: sql/handler.cc:2692: int handler::ha_index_first(uchar*): Assertion `table_share->tmp_table != NO_TMP_TABLE \|\| m_lock_type != 2' failed. (branch bb-10.2-jan) Unlike MySQL, InnoDB still uses THR_LOCK in MariaDB MDEV-10548 Some of the debug sync waits do not work with InnoDB 5.7 (branch bb-10.2-jan) enable tests that were fixed in MDEV-10549 MDEV-10548 Some of the debug sync waits do not work with InnoDB 5.7 (branch bb-10.2-jan) fix main.innodb_mysql_sync - re-enable online alter for partitioned innodb tables	9 years ago
Jan Lindström	2e814d4702	Merge InnoDB 5.7 from mysql-5.7.9. Contains also MDEV-10547: Test multi_update_innodb fails with InnoDB 5.7 The failure happened because 5.7 has changed the signature of the bool handler::primary_key_is_clustered() const virtual function ("const" was added). InnoDB was using the old signature which caused the function not to be used. MDEV-10550: Parallel replication lock waits/deadlock handling does not work with InnoDB 5.7 Fixed mutexing problem on lock_trx_handle_wait. Note that rpl_parallel and rpl_optimistic_parallel tests still fail. MDEV-10156 : Group commit tests fail on 10.2 InnoDB (branch bb-10.2-jan) Reason: incorrect merge MDEV-10550: Parallel replication can't sync with master in InnoDB 5.7 (branch bb-10.2-jan) Reason: incorrect merge	9 years ago
Sergei Golubchik	720e04ff67	5.6.31	9 years ago
Jan Lindström	69147040a6	MDEV-9236: Dramatically overallocation of InnoDB buffer pool leads to crash Part I: Add diagnostics to page allocation if state is not correct but do not assert if it is incorrect.	10 years ago
Jan Lindström	7e916bb86f	MDEV-8588: Assertion failure in file ha_innodb.cc line 21140 if at least one encrypted table exists and encryption service is not available Analysis: Problem was that in fil_read_first_page we do find that table has encryption information and that encryption service or used key_id is not available. But, then we just printed fatal error message that causes above assertion. Fix: When we open single table tablespace if it has encryption information (crypt_data) store this crypt data to the table structure. When we open a table and we find out that tablespace is not available, check has table a encryption information and from there is encryption service or used key_id is not available. If it is, add additional warning for SQL-layer.	10 years ago
Jan Lindström	a80753594a	MDEV-8591: Database page corruption on disk or a failed space, Assertion failure in file buf0buf.cc line 2856 on querying a table using wrong default encryption key Improved error messaging to show based on original page before encryption is page maybe encrypted or just corrupted.	10 years ago
Jan Lindström	d7f3d889de	MDEV-8272: Encryption performance: Reduce the number of unused memcpy's Removed memcpy's on cases when page is not encrypted and make sure we use the correct buffer for reading/writing.	11 years ago
Jan Lindström	f7002c05ae	MDEV-8250: InnoDB: Page compressed tables are not compressed and compressed+encrypted tables cause crash Analysis: Problem is that both encrypted tables and compressed tables use FIL header offset FIL_PAGE_FILE_FLUSH_LSN_OR_KEY_VERSION to store required metadata. Furhermore, for only compressed tables currently code skips compression. Fixes: - Only encrypted pages store key_version to FIL header offset FIL_PAGE_FILE_FLUSH_LSN_OR_KEY_VERSION, no need to fix - Only compressed pages store compression algorithm to FIL header offset FIL_PAGE_FILE_FLUSH_LSN_OR_KEY_VERSION, no need to fix as they have different page type FIL_PAGE_PAGE_COMPRESSED - Compressed and encrypted pages now use a new page type FIL_PAGE_PAGE_COMPRESSED_ENCRYPTED and key_version is stored on FIL header offset FIL_PAGE_FILE_FLUSH_LSN_OR_KEY_VERSION and compression method is stored after FIL header similar way as compressed size, so that first FIL_PAGE_COMPRESSED_SIZE is stored followed by FIL_PAGE_COMPRESSION_METHOD - Fix buf_page_encrypt_before_write function to really compress pages if compression is enabled - Fix buf_page_decrypt_after_read function to really decompress pages if compression is used - Small style fixes	11 years ago
Sergei Golubchik	6d06fbbd1d	move to storage/innobase	11 years ago
Sergei Golubchik	14a142fca6	move to storage/xtradb	11 years ago
Sergei Golubchik	fd39c56eff	move to storage/xtradb/	11 years ago
Sergei Golubchik	1645930d0b	5.6.23	11 years ago
Sergei Golubchik	67da9e81c7	5.6.22-71.0	11 years ago
Sergei Golubchik	89b6517c4c	percona-server-5.6.20-68.0	11 years ago
Sergei Golubchik	993395bfa0	5.6.20	11 years ago
Sergei Golubchik	6cb3146af8	percona-server-5.6.17-65.0	12 years ago
Sergei Golubchik	60aba058b1	5.6.17	12 years ago
Sergei Golubchik	83759e02dc	5.6.16	12 years ago
Sergei Golubchik	05fd3e6366	percona-server-5.6.16-64.2.tar.gz	12 years ago

1 2 3

115 Commits (14be81438098923e95bda6d635d638991e60116b)