mariadb

Tree: ea37b14409

Author	SHA1	Message	Date
Marko Mäkelä	ea37b14409	MDEV-16678 Prefer MDL to dict_sys.latch for innodb background tasks This is joint work with Thirunarayanan Balathandayuthapani. The MDL interface between InnoDB and the rest of the server (in storage/innobase/dict/dict0dict.cc and in include/) is my work, while most everything else is Thiru's. The collection of InnoDB persistent statistics and the defragmentation were not refactored to use MDL. They will keep relying on lower-level interlocking with fil_check_pending_operations(). The purge of transaction history and the background operations on fulltext indexes will use MDL. We will revert commit `2c4844c9e7` (MDEV-17813) because thanks to MDL, purge cannot conflict with DDL operations anymore. For a similar reason, we will remove the MDEV-16222 test case from gcol.innodb_virtual_debug_purge. Purge is essentially replacing all use of the global dict_sys.latch with MDL. Purge will skip the undo log records for tables whose names start with #sql-ib or #sql2. Theoretically, such tables might be renamed back to visible table names if TRUNCATE fails to create a new table, or the final rename in ALTER TABLE...ALGORITHM=COPY fails. In that case, purge could permanently leave some garbage in the table. Such garbage will be tolerated; the table would not be considered corrupted. To avoid repeated MDL releases and acquisitions, trx_purge_attach_undo_recs() will sort undo log records by table_id, and purge_node_t will keep the MDL and table handle open for multiple successive undo log records. get_purge_table(): A new accessor, used during the purge of history for indexed virtual columns. This interface should ideally not exist at all. thd_mdl_context(): Accessor of THD::mdl_context. Wrapped in a new thd_mdl_service. dict_get_db_name_len(): Define inline. dict_acquire_mdl_shared(): Acquire explicit shared MDL on a table name if needed. dict_table_open_on_id(): Return MDL_ticket, if requested. dict_table_close(): Release MDL ticket, if requested. dict_fts_index_syncing(), dict_index_t::index_fts_syncing: Remove. row_drop_table_for_mysql() no longer needs to check these, because MDL guarantees that a fulltext index sync will not be in progress while MDL_EXCLUSIVE is protecting a DDL operation. dict_table_t::parse_name(): Parse the table name for acquiring MDL. purge_node_t::undo_recs: Change the type to std::list<trx_purge_rec_t*> (different container, and storing also roll_ptr). purge_node_t: Add mdl_ticket, last_table_id, purge_thd, mdl_hold_recs for acquiring MDL and for keeping the table open across multiple undo log records. purge_vcol_info_t, row_purge_store_vsec_cur(), row_purge_restore_vsec_cur(): Remove. We will acquire the MDL earlier. purge_sys_t::heap: Added, for reading undo log records. fts_sync_during_ddl(): Invoked during ALGORITHM=INPLACE operations to ensure that fts_sync_table() will not conflict with MDL_EXCLUSIVE. Uses fts_t::sync_message for bookkeeping.	6 years ago
Vladislav Vaintroub	e47bd0073c	MDEV-21262 MTR does not work with Windows ASAN builds Do not use suppressions on Windows, the ASAN_OPTIONS can't be parsed whenever full file names are used, because file names contain ':' which is ASAN_OPTIONS delimeter.	6 years ago
Vladislav Vaintroub	66de4fef76	MDEV-16264 - some improvements - wait notification, tpool_wait_begin/tpool_wait_end - to notify the threadpool that current thread is going to wait Use it to wait for IOs to complete and also when purge waits for workers.	6 years ago
Marko Mäkelä	d3b2625ba0	MDEV-21259 Assertion failed in mtr_t::write() btr_free_externally_stored_field(): Pass w=mtr_t::OPT to note that the BTR_EXTERN_LEN is not necessarily changing when a multi-page ROW_FORMAT=COMPRESSED off-page column is being freed, and to allow redundant writes to the redo log to be optimized away. Ever since commit `56f6dab1d0` the refactored function mtr_t::write() asserts by default that the page contents is being changed.	6 years ago
Alexander Barkov	d30dbaa20d	A cleanup for MDEV-8844: Fixing compilation failure on Windows Fixing lossy type conversions: - from int64 to int - from size_t to uint	6 years ago
Alexander Barkov	3c6065a270	MDEV-8844 Unreadable control characters printed as is in warnings	6 years ago
Alexander Barkov	00445652db	A cleanup for MDEV-17088 Provide tools to encode/decode mysql-encoded file system names tests mariadb-conv-utf8 and mariadb-conv-cp932 failed on PPC, because "char" is "unsigned char" on PPC. Adding a cast from "char" to "signed char" in the two affected places.	6 years ago
Kentoku	1b040ce570	fix compiler warnings	6 years ago
Marko Mäkelä	6189774c37	MDEV-21174: Fix undefined behaviour ibuf_bitmap_page_set_bits(): Do not attempt to shift by a negative amount. This bug was introduced in commit `87839258f8`.	6 years ago
Marko Mäkelä	46fc3bdbca	MDEV-21225: Fix warnings Datafile::find_space_id(): Remove an unused variable buf_chunk_init(): warning C4319: '~': zero extending 'ulong' to 'ulint' of greater size	6 years ago
Alexander Barkov	2c7b6214e7	A cleanup for MDEV-17088 Provide tools to encode/decode mysql-encoded file system names - Load and convert the entire input file at once, rather than reading string-by-string using fgets(). This change makes it possible to convert from UCS2, UTF16, UTF32 data. - Adding the --delimiter command, to treat the specified characters as delimiters rather than data to convert. Useful in combination with `-f filename` or `-t filename`. The delimiter characters are not converted, they are copied from the input to the output as is. - Adding diagnostics with line number and position if: * an illegal input byte sequence was found * a character cannot be converted to the target character set	6 years ago
Marko Mäkelä	42a4ae54c2	MDEV-21225 Remove ut_align() and use aligned_malloc() Before commit `90c52e5291` introduced aligned_malloc(), InnoDB always used a pattern of over-allocating memory and invoking ut_align() to guarantee the desired alignment. It is cleaner to invoke aligned_malloc() and aligned_free() directly. ut_align(): Remove. In assertions, ut_align_down() can be used instead.	6 years ago
Marko Mäkelä	504202bd7f	MDEV-21216: Remove fsp_header_get_space_id() The function fsp_header_get_space_id() returns ulint instead of uint32_t, only to be able to complain that the two adjacent tablespace ID fields in the page differ. Remove the function, and merge the check to the callers. Also, make some more use of aligned_malloc().	6 years ago
Marko Mäkelä	2352f51625	MDEV-21174: Fix Galera trx_rseg_write_wsrep_checkpoint(): Use mtr_t::OPT, because much of the time, the redo log writes would be redundant. This was broken in commit `56f6dab1d0`.	6 years ago
Marko Mäkelä	6b5cdd4ff7	MDEV-19514: Update stale comments	6 years ago
Marko Mäkelä	95e903261e	MDEV-21216 InnoDB does dirty read of TRX_SYS page before recovery InnoDB startup was discovering undo tablespaces in a dirty way. It was reading a possibly stale copy of the TRX_SYS page before processing any redo log records. srv_start(): Do not call buf_pool_invalidate(). Invoke trx_rseg_get_n_undo_tablespaces() after the recovery has been initiated. recv_recovery_from_checkpoint_start(): Assert that the buffer pool is empty. This used to be guaranteed by the buf_pool_invalidate() call. trx_rseg_get_n_undo_tablespaces(): Move to the calling compilation unit, and reimplement in a simpler way. srv_undo_tablespace_create(): Remove the constant parameter size=SRV_UNDO_TABLESPACE_SIZE_IN_PAGES. srv_undo_tablespace_open(): Reimplement in a cleaner way, with more robust error handling. srv_all_undo_tablespaces_open(): Split from srv_undo_tablespaces_init(). srv_undo_tablespaces_init(): Read all "undo001","undo002" tablespace files directly, without consulting the TRX_SYS page via calling trx_rseg_get_n_undo_tablespaces(). This is joint work with Thirunarayanan Balathandayuthapani.	6 years ago
Marko Mäkelä	e5dfdc5606	Cleanup: use constexpr for SRV_UNDO_TABLESPACE_SIZE_IN_PAGES	6 years ago
Vladislav Vaintroub	bf3034195f	Part2: MDEV-12518 Unify sql_yacc.yy and sql_yacc_ora.yy Generate sql_yacc_ora.yy from sql_yacc.yy , by using cmake script which does some simple string(REPLACE)	6 years ago
Alexander Barkov	3f9d7072e9	Part1: MDEV-12518 Unify sql_yacc.yy and sql_yacc_ora.yy Mixing %type and %expect declarations: - sql_mode=ORACLE declarations look like an empty C code section inside sql_yacc.yy, consisting of an inactive #ifdef..#endif block. - sql_mode=DEFAULT declarations look like an empty C code section inside sql_yacc_ora.yy, consisting of an inactive #ifdef..#endif block. Mixing rules: - Adding a special rule _empty to the shared rule section. - Changing all instances of /Empty/ in sql_mode=DEFAULT and sql_mode=ORACLE specific sections to _empty. - Changing the rest of C style comments /xxx/ in sql_mode=DEFAULT and sql_mode=ORACLE specific blocks to C++ style: //xxx - Mixing sql_yacc.yy and sql_yacc_ora.yy, so sql_mode=ORACLE specific blocks sit in a comment inside sql_yacc.yy, and sql_mode=DEFAULT specific blocks sit in a comment inside sql_yacc_ora.yy.	6 years ago
Marko Mäkelä	bb45941685	MDEV-21205 Assertion failure in btr_sec_min_rec_mark In commit `af5947f433` the function btr_discard_page() is invoking btr_set_min_rec_mark() with the wrong buf_block_t* object. node_ptr is on merge_block, not block. btr_discard_page(): Remove the variables merge_page, page, and always refer to block->frame or merge_block->frame instead. Also, limit the scope of node_ptr and avoid duplicated conditions. btr_set_min_rec_mark(): Add a template parameter, so that the caller can specify whether the page is supposed to have a left sibling. Otherwise, the assertion (which was introduced in the same commit) would fail in btr_discard_page().	6 years ago
Daniel Bartholomew	3b9a978a3b	bump the VERSION	6 years ago
Marko Mäkelä	2ac0e64cad	MDEV-21174: Fix the 32-bit build mtr_t::write(): Add explicit narrowing type casts to avoid warnings about lossy implicit conversions.	6 years ago
Marko Mäkelä	af5947f433	MDEV-21174: Replace mlog_write_string() with mtr_t::memcpy() mtr_t::memcpy(): Replaces mlog_write_string(), mlog_log_string(). The buf_block_t is passed a parameter, so that mlog_write_initial_log_record_low() can be used instead of mlog_write_initial_log_record_fast(). fil_space_crypt_t::write_page0(): Remove the fil_space_t* parameter.	6 years ago
Marko Mäkelä	87839258f8	MDEV-21174: Replace mlog_memset() with mtr_t::memset() Passing buf_block_t helps us avoid calling mlog_write_initial_log_record_fast() and page_get_page_no(), and allows us to implement more debug checks, such as that on ROW_FORMAT=COMPRESSED index pages, only the page header may be modified by MLOG_MEMSET records. fseg_n_reserved_pages(): Add a buf_block_t parameter.	6 years ago
Marko Mäkelä	caea64df18	Cleanup: Remove some page_get_page_no() calls Refer to buf_page_t::id instead of parsing the tablespace identifier or page number from the buffer pool page.	6 years ago
Marko Mäkelä	56f6dab1d0	MDEV-21174: Replace mlog_write_ulint() with mtr_t::write() mtr_t::write(): Replaces mlog_write_ulint(), mlog_write_ull(). Optimize away writes if the page contents does not change, except when a dummy write has been explicitly requested. Because the member function template takes a block descriptor as a parameter, it is possible to introduce better consistency checks. Due to this, the code for handling file-based lists, undo logs and user transactions was refactored to pass around buf_block_t.	6 years ago
Marko Mäkelä	504823bcce	MDEV-21174: Cleanup MLOG_PAGE_CREATE page_create_write_log(), mlog_write_initial_log_record(): Merge to the only caller, and use mlog_write_initial_log_record_low() for writing the log record.	6 years ago
Marko Mäkelä	57444a3b30	MDEV-16264: Minor cleanup aio_linux::m_max_io_count: Unused data member; remove. aiocb::m_ret_len: Declare as the more compatible type size_t. Unfortunately, ssize_t is not available on Microsoft Visual Studio.	6 years ago
Marko Mäkelä	cd92c6c83d	MDEV-12353 preparation: Do not write MLOG_REC_MIN_MARK btr_set_min_rec_mark(): Write MLOG_1BYTE instead of MLOG_REC_MIN_MARK or MLOG_COMP_REC_MIN_MARK. On ROW_FORMAT=COMPRESSED pages, the minimum record flag is not stored at all. The flag is computed for the uncompressed page by page_zip_decompress(). Hence, nothing needs to be logged for ROW_FORMAT=COMPRESSED tables for this operation. To facilitate crash-upgrade and hot backup from older versions, we will retain the code to parse and apply the old log record types MLOG_REC_MIN_MARK and MLOG_COMP_REC_MIN_MARK.	6 years ago
Marko Mäkelä	8ebd91c1a9	MDEV-12353 preparation: Do not write MLOG_FILE_WRITE_CRYPT_DATA The MLOG_FILE_WRITE_CRYPT_DATA record was completely redundant. It can be replaced with a single MLOG_WRITE_STRING record. To facilitate upgrade from older versions, we will retain fil_parse_write_crypt_data(). fil_crypt_parse(): Recover fil_space_crypt_t::write_page0(). fil_space_crypt_t::write_page0(): Write everything in a single MLOG_WRITE_STRING for easy parsing. fil_space_crypt_t::page0_offset: Remove.	6 years ago
Marko Mäkelä	bf2cc46798	MDEV-21133: Remove buf_frame_copy()	6 years ago
Aleksey Midenkov	6f89946892	MDEV-17554 versioning partition tests reorganize Moved partition_rotation, partition_innodb to partition.test	6 years ago
Aleksey Midenkov	9ed8d364cd	MDEV-17554 history partitioning cleanups * Fixed missed warning on condition boundary * REORGANIZE cases * vers_utils.h removed * test cases cleanup	6 years ago
Aleksey Midenkov	8ed646f071	Merge 10.4 into 10.5	6 years ago
Aleksey Midenkov	0b8b11b0b1	Merge 10.3 into 10.4	6 years ago
Aleksey Midenkov	1d46923a0f	MDEV-18929 2nd execution of SP does not detect ER_VERS_NOT_VERSIONED (10.4) Don't do skip_setup_conds() unless all errors are checked. Fixes following errors: ER_PERIOD_NOT_FOUND ER_VERS_QUERY_IN_PARTITION ER_VERS_ENGINE_UNSUPPORTED ER_VERS_NOT_VERSIONED	6 years ago
Aleksey Midenkov	db32d9457e	MDEV-18929 2nd execution of SP does not detect ER_VERS_NOT_VERSIONED Don't do skip_setup_conds() unless all errors are checked. Fixes following errors: ER_PERIOD_NOT_FOUND ER_VERS_QUERY_IN_PARTITION ER_VERS_ENGINE_UNSUPPORTED ER_VERS_NOT_VERSIONED	6 years ago
Aleksey Midenkov	a7cf0db3d8	MDEV-21011 Table corruption reported for versioned partitioned table after DELETE: "Found a misplaced row" LIMIT history partitions cannot be checked by existing algorithm of check_misplaced_rows() because working history partition is incremented each time another one is filled. The existing algorithm gets record and tries to decide partition id for it by get_partition_id(). For LIMIT history it will just get first non-filled partition. To fix such partitions it is required to do REBUILD instead of REPAIR.	6 years ago
Aleksey Midenkov	6dd41e008e	MDEV-21155 Assertion with versioned table upon DELETE from view of view after replacing first view When view is merged by DT_MERGE_FOR_INSERT it is then skipped from processing and doesn't update WHERE clause with vers_setup_conds(). Note that view itself cannot work in vers_setup_conds() because it doesn't have row_start, row_end fields. Thus it is required to descend down to material TABLE_LIST through calls of mysql_derived_prepare() and run vers_setup_conds() from there. Luckily, all views (views of views, views of views of views, etc.) are linked in one list through next_global pointer, so we can skip all views of views and get straight to non-view TABLE_LIST by checking its merge_underlying_list property for zero value (it is assigned by DT_MERGE_FOR_INSERT for merged derived tables). We have to do that only for UPDATE and DELETE. Other DML commands don't use WHERE clause. MDEV-21146 Assertion `m_lock_type == 2' in handler::ha_drop_table upon LOAD DATA LOAD DATA does not use WHERE and the above call of vers_setup_conds() is not needed. unit->prepare() led to wrongly locked temporary table.	6 years ago
Aleksey Midenkov	97aa07abf5	MDEV-21147 Assertion `marked_for_read()' upon UPDATE on versioned table via view "write set" for replication finally got its correct place (mark_columns_per_binlog_row_image()). When done generally in mark_columns_needed_for_update() it affects optimization algorithm. used_key_is_modified, query_plan.using_io_buffer are wrongly set and that leads to wrong prepare_for_keyread() which limits read_set.	6 years ago
Aleksey Midenkov	498a96a478	MDEV-20441 ER_CRASHED_ON_USAGE upon update on versioned Aria table Turn read cache off for update and multi-update for versioned table. no_cache is reinited on each TABLE open because it is applicable for specific algorithms. As a side fix vers_insert_history_row() honors vers_write setting. Aria with row_format=fixed uses IO_CACHE of type READ_CACHE for sequential read in update loop. When history row is inserted inside this loop the cache misses it and fails with error. TODO: Currently maria_extra() does not support SEQ_READ_APPEND. Probably it might be possible to use this type of cache.	6 years ago
Alexander Barkov	523879dd51	A cleanup for MDEV-17088: disabling mariadb-conv-*.test for embedded	6 years ago
Kentoku	a3b63b8da3	MDEV-18973 CLIENT_FOUND_ROWS wrong in spider Get count from last_used_con->info Contributed by willhan at Tencent Games Reviewed by Monty	6 years ago
Kentoku	e066723a41	MDEV-18973 CLIENT_FOUND_ROWS wrong in spider Get count from last_used_con->info Contributed by willhan at Tencent Games	6 years ago
Marko Mäkelä	326515878d	MDEV-21133: Alignment hints for ROW_FORMAT=COMPRESSED page_zip_compress(), page_zip_decompress_low(), page_zip_reorganize(): Make use of memcpy_aligned() and memset_aligned(), so that some operations can be translated more efficiently. No difference was observed for AMD64 on GCC 9.2.1. But, such hints could enable more efficient code on less common instruction set architectures, such as 32-bit ARM, POWER or MIPS.	6 years ago
Marko Mäkelä	5a00792c69	Merge 10.4 into 10.5	6 years ago
Marko Mäkelä	51a4260f00	MDEV-21133: Introduce memmove_aligned() Both variants of the InnoDB page directory are aligned to the entry size (16 bits). Inform the compiler about it.	6 years ago
Sergei Golubchik	57cab7cd51	fix ssl_crl test failures on newer OpenSSL generalize the replacement	6 years ago
Sergei Golubchik	95898ac9bd	Revert "Fix upgrade errors on eoan" This reverts commit `0d345ec2e3`. Upgrades from 8.0 don't work yet, one has to dump/restore manually to get the metadata out of the data dictionary.	6 years ago
Marko Mäkelä	7955e197d0	Cleanup: Remove a constant template parameter dict_table_t::init_instant(): Remove bool replace_dropped=true. The template parameter was unnecessarily added in commit `a044e326a8`.	6 years ago

1 2 3 4 5 ...

187968 Commits (ea37b144094a0c2ebfc6774047fd473c1b2a8658) All Branches Search

187968 Commits (ea37b144094a0c2ebfc6774047fd473c1b2a8658)

All Branches