mariadb

Tree: 5cd982c51b

Author	SHA1	Message	Date
Daniel Black	5cd982c51b	MDEV-34863 RAM Usage Changed Significantly Between 10.11 Releases (postfix) Correct error message to use the correct system variable name - innodb_buffer_pool_size_auto_min	6 months ago
Marko Mäkelä	4d37b1c4b9	MDEV-36886 log_t::get_lsn_approx() isn't lower bound If the execution of the two reads in log_t::get_lsn_approx() is interleaved with concurrent writes of those fields in log_t::write_buf() or log_t::persist(), the returned approximation will be an upper bound. If log_t::append_prepare_wait() is pending, the approximation could be a lower bound. We must adjust each caller of log_t::get_lsn_approx() for the possibility that the return value is larger than MAX(oldest_modification) in buf_pool.flush_list. af_needed_for_redo(): Add a comment that explains why the glitch is not a problem. page_cleaner_flush_pages_recommendation(): Revise the logic for the unlikely case cur_lsn < oldest_lsn. The original logic would have invoked af_get_pct_for_lsn() with a very large age value, which would likely cause an overflow of the local variable lsn_age_factor, and make pct_for_lsn a "random number". Based on that value, total_ratio would be normalized to something between 0.0 and 1.0. Nothing extremely bad should have happened in this case; the innodb_io_capacity_max should not be exceeded.	6 months ago
Marko Mäkelä	7b4b759f13	MDEV-36868: Inconsistency when shrinking innodb_buffer_pool_size buf_pool_t::resize(): After successfully shrinking the buffer pool, announce the success. The size had already been updated in shrunk(). After failing to shrink the buffer pool, re-enable the adaptive hash index if it had been enabled. Reviewed by: Debarun Banerjee	6 months ago
Vladislav Vaintroub	bcd02c79f3	Merge branch '10.6' into 10.11	6 months ago
Vladislav Vaintroub	aba04c562b	Compiling - fix warnings with MSVC 17.14 New warnings come from 3 places 1. Warning C5287: Warning comes from json_lib.c from code like compile_time_assert((int) JSON_VALUE_NULL == (int) JSV_NULL); 2. Warning C5287: Similar warning come from wc_static_assert() from code in wolfSSL's header file 3. Warning C5286 in WolfSSL code, -enum_value (i.e multiplying enum with -1)is used To fix: - Disable warnings in WolfSSL code, using /wd<num> flag. - workaround warning for users of WolfSSL, disable wc_static_assert() with -DWC_NO_STATIC_ASSERT compile flag - Rewrite some compile_time_assert in json_lib.c to avoid warning. - add target_link_libraries(vio ${SSL_LIBRARIES}) so that vio picks up -DWC_NO_STATIC_ASSERT	6 months ago
Thirunarayanan Balathandayuthapani	db188083c3	MDEV-36771 Assertion 'bulk_insert == TRX_NO_BULK' failed in trx_t::assert_freed - InnoDB fails to reset bulk_insert of a transaction while freeing the transaction during shutting down of a server.	6 months ago
Marko Mäkelä	3da36fa130	Merge 10.6 into 10.11	6 months ago
Thirunarayanan Balathandayuthapani	d8962d138f	MDEV-36017 Alter table aborts when temporary directory is full Problem: ======= - In 10.11, During Copy algorithm, InnoDB does use bulk insert for row by row insert operation. When temporary directory ran out of memory, row_mysql_handle_errors() fails to handle DB_TEMP_FILE_WRITE_FAIL. - During inplace algorithm, concurrent DML fails to write the log operation into the temporary file. InnoDB fail to mark the error for the online log. - ddl_log_write() releases the global ddl lock prematurely before release the log memory entry Fix: === row_mysql_handle_errors(): Rollback the transaction when InnoDB encounters DB_TEMP_FILE_WRITE_FAIL convert_error_code_to_mysql(): Report an aborted transaction when InnoDB encounters DB_TEMP_FILE_WRITE_FAIL during alter table algorithm=copy or innodb bulk insert operation row_log_online_op(): Mark the error in online log when InnoDB ran out of temporary space fil_space_extend_must_retry(): Mark the os_has_said_disk_full as true if os_file_set_size() fails btr_cur_pessimistic_update(): Return error code when btr_cur_pessimistic_insert() fails ddl_log_write(): Release the global ddl lock after releasing the log memory entry when error was encountered btr_cur_optimistic_update(): Relax the assertion that blob pointer can be null during rollback because InnoDB can ran out of space while allocating the external page ha_innobase::extra(): Rollback the transaction during DDL before calling convert_error_code_to_mysql(). row_undo_mod_upd_exist_sec(): Remove the assertion which says that InnoDB should fail to build index entry when rollbacking an incomplete transaction after crash recovery. This scenario can happen when InnoDB ran out of space. row_upd_changes_ord_field_binary_func(): Relax the assertion to make that externally stored field can be null when InnoDB ran out of space.	6 months ago
Monty	8bc1643148	MDEV-36860 exec_REDO_LOGREC_REDO_REPAIR_TABLE: Assertion `maria_tmpdir' failed The problem was that aria_backup_client code did not intialize maria_tmpdir, which is used during recovery if repair table is needed to reconstruct indexes.	6 months ago
Thirunarayanan Balathandayuthapani	8a4d3a044f	MDEV-36017 Alter table aborts when temporary directory is full Problem: ======= - During inplace algorithm, concurrent DML fails to write the log operation into the temporary file. InnoDB fail to mark the error for the online log. - ddl_log_write() releases the global ddl lock prematurely before release the log memory entry Fix: === row_log_online_op(): Mark the error in online log when InnoDB ran out of temporary space fil_space_extend_must_retry(): Mark the os_has_said_disk_full as true if os_file_set_size() fails btr_cur_pessimistic_update(): Return error code when btr_cur_pessimistic_insert() fails ddl_log_write(): Release the global ddl lock after releasing the log memory entry when error was encountered btr_cur_optimistic_update(): Relax the assertion that blob pointer can be null during rollback because InnoDB can ran out of space while allocating the external page row_undo_mod_upd_exist_sec(): Remove the assertion which says that InnoDB should fail to build index entry when rollbacking an incomplete transaction after crash recovery. This scenario can happen when InnoDB ran out of space. row_upd_changes_ord_field_binary_func(): Relax the assertion to make that externally stored field can be null when InnoDB ran out of space.	6 months ago
Oleksandr Byelkin	4c8143b451	Make it compiling with last gcc	6 months ago
Marko Mäkelä	96b7671b05	MDEV-34388 fixup: Stack usage in rename_table_in_stat_tables() rename_table_in_stat_tables(): Allocate TABLE_LIST[STATISTICS_TABLES] from the heap instead of the stack, to pass -Wframe-larger-than=16384 in an optimized CMAKE_BUILD_TYPE=RelWithDebInfo build.	6 months ago
Aleksey Midenkov	1037f95941	MDEV-33675 Assertion(reclength < vreclength) in setup_vcols_for_repair() When table2myisam() prepares recinfo structures BIT field was skipped because pack_length_in_rec() returns 0. Instead of BIT field DB_ROW_HASH_1 field was taken into recinfo structure and its length was added to reclength. This is wrong because not stored fields must not be prepared as record columns (MI_COLUMNDEF) in storage layer. 0-length fields are prepared in "reserve space for null bits" branch. The problem only occurs with tables where there is no data for the main record outside of the null bits. The fix updates minpos condition so we avoid fields after stored_rec_length and these are not stored by definition. share->reclength already includes not stored lengths from CREATE TABLE so we cannot use it as minpos starting point. In Aria there is no "reserve space for null bits" and it does not create column definition for BIT. Also there is no setup_vcols_for_repair() to reproduce the issue. But nonetheless it creates column definition for not stored fields redundantly, so we patch table2maria() as well. The test case for Aria tries to demonstrate BIT field works, it does not reproduce any issues (as redundant column definition for not stored field seem to not cause any problems).	6 months ago
Aleksey Midenkov	1a95c2a4b2	MDEV-36817 Server crashes in do_mark_index_columns instead of ER_DUP_ENTRY on partitioned table Now as `c1492f3d07` (MDEV-36115) restores m_last_part table->file points to partition p0 while the error happens in p1, so error index does not match ib_table in innobase_get_mysql_key_number_for_index(). This case is handled by separate code block in innobase_get_mysql_key_number_for_index() which was wrong on using secondary index for dict_index_is_auto_gen_clust() and it was not covered by the tests.	6 months ago
Daniel Bartholomew	69af638a0c	bump the VERSION	6 months ago
Oleksandr Byelkin	b8d3257243	Merge branch '10.11' into 10.11	6 months ago
Aleksey Midenkov	fbd736c872	mysqltest: result_format fix --result_format 2 command fails with assertion: DbugExit (why=0x7fffffff49e0 "missing DBUG_RETURN or DBUG_VOID_RETURN macro in function \"do_result_format_version\"\n") at ../src/dbug/dbug.c:2045	6 months ago
Aleksey Midenkov	d5247592c5	Test sysvars_server failure fix	6 months ago
Daniel Black	2c4fe3557a	MDEV-36337: mroonga_* udf correct ptr types for is_null/error Shows up in mroonga UDF tests under clang with UBSAN: UndefinedBehaviorSanitizer: function-type-mismatch Accepted upstream: https://github.com/mroonga/mroonga/pull/902	8 months ago
Daniel Black	b9a20752a9	MDEV-36337 auth_ed25519 correct UDF pointers for is_null/error Shows up on test plugins.auth_ed25519. There isn't the import to define uchar so left as unsigned char.	8 months ago
Daniel Black	0b5a084e27	MDEV-36337: connect UDF pointers need unsigned char* for is_null/error This is required to match the sql/sql_udf.h	8 months ago
Daniel Black	c91c2e74ff	MDEV-36337: udf_example UDF pointers need unsigned is_null/error This is required to match the sql/sql_udf.h	8 months ago
Marko Mäkelä	1c7209e828	Merge 10.6 into 10.11	6 months ago
Jan Lindström	497d6324bc	MDEV-36627 : galera.MDEV-29142: certification position less than last commited After the membership change on a newly synced node, then this is just a warning and safe to suppress. Signed-off-by: Julius Goryavsky <julius.goryavsky@mariadb.com>	7 months ago
Jan Lindström	7aed06887b	MDEV-36512 : galera_3nodes.GCF-354: certification position less than last committed Test changes only. Both warnings are expected and should be suppressed because we intentionally inject different inconsistencies on two nodes and then join them back with membership change. Signed-off-by: Julius Goryavsky <julius.goryavsky@mariadb.com>	7 months ago
Jan Lindström	7fd5957d55	MDEV-36622 : Hang during galera_evs_suspect_timeout test Test changes only. Add wait_condition so that all nodes are in the expected state and add debug output if issue does reproduce. Signed-off-by: Julius Goryavsky <julius.goryavsky@mariadb.com>	7 months ago
Hemant Dangi	d38558f99b	MDEV-36117: MDL BF-BF conflict on ALTER and UPDATE with multi-level foreign key parents Issue: Mariadb acquires additional MDL locks on UPDATE/INSERT/DELETE statements on table with foreign keys. For example, table t1 references t2, an UPDATE to t1 will MDL lock t2 in addition to t1. A replica may deliver an ALTER t1 and UPDATE t2 concurrently for applying. Then the UPDATE may acquire MDL lock for t1, followed by a conflict when the ALTER attempts to MDL lock on t1. Causing a BF-BF conflict. Solution: Additional keys for the referenced/foreign table needs to be added to avoid potential MDL conflicts with concurrent update and DDLs. Signed-off-by: Julius Goryavsky <julius.goryavsky@mariadb.com>	7 months ago
Jan Lindström	a96367bc67	MDEV-36620 post-fix: galera_toi_ddl_nonconflicting test failure Add wait_condition to wait until all inserted rows are replicated so that show create table is deterministic. Signed-off-by: Julius Goryavsky <julius.goryavsky@mariadb.com>	7 months ago
Marko Mäkelä	82d7419e06	MDEV-34388: Stack overflow on Alpine Linux page_is_corrupted(): Do not allocate the buffers from stack, but from the heap, in xb_fil_cur_open(). row_quiesce_write_cfg(): Issue one type of message when we fail to create the .cfg file. update_statistics_for_table(), read_statistics_for_table(), delete_statistics_for_table(), rename_table_in_stat_tables(): Use a common stack buffer for Index_stat, Column_stat, Table_stat. ha_connect::FileExists(): Invoke push_warning_printf() so that we can avoid allocating a buffer for snprintf(). translog_init_with_table(): Do not duplicate TRANSLOG_PAGE_SIZE_BUFF. Let us also globally enable the GCC 4.4 and clang 3.0 option -Wframe-larger-than=16384 to reduce the possibility of introducing such stack overflow in the future. For RocksDB and Mroonga we relax these limits. Reviewed by: Vladislav Lesin	6 months ago
Julius Goryavsky	2350295643	MDEV-36740: galera.galera_ssl_upgrade fails due to expired certificate The certificate used in tests has been renewed.	7 months ago
Jan Lindström	d7457b4076	MDEV-36628 : galera_vote_during_ist test failed Test changes only. Added wait_conditions to wait for expected database state. Signed-off-by: Julius Goryavsky <julius.goryavsky@mariadb.com>	7 months ago
Marko Mäkelä	8fb09426b9	MDEV-36759: Huge performance drop In commit `b6923420f3` (MDEV-29445) some hash tables were accidentally created with the minimum size (101 entries) instead of correctly deriving the size from the initial innodb_buffer_pool_size. This led to very long hash bucket chains, which are very slow to traverse. ut_find_prime(): Assert that the size is nonzero in order to catch this type of regression in the future. innodb_init_params(): Do not bother reading buf_pool.curr_size() when it is known to be 0, srv_start(): Correctly initialize srv_lock_table_size to 5 times buf_pool.curr_size(), that is, the buffer pool size in pages, between invoking buf_pool.create() and lock_sys.create(). btr_search_enable(), dict_sys_t::create(), dict_sys_t::resize(): Correctly refer to buf_pool.curr_pool_size(), that is, innodb_buffer_pool_size in bytes, when calculating the hash table size. In MDEV-29445 the expressions buf_pool_get_curr_size() were accidentally replaced with buf_pool.curr_size().	7 months ago
Marko Mäkelä	bb48d7bc81	MDEV-36781: Assertion i < BUF_BUDDY_SIZES failed in buf_buddy_shrink() buf_buddy_shrink(): Properly cover the case when KEY_BLOCK_SIZE corresponds to the innodb_page_size, that is, the ROW_FORMAT=COMPRESSED page frame is directly allocated from the buffer pool, not via the binary buddy allocator. buf_LRU_check_size_of_non_data_objects(): Avoid a crash when the buffer pool is being shrunk. buf_pool_t::shrink(): Abort if over 95% of the shrunk buffer pool would be occupied by the adaptive hash index or record locks.	7 months ago
Marko Mäkelä	56e0be34bc	MDEV-36780: InnoDB buffer pool reserves all assigned memory In commit `b6923420f3` (MDEV-29445) we started to specify the MAP_POPULATE flag for allocating the InnoDB buffer pool. This would cause a lot of time to be spent on __mm_populate() inside the Linux kernel, such as 16 seconds to pre-fault or commit innodb_buffer_pool_size=64G. Let us revert to the previous way of allocating the buffer pool at startup. Note: An attempt to increase the buffer pool size by SET GLOBAL innodb_buffer_pool_size (up to innodb_buffer_pool_size_max) will invoke my_virtual_mem_commit(), which will use MAP_POPULATE to zero-fill and prefault the requested additional memory area, blocking buf_pool.mutex. Before MDEV-29445 we allocated the InnoDB buffer pool by invoking mmap(2) once (via my_large_malloc()). After the change, we would invoke mmap(2) twice, first via my_virtual_mem_reserve() and then via my_virtual_mem_commit(). Outside Microsoft Windows, we are reverting back to my_large_malloc() like allocation. my_virtual_mem_reserve(): Define only for Microsoft Windows. Other platforms should invoke my_large_virtual_alloc() and update_malloc_size() instead of my_virtual_mem_reserve() and my_virtual_mem_commit(). my_large_virtual_alloc(): Define only outside Microsoft Windows. Do not specify MAP_NORESERVE nor MAP_POPULATE, to preserve compatibility with my_large_malloc(). Were MAP_POPULATE specified, the mmap() system call would be significantly slower, for example 18 seconds to reserve 64 GiB upfront.	7 months ago
Marko Mäkelä	0c18e5a292	MDEV-36760 log_t::append_prepare_wait(): Bogus assertion on write_lsn log_t::append_prepare_wait(): Do not attempt to read log_sys.write_lsn because it is not protected by log_sys.latch but by write_lock, which we cannot hold here. The assertion could fail if log_t::write_buf() is executing concurrently, and it has not yet executed log_write_buf() or updated log_sys.write_lsn. Fixes up commit `acd071f599` (MDEV-21923)	7 months ago
Brandon Nesterenko	a7278a3024	MDEV-36663: Testcase Fixup There were two issues with the test: 1. A race between a race_condition.inc and status variable, where the status variable Rpl_semi_sync_master_status could be ON before the semi-sync connection finished establishing, resulting in Rpl_semi_sync_master_clients showing 0 (instead of 1). To fix this, we simply instead wait for Rpl_semi_sync_master_clients to be 1 before proceeding. 2. Another race between a race_condition.inc and status variable, where the wait_condition waited on a process_list command of 'BINLOG DUMP' to disappear to infer the binlog dump thread was killed, to where we then verified semi-sync state was correct using status variables. However, the 'BINLOG DUMP' command is overridden with a killed status before the semi-sync tear-down happens, and thereby we could see invalid values. The fix for this is to change the wait_condition to instead wait for the connection with the replication user is gone, because that stays through the binlog dump thread tear-down life-cycle	7 months ago
Marko Mäkelä	791fcea1d7	bump the VERSION	7 months ago
Marko Mäkelä	84dd2437c5	MDEV-36760 log_t::append_prepare_wait(): Bogus assertion on write_lsn log_t::append_prepare_wait(): Do not attempt to read log_sys.write_lsn because it is not protected by log_sys.latch but by write_lock, which we cannot hold here. The assertion could fail if log_t::write_buf() is executing concurrently, and it has not yet executed log_write_buf() or updated log_sys.write_lsn. Fixes up commit `acd071f599` (MDEV-21923)	7 months ago
Daniel Black	60f046d7e6	MDEV-35009: Initialize affected_rows in SQL service The SQL service leaves the affected rows uninitialized. The initialization of the spider plugin that uses this service will fail under MSAN because there isn't an initialized value to return at the end of the query.	7 months ago
Yuchen Pei	c626715439	MDEV-35452 [fixup] fix spider/bugfix.perfschema view protocol mtr with --view-protocol creates new threads	7 months ago
Yuchen Pei	76e8e24b0b	Merge branch '10.5' into 10.6	7 months ago
Sergey Vojtovich	55ddfe1c95	MDEV-36684 - main.mdl_sync fails under valgrind (test for Bug#42643) Valgrind is single threaded and only changes threads as part of system calls or waits. Some busy loops were identified and fixed where the server assumes that some other thread will change the state, which will not happen with valgrind. Based on patch by Monty. Original patch introduced VALGRIND_YIELD, which emits pthread_yield() only in valgrind builds. However it was agreed that it is a good idea to emit yield() unconditionally, such that other affected schedulers (like SCHED_FIFO) benefit from this change. Also avoid pthread_yield() in favour of standard std::this_thread::yield().	7 months ago
Sergei Golubchik	83e0438f62	MDEV-36536 post-review changes that were apparently partially lost in a rebase	7 months ago
Oleksandr Byelkin	739578915f	Make the test more stable.	7 months ago
Sergei Petrunia	cafd22db79	Code cleanup in mark_common_columns(): nj_col_2 is non-NULL here Done after fix for MDEV-36592	7 months ago
Sergei Petrunia	5033da6ed6	Fix rocksdb_sys_vars.rocksdb_stats_level_basic test The minimum statistics level now is rocksdb::StatsLevel::kDisableAll. The default remains rocksdb::StatsLevel::kExceptHistogramOrTimers which is now 1 (it used to be 0).	7 months ago
Monty	d9cd4e1f75	MDEV-22250 InnoDB: Failing assertion: opt_no_lock during mariabackup --backup Ensure that backup_reset_alter_copy_lock() is called in case of rollback or error in mysql_inplace_alter_table() or copy_data_between_tables(). Other things: - Improved error from mariabackup when unexpected DDL operation is encountered. - Added assert if backup_ddl_log() is called in the wrong context.	7 months ago
Monty	1b934a387c	MDEV-36536 Add option to not collect statistics for long char/varchars This is needed to make it easy for users to automatically ignore long char and varchars when using ANALYZE TABLE PERSISTENT. These fields can cause problems as they will consume 'CHARACTERS * MAX_CHARACTER_LENGTH * 2 * number_of_rows' space on disk during analyze, which can easily be much bigger than the analyzed table. This commit adds a new user variable, analyze_max_length, default value 4G. Any field that is bigger than this in bytes, will be ignored by ANALYZE TABLE PERSISTENT unless it is specified in FOR COLUMNS(). While doing this patch, I noticed that we do not skip GEOMETRY columns from ANALYZE TABLE, like we do with BLOB. This should be fixed when merging to the 'main' branch. At the same time we should add a resonable default value for analyze_max_length, probably 1024, like we have for max_sort_length.	7 months ago
bsrikanth-mariadb	2263c8a1f7	MDEV-36461: Optimizer trace: remove join_execution node In non-EXPLAIN queries with subqueries, the trace was flooded with empty "join_execution":{} nodes. Now, they are gone. The "Range checked for each record" optimization still prints content into trace on join execution. Now, we wrap it into "range-checked-for-each-record" to delimit the invocations. This new object has fields "select_id" which corresponds to the outer query block, and the "loop" which corresponds to the inner query block iteration number. Additionally, the field "row_estimation" which itself is an object has "table", and "range_analysis" fields that were moved from the old "join_execution"'s steps array.	7 months ago
Brandon Nesterenko	0db5622c4f	MDEV-36663: Testcase Fixup There were two issues with the test: 1. A race between a race_condition.inc and status variable, where the status variable Rpl_semi_sync_master_status could be ON before the semi-sync connection finished establishing, resulting in Rpl_semi_sync_master_clients showing 0 (instead of 1). To fix this, we simply instead wait for Rpl_semi_sync_master_clients to be 1 before proceeding. 2. Another race between a race_condition.inc and status variable, where the wait_condition waited on a process_list command of 'BINLOG DUMP' to disappear to infer the binlog dump thread was killed, to where we then verified semi-sync state was correct using status variables. However, the 'BINLOG DUMP' command is overridden with a killed status before the semi-sync tear-down happens, and thereby we could see invalid values. The fix for this is to change the wait_condition to instead wait for the connection with the replication user is gone, because that stays through the binlog dump thread tear-down life-cycle	7 months ago

1 2 3 4 5 ...

201525 Commits (5cd982c51b63f6e18a7ebe0cec64e2b21da7089f) All Branches Search

201525 Commits (5cd982c51b63f6e18a7ebe0cec64e2b21da7089f)

All Branches