mariadb

Commit Graph

Author	SHA1	Message	Date
Oleksandr Byelkin	5018b998a7	return --help option	5 years ago
Marko Mäkelä	cfd3d70ccb	MDEV-22871: Remove pointer indirection for InnoDB hash_table_t hash_get_n_cells(): Remove. Access n_cells directly. hash_get_nth_cell(): Remove. Access array directly. hash_table_clear(): Replaced with hash_table_t::clear(). hash_table_create(), hash_table_free(): Remove. hash0hash.cc: Remove.	5 years ago
Vlad Lesin	205b0ce6ad	MDEV-22894: Mariabackup should not read [mariadb-client] option group from configuration files	5 years ago
Vlad Lesin	9bdf35e90f	MDEV-18215: mariabackup does not report unknown command line options MDEV-21298: mariabackup doesn't read from the [mariadbd] and [mariadbd-X.Y] server option groups from configuration files MDEV-21301: mariabackup doesn't read [mariadb-backup] option group in configuration file All three issues require to change the same code, that is why their fixes are joined in one commit. The fix is in invoking load_defaults_or_exit() and handle_options() for backup-specific groups separately from client-server groups to let the last handle_options() call fail on unknown backup-specific options. The order of options procesing is the following: 1) Load server groups and process server options, ignore unknown options 2) Load client groups and process client options, ignore unknown options 3) Load backup groups and process client-server options, exit on unknown option 4) Process --mysqld-args command line options, ignore unknown options New global flag my_handle_options_init_variables was added to have ability to invoke handle_options() for the same allowed options set several times without re-initialising previously set option values. --password value destroying is moved from option processing callback to mariabackup's handle_options() function to have ability to invoke server's handle_options() several times for the same possible allowed options set. Galera invokes wsrep_sst_mariabackup.sh with mysqld command line options to configure mariabackup as close to the server as possible. It is not known what server options are supported by mariabackup when the script is invoked. That is why new mariabackup option "--mysqld-args" is added, all unknown options that follow this option will be silently ignored. wsrep_sst_mariabackup.sh was also changed to: - use "--mysqld-args" mariabackup option to pass mysqld options, - remove deprecated innobackupex mode, - remove unsupported mariabackup options: --encrypt --encrypt-key --rebuild-indexes --rebuild-threads	5 years ago
mysqlonarm	dec3f8ca69	MDEV-22641: Provide SIMD optimized wrapper for zlib crc32() (#1558 ) Existing implementation used my_checksum (from mysys) for calculating table checksum and binlog checksum. This implementation was optimized for powerpc only and lacked SIMD implementation for x86 (using clmul) and ARM (using ACLE) instead used zlib-crc32. mariabackup had its own copy of the crc32 implementation using hardware optimized implementation only for x86 and lagged hardware based implementation for powerpc and ARM. Patch helps unifies all such calls and help aggregate all of them using an unified interface my_checksum(). Said unification also enables hardware optimized calls for all architecture viz. x86, ARM, POWERPC. Default always fallback to zlib crc32. Thanks to Daniel Black for reviewing, fixing and testing PowerPC changes. Thanks to Marko and Daniel for early code feedback.	6 years ago
Vlad Lesin	0f9bfcc323	MDEV-22554: "mariabackup --prepare" exits with code 0 even though innodb error is logged The fix is to set flag in ib::error::~error() and check it in mariabackup. ib::error::error() is replaced with ib::warn::warn() in AIO::linux_create_io_ctx() because of two reasons: 1) if we leave it as is, then mariabackup MTR tests will fail with --mem option, because Linux AIO can not be used on tmpfs, 2) when Linux AIO can not be initialized, InnoDB falls back to simulated AIO, so such sutiation is not fatal error, it should be treated as warning.	6 years ago
Eugene Kosov	89ff4176c1	MDEV-22437 make THR_THD* variable thread_local Now all access goes through _current_thd() and set_current_thd() functions. Some functions like THD::store_globals() can not fail now.	6 years ago
Vlad Lesin	0efe1971c6	MDEV-19347: Mariabackup does not honor ignore_db_dirs from server config. The solution is to read the system variable value on startup and to fill databases_exclude_hash. xb_load_list_string() became non-static and was reformatted. The system variable value is read and processed in get_mysql_vars(), which was also reformatted.	6 years ago
Thirunarayanan Balathandayuthapani	6bbc0eedc6	MDEV-22193 Avoid un-necessary page initialization during recovery - InnoDB is doing un-necessary redo log page initialisation during recovery and unnecessary traversal of redo log during last phase. This patch does the optimization of removing unnecessary redo log page initialisation and detects the memory exhaust earlier.	6 years ago
Vlad Lesin	5836191c8f	MDEV-21168: Active XA transactions stop slave from working after backup was restored. Optionally rollback prepared XA's on "mariabackup --prepare". The fix MUST NOT be ported on 10.5+, as MDEV-742 fix solves the issue for slaves.	6 years ago
Marko Mäkelä	aae3f921ad	Cleanup recv_sys: Move things to members recv_sys.recovery_on: Replaces recv_recovery_on. recv_sys_t::apply(): Replaces recv_apply_hashed_log_recs(). recv_sys_var_init(): Remove. recv_sys_t::recover_low(): Attempt to initialize a page based on buffered redo log records.	6 years ago
Vladislav Vaintroub	1b58ef7d3f	Build cleanups. Fix clang-cl built	6 years ago
Marko Mäkelä	f224525204	MDEV-21907: InnoDB: Enable -Wconversion on clang and GCC The -Wconversion in GCC seems to be stricter than in clang. GCC at least since version 4.4.7 issues truncation warnings for assignments to bitfields, while clang 10 appears to only issue warnings when the sizes in bytes rounded to the nearest integer powers of 2 are different. Before GCC 10.0.0, -Wconversion required more casts and would not allow some operations, such as x<<=1 or x+=1 on a data type that is narrower than int. GCC 5 (but not GCC 4, GCC 6, or any later version) is complaining about x\|=y even when x and y are compatible types that are narrower than int. Hence, we must rewrite some x\|=y as x=static_cast<byte>(x\|y) or similar, or we must disable -Wconversion. In GCC 6 and later, the warning for assigning wider to bitfields that are narrower than 8, 16, or 32 bits can be suppressed by applying a bitwise & with the exact bitmask of the bitfield. For older GCC, we must disable -Wconversion for GCC 4 or 5 in such cases. The bitwise negation operator appears to promote short integers to a wider type, and hence we must add explicit truncation casts around them. Microsoft Visual C does not allow a static_cast to truncate a constant, such as static_cast<byte>(1) truncating int. Hence, we will use the constructor-style cast byte(~1) for such cases. This has been tested at least with GCC 4.8.5, 5.4.0, 7.4.0, 9.2.1, 10.0.0, clang 9.0.1, 10.0.0, and MSVC 14.22.27905 (Microsoft Visual Studio 2019) on 64-bit and 32-bit targets (IA-32, AMD64, POWER 8, POWER 9, ARMv8).	6 years ago
Sergei Golubchik	7c58e97bf6	perfschema memory related instrumentation changes	6 years ago
Eugene Kosov	2b8b85bd0a	fix use-after-free	6 years ago
Marko Mäkelä	4383897a01	MDEV-14425 preparation: Remove log_header_read() The function log_header_read() was only used during server startup, and it will mostly be used only for reading checkpoint information from pre-MDEV-14425 format redo log files. Let us replace the function with more direct calls, so that it is clearer what is going on. It is not strictly necessary to hold any mutex during this operation, and because there will be only a limited number of operations during early server startup, it is not necessary to increment any I/O counters.	6 years ago
Marko Mäkelä	8511f04fdb	Cleanup: Remove srv_start_lsn Most of the time, we can refer to recv_sys.recovered_lsn.	6 years ago
Eugene Kosov	9ef2d29ff4	MDEV-14425 deprecate and ignore innodb_log_files_in_group Now there can be only one log file instead of several which logically work as a single file. Possible names of redo log files: ib_logfile0, ib_logfile101 (for just created one) innodb_log_fiels_in_group: value of this variable is not used by InnoDB. Possible values are still 1..100, to not break upgrade LOG_FILE_NAME: add constant of value "ib_logfile0" LOG_FILE_NAME_PREFIX: add constant of value "ib_logfile" get_log_file_path(): convenience function that returns full path of a redo log file SRV_N_LOG_FILES_MAX: removed srv_n_log_files: we can't remove this for compatibility reasons, but now server doesn't use this variable log_sys_t::file::fd: now just one, not std::vector log_sys_t::log_capacity: removed word 'group' find_and_check_log_file(): part of logic from huge srv_start() moved here recv_sys_t::files: file descriptors of redo log files. There can be several of those in case we're upgrading from older MariaDB version. recv_sys_t::remove_extra_log_files: whether to remove ib_logfile{1,2,3...} after successfull upgrade. recv_sys_t::read(): open if needed and read from one of several log files recv_sys_t::files_size(): open if needed and return files count redo_file_sizes_are_correct(): check that redo log files sizes are equal. Just to log an error for a user. Corresponding check was moved from srv0start.cc namespace deprecated: put all deprecated variables here to prevent usage of it by us, developers	6 years ago
Marko Mäkelä	f8a9f90667	MDEV-12353: Remove support for crash-upgrade We tighten some assertions regarding dict_index_t::is_dummy and crash recovery, now that redo log processing will no longer create dummy objects.	6 years ago
Marko Mäkelä	7ae21b18a6	MDEV-12353: Change the redo log encoding log_t::FORMAT_10_5: physical redo log format tag log_phys_t: Buffered records in the physical format. The log record bytes will follow the last data field, making use of alignment padding that would otherwise be wasted. If there are multiple records for the same page, also those may be appended to an existing log_phys_t object if the memory is available. In the physical format, the first byte of a record identifies the record and its length (up to 15 bytes). For longer records, the immediately following bytes will encode the remaining length in a variable-length encoding. Usually, a variable-length-encoded page identifier will follow, followed by optional payload, whose length is included in the initially encoded total record length. When a mini-transaction is updating multiple fields in a page, it can avoid repeating the tablespace identifier and page number by setting the same_page flag (most significant bit) in the first byte of the log record. The byte offset of the record will be relative to where the previous record for that page ended. Until MDEV-14425 introduces a separate file-level log for redo log checkpoints and file operations, we will write the file-level records in the page-level redo log file. The record FILE_CHECKPOINT (which replaces MLOG_CHECKPOINT) will be removed in MDEV-14425, and one sequential scan of the page recovery log will suffice. Compared to MLOG_FILE_CREATE2, FILE_CREATE will not include any flags. If the information is needed, it can be parsed from WRITE records that modify FSP_SPACE_FLAGS. MLOG_ZIP_WRITE_STRING: Remove. The record was only introduced temporarily as part of this work, before being replaced with WRITE (along with MLOG_WRITE_STRING, MLOG_1BYTE, MLOG_nBYTES). mtr_buf_t::empty(): Check if the buffer is empty. mtr_t::m_n_log_recs: Remove. It suffices to check if m_log is empty. mtr_t::m_last, mtr_t::m_last_offset: End of the latest m_log record, for the same_page encoding. page_recv_t::last_offset: Reflects mtr_t::m_last_offset. Valid values for last_offset during recovery should be 0 or above 8. (The first 8 bytes of a page are the checksum and the page number, and neither are ever updated directly by log records.) Internally, the special value 1 indicates that the same_page form will not be allowed for the subsequent record. mtr_t::page_create(): Take the block descriptor as parameter, so that it can be compared to mtr_t::m_last. The INIT_INDEX_PAGE record will always followed by a subtype byte, because same_page records must be longer than 1 byte. trx_undo_page_init(): Combine the writes in WRITE record. trx_undo_header_create(): Write 4 bytes using a special MEMSET record that includes 1 bytes of length and 2 bytes of payload. flst_write_addr(): Define as a static function. Combine the writes. flst_zero_both(): Replaces two flst_zero_addr() calls. flst_init(): Do not inline the function. fsp_free_seg_inode(): Zerofill the whole inode. fsp_apply_init_file_page(): Initialize FIL_PAGE_PREV,FIL_PAGE_NEXT to FIL_NULL when using the physical format. btr_create(): Assert !page_has_siblings() because fsp_apply_init_file_page() must have been invoked. fil_ibd_create(): Do not write FILE_MODIFY after FILE_CREATE. fil_names_dirty_and_write(): Remove the parameter mtr. Write the records using a separate mini-transaction object, because any FILE_ records must be at the start of a mini-transaction log. recv_recover_page(): Add a fil_space_t* parameter. After applying log to the a ROW_FORMAT=COMPRESSED page, invoke buf_zip_decompress() to restore the uncompressed page. buf_page_io_complete(): Remove the temporary hack to discard the uncompressed page of a ROW_FORMAT=COMPRESSED page. page_zip_write_header(): Remove. Use mtr_t::write() or mtr_t::memset() instead, and update the compressed page frame separately. trx_undo_header_add_space_for_xid(): Remove. trx_undo_seg_create(): Perform the changes that were previously made by trx_undo_header_add_space_for_xid(). btr_reset_instant(): New function: Reset the table to MariaDB 10.2 or 10.3 format when rolling back an instant ALTER TABLE operation. page_rec_find_owner_rec(): Merge with the only callers. page_cur_insert_rec_low(): Combine writes by using a local buffer. MEMMOVE data from the preceding record whenever feasible (copying at least 3 bytes). page_cur_insert_rec_zip(): Combine writes to page header fields. PageBulk::insertPage(): Issue MEMMOVE records to copy a matching part from the preceding record. PageBulk::finishPage(): Combine the writes to the page header and to the sparse page directory slots. mtr_t::write(): Only log the least significant (last) bytes of multi-byte fields that actually differ. For updating FSP_SIZE, we must always write all 4 bytes to the redo log, so that the fil_space_set_recv_size() logic in recv_sys_t::parse() will work. mtr_t::memcpy(), mtr_t::zmemcpy(): Take a pointer argument instead of a numeric offset to the page frame. Only log the last bytes of multi-byte fields that actually differ. In fil_space_crypt_t::write_page0(), we must log also any unchanged bytes, so that recovery will recognize the record and invoke fil_crypt_parse(). Future work: MDEV-21724 Optimize page_cur_insert_rec_low() redo logging MDEV-21725 Optimize btr_page_reorganize_low() redo logging MDEV-21727 Optimize redo logging for ROW_FORMAT=COMPRESSED	6 years ago
Marko Mäkelä	67c76704a8	MDEV-12353: Remove MLOG_INDEX_LOAD (innodb_log_optimize_ddl) NOTE: This may break crash-upgrade from a dataset that was created with innodb_log_optimize_ddl=ON. Also due to ROW_FORMAT=COMPRESSED pages, it will be easiest to disallow crash-upgrade. It would be more robust to disable the MDEV-12699 logic when crash-upgrading from old redo log format. log_optimized_ddl_op: Remove. fil_space_t::enable_lsn, file_name_t::enable_lsn: Remove. ddl_tracker_t::optimized_ddl: Remove. TODO: Remove ddl_tracker	6 years ago
Marko Mäkelä	1a6f708ec5	MDEV-15058: Deprecate and ignore innodb_buffer_pool_instances Our benchmarking efforts indicate that the reasons for splitting the buf_pool in commit `c18084f71b` have mostly gone away, possibly as a result of mysql/mysql-server@ce6109ebfdedfdf185e391a0c97dc6d33867ed78 or similar work. Only in one write-heavy benchmark where the working set size is ten times the buffer pool size, the buf_pool->mutex would be less contended with 4 buffer pool instances than with 1 instance, in buf_page_io_complete(). That contention could be alleviated further by making more use of std::atomic and by splitting buf_pool_t::mutex further (MDEV-15053). We will deprecate and ignore the following parameters: innodb_buffer_pool_instances innodb_page_cleaners There will be only one buffer pool and one page cleaner task. In a number of INFORMATION_SCHEMA views, columns that indicated the buffer pool instance will be removed: information_schema.innodb_buffer_page.pool_id information_schema.innodb_buffer_page_lru.pool_id information_schema.innodb_buffer_pool_stats.pool_id information_schema.innodb_cmpmem.buffer_pool_instance information_schema.innodb_cmpmem_reset.buffer_pool_instance	6 years ago
Eugene Kosov	691c691adc	clean up redo log main change: rename first redo log without file close second change: use os_offset_t to represent offset in a file third change: fix log texts	6 years ago
Marko Mäkelä	50324ce624	MDEV-21351 Replace recv_sys.heap with list of buf_block_t InnoDB crash recovery used a special type of mem_heap_t that allocates backing store from the buffer pool. That incurred a significant overhead, leading to underutilization of memory, and limiting the maximum contiguous allocated size of a log record. recv_sys_t::blocks: A linked list of buf_block_t that are allocated by buf_block_alloc() for redo log records. Replaces recv_sys_t::heap. We repurpose buf_block_t::unzip_LRU for linking the elements. recv_sys_t::max_log_blocks: Renamed from recv_n_pool_free_frames. recv_sys_t::max_blocks(): Accessor for max_log_blocks. recv_sys_t::alloc(): Allocate memory from the current recv_sys_t::blocks element, or allocate another block. In debug builds, various free() member functions must be invoked, because we repurpose buf_page_t::buf_fix_count for tracking allocations. recv_sys_t::free_corrupted_page(): Renamed from recv_recover_corrupt_page() recv_sys_t::is_memory_exhausted(): Renamed from recv_sys_heap_check() recv_sys_t::pages and its elements are allocated directly by the system memory allocator. recv_parse_log_recs(): Remove the parameter available_memory. We rename some variables 'store_to_hash' to 'store', because recv_sys.pages is not actually a hash table. This is joint work with Thirunarayanan Balathandayuthapani.	6 years ago
Julius Goryavsky	982294ac16	MDEV-17601: MariaDB Galera does not expect 'mbstream' as streamfmt Setting "streamfmt=mbstream" in the "[sst]" section causes SST to fail because the format automatically switches to 'tar' by default (insead of mbstream). To fix this, we need to add mbstream to the list of valid values for the format, making it synonymous with xbstream. This must be done both in the SST script and when parsing the options of the corresponding utilities.	6 years ago
Eugene Kosov	e9de6386ad	MDEV-18115 remove now unneeded constraint log_group_max_size: is not needed because redo log do not use fil_io() now	6 years ago
Eugene Kosov	562c037b48	MDEV-18115 Remove dummy tablespace for the redo log Redo log subsystem was decoupled from tablespace subsystem. It now manages file descriptors for redo log files by itself. FIL_TYPE_LOG: removed, code in various places was simplified SRV_LOG_SPACE_FIRST_ID: renamed to SRV_SPACE_ID_UPPER_BOUND to better match its purpose. Code in various places was simplified fil_n_log_flushes: replaced with log_sys::flushes fil_n_pending_log_flushes: replaced with log_sys::pending_flushes log_t::files::files: redo log file descriptors log_t::files::file_names: redo log file names log_t::files::set_file_names(): set file names without opening them log_t::files::open_files(): opens redo log files log_t::files::read(): treats several files as one big log_t::files::write(): treats several files as one big log_t::files::fsync(): flushes page cache to disk log_t::files::close_files(): closes redo log files fil_open_log_and_system_tablespace_files(): renamed to fil_open_system_tablespace_files() and obviously it now doesn't open redo log files global files[1000]: removed. Why it was needed at all?	6 years ago
Thirunarayanan Balathandayuthapani	bba59abb03	MDEV-19176 Reduce the memory usage during recovery - Moved the recv_sys->heap memory condition inside recv_parse_log_recs(). So that, InnoDB can mark the status as STORE_NO earlier. - InnoDB uses one third of buffer pool chunk size for reading the redo log records. In that case, we can avoid the scenario where buffer ran out of memory issue during recovery.	6 years ago
Vlad Lesin	beec9c0e19	MDEV-21255: Deadlock of parallel slave and mariabackup (with failed log copy thread) mariabackup hangs waiting until innodb redo log thread read log till certain LSN, and it waits under FTWRL. If there is redo log read error in the thread, it is finished, and main thread knows nothing about it, what leads to hanging. As it hangs under FTWRL, slave threads on server side can be blocked due to MDL lock conflict. The fix is to finish mariabackup with error message on innodb redo log read failure.	6 years ago
Vladislav Vaintroub	202a62deb0	MDEV-11345 Compile english error messages into mysqld executable. Simplify loading messages into mariabackup. Do the same as server does We're forcing english, so there is no attempt to load errmsg.sys	6 years ago
Marko Mäkelä	42a4ae54c2	MDEV-21225 Remove ut_align() and use aligned_malloc() Before commit `90c52e5291` introduced aligned_malloc(), InnoDB always used a pattern of over-allocating memory and invoking ut_align() to guarantee the desired alignment. It is cleaner to invoke aligned_malloc() and aligned_free() directly. ut_align(): Remove. In assertions, ut_align_down() can be used instead.	6 years ago
Faustin Lammler	2df2238cb8	Lintian complains on spelling error The lintian check complains on spelling error: https://salsa.debian.org/mariadb-team/mariadb-10.3/-/jobs/95739	7 years ago
Marko Mäkelä	312569e2fd	MDEV-21132 Remove buf_page_t::newest_modification At each mini-transaction commit, the log sequence number of the mini-transaction must be written to each modified page, so that it will be available in the FIL_PAGE_LSN field when the page is being read in crash recovery. InnoDB was unnecessarily allocating redundant storage for the field, in buf_page_t::newest_modification. Let us access FIL_PAGE_LSN directly. Furthermore, on ALTER TABLE...IMPORT TABLESPACE, let us write 0 to FIL_PAGE_LSN instead of using log_sys.lsn. buf_flush_init_for_writing(), buf_flush_update_zip_checksum(), fil_encrypt_buf_for_full_crc32(), fil_encrypt_buf(), fil_space_encrypt(): Remove the parameter lsn. buf_page_get_newest_modification(): Merge with the only caller. buf_tmp_reserve_compression_buf(), buf_tmp_page_encrypt(), buf_page_encrypt(): Define static in the same compilation unit with the only caller. PageConverter::m_current_lsn: Remove. Write 0 to FIL_PAGE_LSN on ALTER TABLE...IMPORT TABLESPACE.	6 years ago
Vladislav Vaintroub	5e62b6a5e0	MDEV-16264 Use threadpool for Innodb background work. Almost all threads have gone - the "ticking" threads, that sleep a while then do some work) (srv_monitor_thread, srv_error_monitor_thread, srv_master_thread) were replaced with timers. Some timers are periodic, e.g the "master" timer. - The btr_defragment_thread is also replaced by a timer , which reschedules it self when current defragment "item" needs throttling - the buf_resize_thread and buf_dump_threads are substitutes with tasks Ditto with page cleaner workers. - purge workers threads are not tasks as well, and purge cleaner coordinator is a combination of a task and timer. - All AIO is outsourced to tpool, Innodb just calls thread_pool::submit_io() and provides the callback. - The srv_slot_t was removed, and innodb_debug_sync used in purge is currently not working, and needs reimplementation.	6 years ago
Marko Mäkelä	78d0d2cdc5	Cleanup: Remove mach_read_ulint() The function mach_read_ulint() is a wrapper for the lower-level functions mach_read_from_1(), mach_read_from_2(), mach_read_from_8(). Invoke those functions directly, for better readability of the code. mtr_t::read_ulint(), mtr_read_ulint(): Remove. Yes, we will lose the ability to assert that the read is covered by the mini-transaction. We would still check that on writes, and any writes that wrongly bypass mini-transaction logging would likely be caught by stress testing with Mariabackup.	6 years ago
Marko Mäkelä	5164f8c206	Fix GCC 9.2.1 -Wstringop-truncation dict_table_rename_in_cache(): Use strcpy() instead of strncpy(), because they are known to be equivalent in this case (the length of old_name was already validated). mariabackup: Invoke strncpy() with one less than the buffer size, and explicitly add NUL as the last byte of the buffer.	6 years ago
Marko Mäkelä	88cdfc5c7d	MDEV-20487 Set innodb_adaptive_hash_index=OFF by default Based on the performance testing that was conducted in MDEV-17492, the InnoDB adaptive hash index could only help performance in specific, almost-read-only workloads. It could slow down all kinds of workloads (especially DROP TABLE, TRUNCATE TABLE, ALTER TABLE, or DROP INDEX operations), and it can become corrupted, causing crashes (such as MDEV-18815, MDEV-20203) and possibly data corruption. Furthermore, the adaptive hash index consumes space from the InnoDB buffer pool, which could hurt performance when the working set would almost fit in the buffer pool. Given all this, it is best to disable the adaptive hash index by default.	6 years ago
Sergei Golubchik	f217612fad	MDEV-12684 Show what config file a sysvar got a value from change get_one_option() prototype to pass the filename and not to pass the redundant optid.	6 years ago
Vlad Lesin	edda2fd149	MDEV-20703: mariabackup creates binlog files in server binlog directory on --prepare --export step When "--export" mariabackup option is used, mariabackup starts the server in bootstrap mode to generate .cfg files for the certain innodb tables. The started instance of the server reads options from the file, pointed out in "--defaults-file" mariabackup option. If the server uses the same config file as mariabackup, and binlog is switched on in that config file, then "mariabackup --prepare --export" will create binary log files in the server's binary log directory, what can cause issues. The fix is to add "--skip-log-bin" in mysld options when the server is started to generate .cfg files.	6 years ago
Simon Lipp	c0db3fe6da	MDEV-18438 Don't stream xtrabackup_info of extra-lsndir	7 years ago
Vlad Lesin	d39d5dd2bc	MDEV-20060: Failing assertion: srv_log_file_size <= 512ULL << 30 while preparing backup The general reason why innodb redo log file is limited by 512G is that log_block_convert_lsn_to_no() returns value limited by 1G. But there is no need to have unique log block numbers in log group. The fix removes 512G limit and limits log group size by (uint32_t maximum value) * (minimum page size), which, in turns, can be removed if fil_io() is no longer used for innodb redo log io.	6 years ago
Alexander Barkov	3e7e87ddcc	MDEV-19897 Rename source code variable names from utf8 to utf8mb3	6 years ago
Vlad Lesin	6d2b236568	MDEV-19865: AddressSanitizer error in open_or_create_log_file() in mariabackup Decrease array on stack in open_or_create_log_file() to avoid stack overflow.	6 years ago
Georg Richter	d13080133f	MDEV-14101 Provide an option to select TLS protocol version Server and command line tools now support option --tls_version to specify the TLS version between client and server. Valid values are TLSv1.0, TLSv1.1, TLSv1.2, TLSv1.3 or a combination of them. E.g. --tls_version=TLSv1.3 --tls_version=TLSv1.2,TLSv1.3 In case there is a gap between versions, the lowest version will be used: --tls_version=TLSv1.1,TLSv1.3 -> Only TLSv1.1 will be available. If the used TLS library doesn't support the specified TLS version, it will use the default configuration. Limitations: SSLv3 is not supported. The default configuration doesn't support TLSv1.0 anymore. TLSv1.3 protocol currently is only supported by OpenSSL 1.1.0 (client and server) and GnuTLS 3.6.5 (client only). Overview of TLS implementations and protocols Server: +-----------+-----------------------------------------+ \| Library \| Supported TLS versions \| +-----------+-----------------------------------------+ \| WolfSSL \| TLSv1.1, TLSv1,2 \| +-----------+-----------------------------------------+ \| OpenSSL \| (TLSv1.0), TLSv1.1, TLSv1,2, TLSv1.3 \| +-----------+-----------------------------------------+ \| LibreSSL \| (TLSv1.0), TLSv1.1, TLSv1,2, TLSv1.3 \| +-----------+-----------------------------------------+ Client (MariaDB Connector/C) +-----------+-----------------------------------------+ \| Library \| Supported TLS versions \| +-----------+-----------------------------------------+ \| GnuTLS \| (TLSv1.0), TLSv1.1, TLSv1.2, TLSv1.3 \| +-----------+-----------------------------------------+ \| Schannel \| (TLSv1.0), TLSv1.1, TLSv1.2 \| +-----------+-----------------------------------------+ \| OpenSSL \| (TLSv1.0), TLSv1.1, TLSv1,2, TLSv1.3 \| +-----------+-----------------------------------------+ \| LibreSSL \| (TLSv1.0), TLSv1.1, TLSv1,2, TLSv1.3 \| +-----------+-----------------------------------------+	6 years ago
Monty	05f8120d0e	Fixed compiler warning Wrong compiler warning from GCC: ‘snprintf’ output 2 or more bytes (assuming 4001) into a destination of size 4000	7 years ago
Marko Mäkelä	5d2619b693	MDEV-19584 Allocate recv_sys statically There is only one InnoDB crash recovery subsystem. Allocating recv_sys statically removes one level of pointer indirection and makes code more readable, and removes the awkward initialization of recv_sys->dblwr. recv_sys_t::create(): Replaces recv_sys_init(). recv_sys_t::debug_free(): Replaces recv_sys_debug_free(). recv_sys_t::close(): Replaces recv_sys_close(). recv_sys_t::add(): Replaces recv_add_to_hash_table(). recv_sys_t::empty(): Replaces recv_sys_empty_hash().	7 years ago
Vlad Lesin	bff9b8026b	MDEV-14192: mariabackup.incremental_backup failed in buildbot with Failing assertion: byte_offset % OS_FILE_LOG_BLOCK_SIZE == 0 In some cases it's possible that InnoDB redo log file header is re-written so, that checkpoint lsn and checkpoint lsn offset are updated, but checkpoint number stays the same. The fix is to re-read redo log header if at least one of those three parametes is changed at backup start. Repeat the logic of log_group_checkpoint() on choosing InnoDB checkpoint info field on backup start. This does not influence backup correctness, but simplifies bugs analysis.	7 years ago
Marko Mäkelä	893472d005	MDEV-19570 Deprecate and ignore innodb_undo_logs, remove innodb_rollback_segments The option innodb_rollback_segments was deprecated already in MariaDB Server 10.0. Its misleadingly named replacement innodb_undo_logs is of very limited use. It makes sense to always create and use the maximum number of rollback segments. Let us remove the deprecated parameter innodb_rollback_segments and deprecate&ignore the parameter innodb_undo_logs (to be removed in a later major release). This work involves some cleanup of InnoDB startup. Similar to other write operations, DROP TABLE will no longer be allowed if innodb_force_recovery is set to a value larger than 3.	7 years ago
Marko Mäkelä	1a6f470464	MDEV-19544 Remove innodb_locks_unsafe_for_binlog The transaction isolation levels READ COMMITTED and READ UNCOMMITTED should behave similarly to the old deprecated setting innodb_locks_unsafe_for_binlog=1, that is, avoid acquiring gap locks. row_search_mvcc(): Reduce the scope of some variables, and clean up the initialization and use of the variable set_also_gap_locks.	7 years ago
Marko Mäkelä	47cede646b	MDEV-19543 Deprecate and ignore innodb_log_checksums The parameter innodb_log_checksums that was introduced in MariaDB 10.2.2 via mysql/mysql-server@af0acedd885eb7103e319f79d25fda7386ef1506 does not make much sense. The original motivation of introducing this parameter (initially called innodb_log_checksum_algorithm in mysql/mysql-server@22ba38218e1d76c24f69b5a5595ad3bf5933acb0) was that the InnoDB redo log used the slow and insecure innodb algorithm. With hardware or SIMD accelerated CRC-32C, there should be no reason to allow checksums to be disabled on the redo log. The parameter innodb_encrypt_log already implies innodb_log_checksums=ON. Let us deprecate the parameter innodb_log_checksums and always compute redo log checksums, even if innodb_log_checksums=OFF is specified. An upgrade from MariaDB 10.2.2 or later will only be possible after using the default value innodb_log_checksums=ON. If the non-default value innodb_log_checksums=OFF was in effect when the server was shut down, a log block checksum mismatch will be reported and the upgraded server will fail to start up.	7 years ago

1 2 3 4 5 ...

395 Commits (5018b998a76b99a30efe96ca007a95a5a1c83bb5)