mariadb

Tree: 420f8e24ab

Author	SHA1	Message	Date
Vladislav Vaintroub	30b1fbda28	MDEV-14746 : read [mariabackup] section from config file.	8 years ago
Vladislav Vaintroub	054051bb76	MDEV-14150 - backup should fail early if rsync is missing check availability of rsync utility early , when starting backup with --rsync. Fail if it is not there.	8 years ago
Vladislav Vaintroub	e30cdb68e5	mariabackup : improve help text	8 years ago
Marko Mäkelä	2281fcf38a	Follow-up fix to MDEV-14952 for Mariabackup innodb_init_param(): Initialize btr_ahi_parts=1 for Mariabackup. btr_search_enabled: Let the adaptive hash index be disabled in Mariabackup. This would potentially only matter during --export, and --export performs a table scan, not many index lookups.	8 years ago
Marko Mäkelä	16d308e21d	MDEV-14874 innodb_encrypt_log corrupts the log when the LSN crosses 32-bit boundary This bug affects both writing and reading encrypted redo log in MariaDB 10.1, starting from version 10.1.3 which added support for innodb_encrypt_log. That is, InnoDB crash recovery and Mariabackup will sometimes fail when innodb_encrypt_log is used. MariaDB 10.2 or Mariabackup 10.2 or later versions are not affected. log_block_get_start_lsn(): Remove. This function would cause trouble if a log segment that is being read is crossing a 32-bit boundary of the LSN, because this function does not allow the most significant 32 bits of the LSN to change. log_blocks_crypt(), log_encrypt_before_write(), log_decrypt_after_read(): Add the parameter "lsn" for the start LSN of the block. log_blocks_encrypt(): Remove (unused function).	8 years ago
Monty	83a8b0e9d3	Fixed compiler warning	8 years ago
Sergey Vojtovich	2cd3169113	MDEV-14265 - RPMLint warning: shared-lib-calls-exit find_type_or_exit() client helper did exit(1) on error, exit(1) moved to clients. mysql_read_default_options() did exit(1) on error, error is passed through and handled now. my_str_malloc_default() did exit(1) on error, replaced my_str_ allocator functions with normal my_malloc()/my_realloc()/my_free(). sql_connect.cc did many exit(1) on hash initialisation failure. Removed error check since my_hash_init() never fails. my_malloc() did exit(1) on error. Replaced with abort(). my_load_defaults() did exit(1) on error, replaced with return 2. my_load_defaults() still does exit(0) when invoked with --print-defaults.	8 years ago
Vladislav Vaintroub	f32063c513	MDEV-13620 - improve help message for 'plugin-dir' and 'plugin-load' options.	8 years ago
Marko Mäkelä	a9a4089175	Plug a small memory leak in mariabackup --backup	8 years ago
Monty	60df17e95a	Remove compiler warnings	8 years ago
Vladislav Vaintroub	f59a1826f8	MDEV-14536 : during backup, retry read of log blocks, if there is (possibly intermittent) checksum mismatch.	8 years ago
Vladislav Vaintroub	bf6d11c4d6	MDEV-14536 : In mariabackup, reread redo log blocks , if checksum mismatch is detected. The checksum mismatch can be due to partial write, thus retry the read	8 years ago
Marko Mäkelä	23d2dae5f0	Fix some integer type mismatch warnings	8 years ago
Vladislav Vaintroub	414d3a3e17	Fix warning.	8 years ago
Marko Mäkelä	6979d20426	MDEV-14499 Mariabackup 10.2 fails to back up a multi-file InnoDB system tablespace When Mariabackup is invoked on an instance that uses a multi-file InnoDB system tablespace, it may fail to other files of the system tablespace than the first one. This was revealed by the MDEV-14447 test case. The offending code is assuming that the first page of each data file is page 0. But, in multi-file system tablespaces that is not the case. xb_fil_cur_open(): Instead of re-reading the first page of the file, rely on the fil_space_t metadata that already exists in memory. xb_get_space_flags(): Remove.	8 years ago
Vladislav Vaintroub	316f0d8fe3	MDEV-14447 mariabackup incremental incorrectly extends system tablespace for multi-file innodb_data_file_path. Use fil_extend_space_to_desired_size() to correctly extend system tablespace. Make sure to get tablespace size from the first tablespace part.	8 years ago
Marko Mäkelä	c19ef508b8	InnoDB: Remove ut_snprintf() and the use of my_snprintf(); use snprintf()	8 years ago
Marko Mäkelä	909cdafd35	MDEV-13496 Use "mariabackup" rather than "xtrabackup" in console output Replace the remaining "xtrabackup:" references in the code.	8 years ago
Vladislav Vaintroub	72407e544e	MDEV-13496 Use "mariabackup" rather than "xtrabackup" in console output	8 years ago
Vladislav Vaintroub	125ce6f82f	MDEV-14102 restore --remove-original options for mariabackup	8 years ago
Sergei Golubchik	2aa51f528f	Various compier warnings gcc 5.4 and 7.1, Debug and Release builds	8 years ago
Marko Mäkelä	9ee840cd0a	mariabackup: Properly call os_thread_exit() with detach=true There is no call to os_thread_join(), so we should detach the thread handles in order to avoid any resource leaks.	8 years ago
Marko Mäkelä	1b478a7aba	MDEV-13311 Presence of old logs in 10.2.7 will corrupt restored instance (change in behavior) Mariabackup 10.2.7 would delete the redo log files after a successful --prepare operation. If the user is manually copying the prepared files instead of using the --copy-back option, it could happen that some old redo log file would be preserved in the restored location. These old redo log files could cause corruption of the restored data files when the server is started up. We prevent this scenario by creating a "poisoned" redo log file ib_logfile0 at the end of the --prepare step. The poisoning consists of simply truncating the file to an empty file. InnoDB will refuse to start up on an empty redo log file. copy_back(): Delete all redo log files in the target if the source file ib_logfile0 is empty. (Previously we did this if the source file is missing.) SRV_OPERATION_RESTORE_EXPORT: A new variant of SRV_OPERATION_RESTORE when the --export option is specified. In this mode, we will keep deleting all redo log files, instead of truncating the first one. delete_log_files(): Add a parameter for the first file to delete, to be passed as 0 or 1. innobase_start_or_create_for_mysql(): In mariabackup --prepare, tolerate an empty ib_logfile0 file. Otherwise, require the first redo log file to be longer than 4 blocks (2048 bytes). Unless --export was specified, truncate the first log file at the end of --prepare.	8 years ago
Vladislav Vaintroub	fe18e6b064	MDEV-13822 mariabackup incremental prepare incorrectly sets file size. MDEV-13310 Preparing an incremental backup twice can corrupt data	8 years ago
Vladislav Vaintroub	b731a5bcf2	Innodb : Refactor os_file_set_size() to be compatible 10.1 The last parameter to this function is now,"bool is_sparse", like in 10.1 rather than the unused/useless "bool is_readonly", merged from MySQL 5.7 Like in 10.1, this function now supports sparse files, and efficient platform specific mechanisms for file extension os_file_set_size() is now consistenly used in all places where innodb files are extended.	8 years ago
Vladislav Vaintroub	bb3f4fbb59	MDEV-13310 Preparing an incremental backup twice can corrupt data Remove .delta file after it was successfully applied	8 years ago
Vladislav Vaintroub	0f8295d7d5	MDEV-13822 mariabackup incremental prepare incorrectly sets file size. Fix incremental prepare to change file size while applying delta file, if delta file contains page 0 with the new size.	8 years ago
Vladislav Vaintroub	eba44874ca	MDEV-13844 : Fix Windows warnings. Fix DBUG_PRINT. - Fix win64 pointer truncation warnings (usually coming from misusing 0x%lx and long cast in DBUG) - Also fix printf-format warnings Make the above mentioned warnings fatal. - fix pthread_join on Windows to set return value.	8 years ago
Vladislav Vaintroub	d1253e19a1	Fix compilation in mariabackup Compilation got confused about 2 wsrep.h headers in include path Rename backup's wsrep.h to backup_wsrep.h to fixO	8 years ago
Vladislav Vaintroub	ad17e8e518	MDEV-13821 : mariabackup sometimes could lose ib_logf(FATAL) messages, The messages are getting lost because they are written with fprintf() but without fflush(), so abort() would lose buffered text. Applied fix from 10.2, which makes in_logf() use sql_print_information(), which in turn does proper flush after each message	8 years ago
Vladislav Vaintroub	74f677fcc2	MDEV-13802 mariabackup --lock-ddl-per-table fails when table names contain backticks use ut_get_name() for formatting database/table names.	8 years ago
Vladislav Vaintroub	bb7ab40521	mariabackup : Fix enumerate_ibd_files() to include .isl	8 years ago
Marko Mäkelä	66a09bd6ab	MDEV-13318 Crash recovery failure after the server is killed during innodb_encrypt_log startup This fixes several InnoDB bugs related to innodb_encrypt_log and two Mariabackup --backup bugs. log_crypt(): Properly derive the initialization vector from the start LSN of each block. Add a debug assertion. log_crypt_init(): Note that the function should only be used when creating redo log files and that the information is persisted in the checkpoint pages. xtrabackup_copy_log(): Validate data_len. xtrabackup_backup_func(): Always use the chosen checkpoint buffer. log_group_write_buf(), log_write_up_to(): Only log_crypt() the redo log payload, not the padding bytes. innobase_start_or_create_for_mysql(): Do not invoke log_crypt_init() or initiate a redo log checkpoint. recv_find_max_checkpoint(): Return the contents of LOG_CHECKPOINT_NO to xtrabackup_backup_func() in log_sys->next_checkpoint_no.	8 years ago
Vladislav Vaintroub	31774f0ede	MDEV-13563 lock DDL for mariabackup in 10.2 Implement lock-ddl-per-table option that locks tables before it is copied to backup, and helds the lock until backup finished The "DDL-lock" itself is implemented as "SELECT * from <table> LIMIT 0", inside a transaction, and "COMMIT" of this transaction is the DDL-unlock.	8 years ago
Vladislav Vaintroub	18e17f134c	Windows : Fix MTR's misuse of servers --console parameter	8 years ago
Vladislav Vaintroub	d471469bd2	MDEV-13466 Implement --export option for MariaDB Backup full server recovery is performed . We start "mysqld" with --bootstrap and pass bootstrap script consisting of several FLUSH TABLES FOR export/ UNLOCK TABLES	8 years ago
Jan Lindström	eca238aea7	MDEV-13557: Startup failure, unable to decrypt ibdata1 Fixes also MDEV-13488: InnoDB writes CRYPT_INFO even though encryption is not enabled. Fixes also MDEV-13093: Leak of Datafile::m_crypt_info on shutdown after failed startup. Problem was that we created encryption metadata (crypt_data) for system tablespace even when no encryption was enabled and too early. System tablespace can be encrypted only using key rotation. Test innodb-key-rotation-disable, innodb_encryption, innodb_lotoftables require adjustment because INFORMATION_SCHEMA INNODB_TABLESPACES_ENCRYPTION contain row only if tablespace really has encryption metadata. xb_load_single_table_tablespace(): Do not call fil_space_destroy_crypt_data() any more, because Datafile::m_crypt_data has been removed. fil_crypt_realloc_iops(): Avoid divide by zero. fil_crypt_set_thread_cnt(): Set fil_crypt_threads_event if encryption threads exist. This is required to find tablespaces requiring key rotation if no other changes happen. fil_crypt_find_space_to_rotate(): Decrease the amount of time waiting when nothing happens to better enable key rotation on startup. fil_ibd_open(), fil_ibd_load(): Load possible crypt_data from first page. class Datafile, class SysTablespace : remove m_crypt_info field. Datafile::get_first_page(): Return a pointer to first page buffer. fsp_header_init(): Write encryption metadata to page 0 only if tablespace is encrypted or encryption is disabled by table option. i_s_dict_fill_tablespaces_encryption(): Skip tablespaces that do not contain encryption metadata. This is required to avoid too early wait condition trigger in encrypted -> unencrypted state transfer.	8 years ago
Michael Widenius	458d5ed8aa	Lots of small cleanups - Simplified use_trans_cache() to return at once if is_transactional is set - Indentation and spelling errors fixed - Don't call signal_update() if update_binlog_end_pos() is called as the function already calls signal_update() - Removed not used function wait_for_update_bin_log(), which would cause errors if ever used. - Simplified handler::clone() by always allocating 'ref' in ha_open(). To do this I added an optional MEM_ROOT argument to ha_open() to be used when allocating 'ref' - Changed arguments to get_system_var() from LEX_CSTRING to LEX_CSTRING* - Added THD as argument to create_select_for_variable(). Changed also char* argument to LEX_CSTRING to avoid strlen() call. - Change calls to append() to use LEX_CSTRING	8 years ago
Vladislav Vaintroub	9af7561eb4	MDEV-13608 : set client plugin directory with mysql_options() if plugin_dir is specified. Also, allow to specify protocol (e.g pipe)	8 years ago
Marko Mäkelä	8a3e2970ad	MDEV-13575 On failure, Mariabackup --backup --safe-slave-backup may forget to START SLAVE SQL_THREAD backup_release(): New function, refactored from backup_finish(). Release some resources that may have been acquired by backup_startup() and should be released even after a failed operation. xtrabackup_backup_low(): Refactored from xtrabackup_backup_func(). xtrabackup_backup_func(): Always call backup_release() after calling backup_start().	8 years ago
Marko Mäkelä	e9e051d297	Follow-up fix to MDEV-12988 backup fails if innodb_undo_tablespaces>0 The fix broke mariabackup --prepare --incremental. The restore of an incremental backup starts up (parts of) InnoDB twice. First, all data files are discovered for applying .delta files. Then, after the .delta files have been applied, InnoDB will be restarted more completely, so that the redo log records will be applied via the buffer pool. During the first startup, the buffer pool is not initialized, and thus trx_rseg_get_n_undo_tablespaces() must not be invoked. The apply of the .delta files will currently assume that the --innodb-undo-tablespaces option correctly specifies the number of undo tablespace files, just like --backup does. The second InnoDB startup of --prepare for applying the redo log will properly invoke trx_rseg_get_n_undo_tablespaces(). enum srv_operation_mode: Add SRV_OPERATION_RESTORE_DELTA for distinguishing the apply of .delta files from SRV_OPERATION_RESTORE. srv_undo_tablespaces_init(): In mariabackup --prepare --incremental, in the initial SRV_OPERATION_RESTORE_DELTA phase, do not invoke trx_rseg_get_n_undo_tablespaces() because the buffer pool or the redo logs are not available. Instead, blindly rely on the parameter --innodb-undo-tablespaces.	8 years ago
Marko Mäkelä	5a43c8bae7	MDEV-13416 mariabackup --backup fails to copy log if LSN is above 4294967295 xtrabackup_copy_log(): Use a 64-bit bitmask (lsn_t), not 32-bit (int).	8 years ago
Monty	19f2b3d02f	Fixed compiler warnings	8 years ago
Vladislav Vaintroub	f58142f644	Mariabackup : don't change argv[0] to "innobackupex" in innobackupex mode. addr2line utility optionally used to output stacktrace relies relies on correct my_progname, which is initialized from argv[0] from main function. Thus, changing argv[0] can confuse stacktrace output.	8 years ago
Marko Mäkelä	99d52c45cb	Mariabackup: Copy all of the redo log correctly xtrabackup_copy_log(), xtrabackup_copy_logfile(): Change the Boolean parameter to an enum, with the values COPY_FIRST, COPY_ONLINE, COPY_LAST. xtrabackup_copy_log(): Return the latest scanned LSN, which may be less than the last copied LSN. Remove some dead code that was duplicating some logic that in 10.2 has been moved to log_group_read_log_seg(). log_copying_thread(): Correct the termination condition. stop_backup_threads(): Shut down the threads that were created during backup.	8 years ago
Marko Mäkelä	d7b21a49c2	Mariabackup: Remove unused parameters and fix some memory leaks Use GET_STR instead of GET_STR_ALLOC, so that the memory will cannot be leaked. For some reason, calling my_cleanup_options() on xb_server_options or xb_client_options would not work.	8 years ago
Marko Mäkelä	8c71c6aa8b	MDEV-12548 Initial implementation of Mariabackup for MariaDB 10.2 InnoDB I/O and buffer pool interfaces and the redo log format have been changed between MariaDB 10.1 and 10.2, and the backup code has to be adjusted accordingly. The code has been simplified, and many memory leaks have been fixed. Instead of the file name xtrabackup_logfile, the file name ib_logfile0 is being used for the copy of the redo log. Unnecessary InnoDB startup and shutdown and some unnecessary threads have been removed. Some help was provided by Vladislav Vaintroub. Parameters have been cleaned up and aligned with those of MariaDB 10.2. The --dbug option has been added, so that in debug builds, --dbug=d,ib_log can be specified to enable diagnostic messages for processing redo log entries. By default, innodb_doublewrite=OFF, so that --prepare works faster. If more crash-safety for --prepare is needed, double buffering can be enabled. The parameter innodb_log_checksums=OFF can be used to ignore redo log checksums in --backup. Some messages have been cleaned up. Unless --export is specified, Mariabackup will not deal with undo log. The InnoDB mini-transaction redo log is not only about user-level transactions; it is actually about mini-transactions. To avoid confusion, call it the redo log, not transaction log. We disable any undo log processing in --prepare. Because MariaDB 10.2 supports indexed virtual columns, the undo log processing would need to be able to evaluate virtual column expressions. To reduce the amount of code dependencies, we will not process any undo log in prepare. This means that the --export option must be disabled for now. This also means that the following options are redundant and have been removed: xtrabackup --apply-log-only innobackupex --redo-only In addition to disabling any undo log processing, we will disable any further changes to data pages during --prepare, including the change buffer merge. This means that restoring incremental backups should reliably work even when change buffering is being used on the server. Because of this, preparing a backup will not generate any further redo log, and the redo log file can be safely deleted. (If the --export option is enabled in the future, it must generate redo log when processing undo logs and buffered changes.) In --prepare, we cannot easily know if a partial backup was used, especially when restoring a series of incremental backups. So, we simply warn about any missing files, and ignore the redo log for them. FIXME: Enable the --export option. FIXME: Improve the handling of the MLOG_INDEX_LOAD record, and write a test that initiates a backup while an ALGORITHM=INPLACE operation is creating indexes or rebuilding a table. An error should be detected when preparing the backup. FIXME: In --incremental --prepare, xtrabackup_apply_delta() should ensure that if FSP_SIZE is modified, the file size will be adjusted accordingly.	8 years ago
Marko Mäkelä	4fe89773d8	Mariabackup: Clean up xtrabackup options Fix the binding of databases_file. It was incorrectly mapped to OPT_XTRA_TABLES_FILE. Remove some unused options and variables.	8 years ago
Marko Mäkelä	9a646c91dd	Mariabackup: Remove the --stats option The option was basically duplicating InnoDB functionality. Persistent statistics can be accessed via the tables mysql.innodb_table_stats and mysql.innodb_index_stats.	8 years ago
Marko Mäkelä	cede2b6f0f	Mariabackup: Remove support for .xbcrypt files	8 years ago

... 3 4 5 6 7 ...

461 Commits (420f8e24ab73ef00b323aaa9423f365fb38e9306)