mariadb

Tree: 5018b998a7

Author	SHA1	Message	Date
Vicențiu Ciorbaru	c0ac0b8860	Update FSF address	7 years ago
Vladislav Vaintroub	4b0f010b88	MDEV-18544 "missing required privilege PROCESS on ." using mariabackup for SST If required privilege is missing, dump the output from "SHOW GRANTS" into mariabackup log. This will help troubleshooting, and make the bug reproducible.	7 years ago
Vladislav Vaintroub	e03ad4f71a	Fix a typo	7 years ago
Eugene Kosov	6c5c1f0b2f	MDEV-19231 make DB_SUCCESS equal to 0 It's a micro optimization. On most platforms CPUs has instructions to compare with 0 fast. DB_SUCCESS is the most popular outcome of functions and this patch optimized code like (err == DB_SUCCESS) BtrBulk::finish(): bogus assertion fixed fil_node_t::read_page0(): corrected usage of os_file_read() que_eval_sql(): bugus assertion removed. Apparently it checked that the field was assigned after having been zero-initialized at object creation. It turns out that the return type of os_file_read_func() was changed in mysql/mysql-server@98909cefbc37e54efc6452c7e95bccbf64ac9213 (MySQL 5.7) from ibool to dberr_t. The reviewer (if there was any) failed to point out that because of future merges, it could be a bad idea to change the return type of a function without changing the function name. This change was applied to MariaDB 10.2.2 in commit `2e814d4702` but the MariaDB-specific code was not fully adjusted accordingly, e.g. in fil_node_open_file(). Essentially, code like !os_file_read(...) became dead code in MariaDB and later in Mariabackup 10.2, and we could be dealing with an uninitialized buffer after a failed page read.	7 years ago
Vladislav Vaintroub	6b5d3c51b3	Do fast exit with error code and FATAL ERROR message, if innodb cannot start during prepare. Otherwise, it is possible for "prepare" to exit with error code 0, even if it did not work at all.	7 years ago
Monty	a024649081	Fixed compiler warnings form gcc 7.3.1	7 years ago
Vlad Lesin	05b84b2568	MDEV-14192: Add debug assertions	7 years ago
Marko Mäkelä	f120a15b93	MDEV-19212 4GB Limit on large_pages - integer overflow os_mem_alloc_large(): Invoke the macro ut_2pow_round() with the correct argument type. innobase_large_page_size, innobase_use_large_pages, os_use_large_pages, os_large_page_size: Remove. Simply refer to opt_large_page_size, my_use_large_pages.	7 years ago
Vlad Lesin	caa8c20abe	MDEV-14192 Mariabackup assertion failure: byte_offset % OS_FILE_LOG_BLOCK_SIZE == 0 xtrabackup_backup_func(): If the log checkpoint header changed since we last read it, search for the most recent checkpoint again. Otherwise, we could corrupt the backup of the redo log, because the least significant bits of checkpoint_lsn_start would not match log_sys->log.lsn.	7 years ago
Marko Mäkelä	1d30b7b1d2	MDEV-12699 preparation: Clean up recv_sys The recv_sys data structures are accessed not only from the thread that executes InnoDB plugin initialization, but also from the InnoDB I/O threads, which can invoke recv_recover_page(). Assert that sufficient concurrency control is in place. Some code was accessing recv_sys data structures without holding recv_sys->mutex. recv_recover_page(bpage): Refactor the call from buf_page_io_complete() into a separate function that performs necessary steps. The main thread was unnecessarily releasing and reacquiring recv_sys->mutex. recv_recover_page(block,mtr,recv_addr): Pass more parameters from the caller. Avoid redundant lookups and computations. Eliminate some redundant variables. recv_get_fil_addr_struct(): Assert that recv_sys->mutex is being held. That was not always the case! recv_scan_log_recs(): Acquire recv_sys->mutex for the whole duration of the function. (While we are scanning and buffering redo log records, no pages can be read in.) recv_read_in_area(): Properly protect access with recv_sys->mutex. recv_apply_hashed_log_recs(): Check recv_addr->state only once, and continuously hold recv_sys->mutex. The mutex will be released and reacquired inside recv_recover_page() and recv_read_in_area(), allowing concurrent processing by buf_page_io_complete() in I/O threads.	7 years ago
FaramosCZ	9c9bf9642e	Fix resource leak	7 years ago
Marko Mäkelä	5c1720f8af	Remove unused variables from non-debug build	7 years ago
Marko Mäkelä	3d1b6f68f1	Mariabackup: Ensure NUL termination in strncpy()	7 years ago
Sergei Golubchik	f1134d5676	post-merge: gcc 8 warnings note: Inherit String from Sql_alloc, to get operators new and new[] in sync in rocksdb gcc was complaining that non-lvalue was cast to const.	7 years ago
Vladislav Vaintroub	cd805a5f1c	Cleanup - remove unused variables and functions after MDEV-18917	7 years ago
Vladislav Vaintroub	396cf60ac0	MDEV-18917 Don't create xtrabackup_binlog_pos_innodb with Mariabackup	7 years ago
Thirunarayanan Balathandayuthapani	d038806dfe	MDEV-18855 Mariabackup should fetch innodb_compression_level from running server - Fetch innodb_compression_level from the running server.Add the value of innodb_compression_level in backup-my.cnf file during backup phase. So that prepare can use the innodb_compression_level variable from backup-my.cnf	7 years ago
Marko Mäkelä	b761211685	MDEV-18659: Fix string truncation/overflow in InnoDB and XtraDB Fix the warnings issued by GCC 8 -Wstringop-truncation and -Wstringop-overflow in InnoDB and XtraDB. This work is motivated by Jan Lindström. The patch mainly differs from his original one as follows: (1) We remove explicit initialization of stack-allocated string buffers. The minimum amount of initialization that is needed is a terminating NUL character. (2) GCC issues a warning for invoking strncpy(dest, src, sizeof dest) because if strlen(src) >= sizeof dest, there would be no terminating NUL byte in dest. We avoid this problem by invoking strncpy() with a limit that is 1 less than the buffer size, and by always writing NUL to the last byte of the buffer. (3) We replace strncpy() with memcpy() or strcpy() in those cases when the result is functionally equivalent. Note: fts_fetch_index_words() never deals with len==UNIV_SQL_NULL. This was enforced by an assertion that limits the maximum length to FTS_MAX_WORD_LEN. Also, the encoding that InnoDB uses for the compressed fulltext index is not byte-order agnostic, that is, InnoDB data files that use FULLTEXT INDEX are not portable between big-endian and little-endian systems.	7 years ago
Vladislav Vaintroub	945c748adc	MDEV-18669 mariabackup writes timestamp in version line Use fprintf(stderr) instead of msg() to print the version line	7 years ago
Vladislav Vaintroub	39a2984dc0	Revert "MDEV-18575 Cleanup : remove innodb-encrypt-log parameter from mariabackup" This reverts commit `3262967008`. It was checked in by mistake	7 years ago
Thirunarayanan Balathandayuthapani	c0f47a4a58	MDEV-12026: Implement innodb_checksum_algorithm=full_crc32 MariaDB data-at-rest encryption (innodb_encrypt_tables) had repurposed the same unused data field that was repurposed in MySQL 5.7 (and MariaDB 10.2) for the Split Sequence Number (SSN) field of SPATIAL INDEX. Because of this, MariaDB was unable to support encryption on SPATIAL INDEX pages. Furthermore, InnoDB page checksums skipped some bytes, and there are multiple variations and checksum algorithms. By default, InnoDB accepts all variations of all algorithms that ever existed. This unnecessarily weakens the page checksums. We hereby introduce two more innodb_checksum_algorithm variants (full_crc32, strict_full_crc32) that are special in a way: When either setting is active, newly created data files will carry a flag (fil_space_t::full_crc32()) that indicates that all pages of the file will use a full CRC-32C checksum over the entire page contents (excluding the bytes where the checksum is stored, at the very end of the page). Such files will always use that checksum, no matter what the parameter innodb_checksum_algorithm is assigned to. For old files, the old checksum algorithms will continue to be used. The value strict_full_crc32 will be equivalent to strict_crc32 and the value full_crc32 will be equivalent to crc32. ROW_FORMAT=COMPRESSED tables will only use the old format. These tables do not support new features, such as larger innodb_page_size or instant ADD/DROP COLUMN. They may be deprecated in the future. We do not want an unnecessary file format change for them. The new full_crc32() format also cleans up the MariaDB tablespace flags. We will reserve flags to store the page_compressed compression algorithm, and to store the compressed payload length, so that checksum can be computed over the compressed (and possibly encrypted) stream and can be validated without decrypting or decompressing the page. In the full_crc32 format, there no longer are separate before-encryption and after-encryption checksums for pages. The single checksum is computed on the page contents that is written to the file. We do not make the new algorithm the default for two reasons. First, MariaDB 10.4.2 was a beta release, and the default values of parameters should not change after beta. Second, we did not yet implement the full_crc32 format for page_compressed pages. This will be fixed in MDEV-18644. This is joint work with Marko Mäkelä.	7 years ago
Marko Mäkelä	ca76fc4a3a	MDEV-18611: mariabackup silently ended during xtrabackup_copy_logfile() log_group_read_log_seg(): Always return false when returning before reading end_lsn. xtrabackup_copy_logfile(): On error, indicate whether a corrupt log record was encountered. Only xtrabackup_copy_logfile() in Mariabackup cared about the return value of the function. InnoDB crash recovery was not affected by this bug.	7 years ago
Vladislav Vaintroub	282ba973e7	MDEV-18549 Failing assertion: opt_no_lock during mariabackup --backup The assertion happens under BACKUP STAGE BLOCK_COMMIT, when a DDL on a temporary table (#sql-xxx) is found. Apparently, assumption that all DDLs are blocked under FTWRL does not hold for BACKUP STAGE, and temporary tables can still have ALTERs The fix is to relax the assertion, and only check for opt_no_lock if backup is really inconsistent, i.e either optimized DDL or CREATE/RENAME are done on the tables that were not skipped during backup.	7 years ago
Vladislav Vaintroub	40b4f9c907	MDEV-18575 remove innodb-encrypt-log parameter from mariabackup The variable is obsolete. In mariabackup --backup, encryption plugin loading code sets the value this parameter to the same as in server. In mariabackup --prepare, no new redo log is generated, and xtrabackup_logfile is removed after it anyway.	7 years ago
Vladislav Vaintroub	3262967008	MDEV-18575 Cleanup : remove innodb-encrypt-log parameter from mariabackup	7 years ago
Marko Mäkelä	0a1c3477bf	MDEV-18493 Remove page_size_t MySQL 5.7 introduced the class page_size_t and increased the size of buffer pool page descriptors by introducing this object to them. Maybe the intention of this exercise was to prepare for a future where the buffer pool could accommodate multiple page sizes. But that future never arrived, not even in MySQL 8.0. It is much easier to manage a pool of a single page size, and typically all storage devices of an InnoDB instance benefit from using the same page size. Let us remove page_size_t from MariaDB Server. This will make it easier to remove support for ROW_FORMAT=COMPRESSED (or make it a compile-time option) in the future, just by removing various occurrences of zip_size.	7 years ago
Thirunarayanan Balathandayuthapani	f669cecbe3	MDEV-18415 mariabackup.mdev-14447 test case fails with Table 'test.t' doesn't exist in engine - Added retry logic if validation of first page fails with checksum mismatch.	7 years ago
Vladislav Vaintroub	6e2af7d084	mariabackup : Remove unused parameter innodb_buffer_pool_size	7 years ago
Geoff Montee	f17c284c57	MDEV-18347: mariabackup doesn't read all server option groups from configuration files	7 years ago
Vladislav Vaintroub	5c159c9037	MDEV-18356 Renamed backup-encrypted option introduced in `7158edcba3` Rename it because it caused parser warning whenever --backup was used.	7 years ago
Vladislav Vaintroub	2153aaf66e	mariabackup : use die() macro for fatal exit with error message.	7 years ago
Vladislav Vaintroub	a8a27e65a8	MDEV-18212 mariabackup: Make output format uniform whenever possible	7 years ago
Vladislav Vaintroub	32bb579192	replace hardcoded string "#sql" with tmp_file_prefix	7 years ago
Vladislav Vaintroub	24bc8edcc5	MDEV-18184 Do not copy temporary files (prefixed with #sql-) to backup	7 years ago
FaramosCZ	7372fe4da1	xb_process_datadir(): Fix resource leaks Closes #983, #984	7 years ago
Vladislav Vaintroub	7331c661db	MDEV-18201 : mariabackup- fix processing of rename/create sequence in prepare Fix one more bug in "DDL redo" phase in prepare If table was renamed, and then new table was created with the old name, prepare can be confused, and .ibd can end up with wrong name. Fix the order of how DDL fixup is applied , once again - ".new" files should be processed after renames.	7 years ago
Vladislav Vaintroub	4a872ae1e7	MDEV-18185 - mariabackup - fix specific case of table rename handing in prepare. If, during backup 1) Innodb table is dropped (after being copied to backup) and then 2) Before backup finished, another Innodb table is renamed, and new name is the name of the dropped table in 1) then, --prepare fails with assertion, as DDL fixup code in prepare did not handle this specific case. The fix is to process drops before renames, in prepare DDL-"redo" phase.	7 years ago
Marko Mäkelä	7158edcba3	MDEV-18129 Backup fails for encrypted tables: mariabackup: Database page corruption detected at page 1 If an encrypted table is created during backup, then mariabackup --backup could wrongly fail. This caused a failure of the test mariabackup.huge_lsn once on buildbot. This is due to the way how InnoDB creates .ibd files. It would first write a dummy page 0 with no encryption information. Due to this, xb_fil_cur_open() could wrongly interpret that the table is not encrypted. Subsequently, page_is_corrupted() would compare the computed page checksum to the wrong checksum. (There are both "before" and "after" checksums for encrypted pages.) To work around this problem, we introduce a Boolean option --backup-encrypted that is enabled by default. With this option, Mariabackup will assume that a nonzero key_version implies that the page is encrypted. We need this option in order to be able to copy encrypted tables from MariaDB 10.1 or 10.2, because unencrypted pages that were originally created before MySQL 5.1.48 could contain nonzero garbage in the fields that were repurposed for encryption. Later, MDEV-18128 would clean up the way how .ibd files are created, to remove the need for this option. page_is_corrupted(): Add missing const qualifiers, and do not check space->crypt_data unless --skip-backup-encrypted has been specified. xb_fil_cur_read(): After a failed page read, output a page dump.	7 years ago
Vladislav Vaintroub	ed66acb291	Silence LeakSanitizer by default in mariabackup, so that phanthom "leaks" would not hide more interesting information, like invalid memory accesses. some "leaks" are expected - partly this is due to weird options parsing, that runs twice, and does not free memory after the first run. - also we do not mind to exit() whenever it makes sense, without full cleanup.	7 years ago
Marko Mäkelä	cb2f36d3ea	MDEV-14975: Cleanup check_all_privileges() Remove an unused variable, and propagate the error to the caller instead of calling exit().	7 years ago
Vladislav Vaintroub	773479f5b3	Add test for partial backup for partitioned table.	7 years ago
Vladislav Vaintroub	9f4a4cb401	Cleanup recent mariabackup validation patches. - Refactor code to isolate page validation in page_is_corrupted() function. - Introduce --extended-validation parameter(default OFF) for mariabackup --backup to enable decryption of encrypted uncompressed pages during backup. - mariabackup would still always check checksum on encrypted data, it is needed to detect partially written pages.	7 years ago
Vladislav Vaintroub	74659e55b7	MDEV-16268 : sporadic checksum mismatch when opening system tablespace in backup Attempt to fix by retrying srv_sys_space.open_or_create() if it reports corruption.	7 years ago
Vladislav Vaintroub	5716c71c54	MDEV-14975 mariabackup starts with unprivileged user. ported privilege checking from xtrabackup. Now, mariabackup would terminate early if either RELOAD or PROCESS privilege is not held, not at the very end of backup The behavior can be disabled with nre setting --check-privileges=0. Also , --no-lock does not need all of these privileges, since it skips FTWRL and SHOW ENGINE STATUS INNODB.	7 years ago
Marko Mäkelä	fb252f70c1	MDEV-12112 corruption in encrypted table may be overlooked After validating the post-encryption checksum on an encrypted page, Mariabackup should decrypt the page and validate the pre-encryption checksum as well. This should reduce the probability of accepting invalid pages as valid ones. This is a backport and refactoring of a patch that was originally written by Thirunarayanan Balathandayuthapani for the 10.2 branch.	7 years ago
Vladislav Vaintroub	3efed7533c	Fix lock_ddl_per_table handling in mariabackup. mariabackup.lock_ddl_per_table was broken, after MDEV-17772 got fixed. Fix removes MDL waiter killer, it is is not needed anymore.	7 years ago
Vladislav Vaintroub	c0ca164b1c	MDEV-17308 mariabackup to use the BACKUP statement instead of FTWRL	7 years ago
Marko Mäkelä	e82e216e37	MDEV-17849 Undo tablespace truncation recovery fails to shrink file fil_space_t::add(): Replaces fil_node_create(), fil_node_create_low(). Let the caller pass fil_node_t::handle, to avoid having to close and re-open files. fil_node_t::read_page0(): Refactored from fil_node_open_file(). Read the first page of a data file. fil_node_open_file(): Open the file only once. srv_undo_tablespace_open(): Set the file handle for the opened undo tablespace. This should ensure that ut_ad(file->is_open()) no longer fails in recv_add_trim(). xtrabackup_backup_func(): Remove some dead code. xb_fil_cur_open(): Open files only if needed. Undo tablespaces should already have been opened.	7 years ago
Daniel Black	3859273d04	MDEV-14267: correct FSF address	8 years ago
Marko Mäkelä	8b6b2c3ea1	Fix mariabackup leaks (except my_load_defaults)	7 years ago

1 2 3 4 5 ...

395 Commits (5018b998a76b99a30efe96ca007a95a5a1c83bb5)