mariadb

Tree: d75d8631ed

Author	SHA1	Message	Date
Monty	d75d8631ed	[MDEV-10570] Add Flashback support ==== Description ==== Flashback can rollback the instances/databases/tables to an old snapshot. It's implement on Server-Level by full image format binary logs (--binlog-row-image=FULL), so it supports all engines. Currently, it’s a feature inside mysqlbinlog tool (with --flashback arguments). Because the flashback binlog events will store in the memory, you should check if there is enough memory in your machine. ==== New Arguments to mysqlbinlog ==== --flashback (-B) It will let mysqlbinlog to work on FLASHBACK mode. ==== New Arguments to mysqld ==== --flashback Setup the server to use flashback. This enables binary log in row mode and will enable extra logging for DDL's needed by flashback feature ==== Example ==== I have a table "t" in database "test", we can compare the output with "--flashback" and without. #client/mysqlbinlog /data/mysqldata_10.0/binlog/mysql-bin.000001 -vv -d test -T t --start-datetime="2013-03-27 14:54:00" > /tmp/1.sql #client/mysqlbinlog /data/mysqldata_10.0/binlog/mysql-bin.000001 -vv -d test -T t --start-datetime="2013-03-27 14:54:00" -B > /tmp/2.sql Then, importing the output flashback file (/tmp/2.log), it can flashback your database/table to the special time (--start-datetime). And if you know the exact postion, "--start-postion" is also works, mysqlbinlog will output the flashback logs that can flashback to "--start-postion" position. ==== Implement ==== 1. As we know, if binlog_format is ROW (binlog-row-image=FULL in 10.1 and later), all columns value are store in the row event, so we can get the data before mis-operation. 2. Just do following things: 2.1 Change Event Type, INSERT->DELETE, DELETE->INSERT. For example: INSERT INTO t VALUES (...) ---> DELETE FROM t WHERE ... DELETE FROM t ... ---> INSERT INTO t VALUES (...) 2.2 For Update_Event, swapping the SET part and WHERE part. For example: UPDATE t SET cols1 = vals1 WHERE cols2 = vals2 ---> UPDATE t SET cols2 = vals2 WHERE cols1 = vals1 2.3 For Multi-Rows Event, reverse the rows sequence, from the last row to the first row. For example: DELETE FROM t WHERE id=1; DELETE FROM t WHERE id=2; ...; DELETE FROM t WHERE id=n; ---> DELETE FROM t WHERE id=n; ...; DELETE FROM t WHERE id=2; DELETE FROM t WHERE id=1; 2.4 Output those events from the last one to the first one which mis-operation happened. For example:	9 years ago
Elena Stepanova	b9631b4633	Follow-up for the 10.1 -> 10.2 merge Fix 32-bit rdiffs to get rid of failed hunks	9 years ago
Marko Mäkelä	b05bf8ff0f	Merge 10.1 to 10.2. Most notably, this includes MDEV-11623, which includes a fix and an upgrade procedure for the InnoDB file format incompatibility that is present in MariaDB Server 10.1.0 through 10.1.20. In other words, this merge should address MDEV-11202 InnoDB 10.1 -> 10.2 migration does not work	9 years ago
Igor Babaev	833aa97cec	Fixed bug mdev-11818. When a query containing a WITH clause is printed by EXPLAIN EXTENDED command there should not be any data expansion in the query specifications of the WITH elements of this WITH clause.	9 years ago
Marko Mäkelä	a1315a650a	MDEV-11202 InnoDB 10.1 -> 10.2 migration does not work This is the final preparation for the merge of MDEV-11623 from MariaDB Server 10.1 (correcting the assignment of FSP_SPACE_FLAGS in MariaDB Server 10.1). We must avoid reading FSP_SPACE_FLAGS directly from page 0, because the flags that affect the uncompressed page size cannot be trusted if we are upgrading from MariaDB 10.1. We will pass fil_space_t* instead of a numeric tablespace ID in many low-level functions, so that fil_space_t::flags will be available. This will also reduce the amount o tablespace ID lookups and the pressure on fil_system->mutex. fil_space_is_being_truncated(), fil_space_is_redo_skipped(), PageCallback::set_page_size(), fsp_header_get_page_size(): Remove. fil_node_open_file(), fil_space_get_crypt_data(): Use space->flags. fsp_free_extent(), fseg_get_first_extent(), fsp_get_space_header(), xdes_get_descriptor_with_space_hdr(), xdes_get_descriptor(), xdes_lst_get_descriptor(), fsp_space_modify_check(), fsp_init_file_page(), fsp_alloc_free_extent(), fsp_page_create(), fsp_alloc_free_page(), fsp_free_page(), fsp_alloc_seg_inode_page(), fsp_alloc_seg_inode(), fsp_free_seg_inode(), fseg_fill_free_list(), fseg_mark_page_used(), fseg_free_page_low(), fseg_free_extent(): Take fil_space_t* as a parameter, instead of taking a numeric ID.	9 years ago
Marko Mäkelä	8780b89529	MDEV-11831 Make InnoDB mini-transaction memo checks stricter InnoDB keeps track of buffer-fixed buf_block_t or acquired rw_lock_t within a mini-transaction. There are some memo_contains assertions in the code that document when certain blocks or rw_locks must be held. But, these assertions only check the mini-transaction memo, not the fact whether the rw_lock_t are actually being held by the caller. btr_pcur_store_position(): Remove #ifdef, and assert that the block is always buffer-fixed. rtr_pcur_getnext_from_path(), rtr_pcur_open_low(), ibuf_rec_get_page_no_func(), ibuf_rec_get_space_func(), ibuf_rec_get_info_func(), ibuf_rec_get_op_type_func(), ibuf_build_entry_from_ibuf_rec_func(), ibuf_rec_get_volume_func(), ibuf_get_merge_page_nos_func(), ibuf_get_volume_buffered_count_func() ibuf_get_entry_counter_low_func(), page_set_ssn_id(), row_vers_old_has_index_entry(), row_vers_build_for_consistent_read(), row_vers_build_for_semi_consistent_read(), trx_undo_prev_version_build(): Make use of mtr_memo_contains_page_flagged(). mtr_t::memo_contains(): Take a const memo. Assert rw_lock_own(). FindPage, FlaggedCheck: Assert rw_lock_own_flagged().	9 years ago
Jan Lindström	95ebca7197	Fix test failure on sysvars_innodb. MySQL Compression code was removed but missed this test case.	9 years ago
Jan Lindström	716b87845d	Fix test failure on innodb-page_compression_snappy test. Modified test data to be more easily compressed and fixed mtr_suppression if compression fails.	9 years ago
Sergei Petrunia	25e5ce1982	NOT FIXED: MDEV-10773: ANALYZE FORMAT=JSON query_with_CTE crashes Remove the testcase. It turns out, the crash still happens, just not on all platforms.	9 years ago
Oleksandr Byelkin	9ea0b44c56	Such big blocks in query processing should be represented in the debugging trace.	9 years ago
Marko Mäkelä	08413254b7	Remove references to innodb_file_format. innodb_file_format=Barracuda is the default in MariaDB 10.2. Do not set it, because the option will be removed in MariaDB 10.3. Also, do not set innodb_file_per_table=1 because it is the default. Note that MDEV-11828 should fix the test innodb.innodb-64k already in 10.1.	9 years ago
Marko Mäkelä	085b292a47	MDEV-11824 Allow ROW_FORMAT=DYNAMIC in the InnoDB system tablespace When MySQL 5.7.9 (and MariaDB Server 10.2) introduced innodb_default_row_format and made ROW_FORMAT=DYNAMIC the default, it became possible to create any ROW_FORMAT tables in the InnoDB system tablespace, except ROW_FORMAT=COMPRESSED. In MySQL 5.7, it is possible to create ROW_FORMAT=DYNAMIC tables when TABLESPACE=innodb_system is explicitly specified. Because MariaDB Server 10.2 does not support the MySQL 5.7 TABLESPACE=innodb_system attribute for tables, we should allow ROW_FORMAT=DYNAMIC when innodb_file_per_table=0. Also, remove the test innodb_zip.innodb-create-options, which was an outdated copy of innodb_zip.create_options.	9 years ago
Marko Mäkelä	7cf97ed4ee	MDEV-11816 Disallow CREATE TEMPORARY TABLE…ROW_FORMAT=COMPRESSED MySQL 5.7 allows temporary tables to be created in ROW_FORMAT=COMPRESSED. The usefulness of this is questionable. WL#7899 in MySQL 8.0.0 prevents the creation of such compressed tables, so that all InnoDB temporary tables will be located inside the predefined InnoDB temporary tablespace. Pick up and adjust some tests from MySQL 5.7 and 8.0. dict_tf_to_fsp_flags(): Remove the parameter is_temp. fsp_flags_init(): Remove the parameter is_temporary. row_mysql_drop_temp_tables(): Remove. There cannot be any temporary tables in InnoDB. (This never removed #sql* tables in the datadir which were created by DDL.) dict_table_t::dir_path_of_temp_table: Remove. create_table_info_t::m_temp_path: Remove. create_table_info_t::create_options_are_invalid(): Do not allow ROW_FORMAT=COMPRESSED or KEY_BLOCK_SIZE for temporary tables. create_table_info_t::innobase_table_flags(): Do not unnecessarily prevent CREATE TEMPORARY TABLE with SPATIAL INDEX. (MySQL 5.7 does allow this.) fil_space_belongs_in_lru(): The only FIL_TYPE_TEMPORARY tablespace is never subjected to closing least-recently-used files.	9 years ago
Marko Mäkelä	494e4b99a4	Remove MYSQL_TABLESPACES. MySQL 5.7 introduced partial support for user-created shared tablespaces (for example, import and export are not supported). MariaDB Server does not support tablespaces at this point of time. Let us remove most InnoDB code and data structures that is related to shared tablespaces.	9 years ago
Marko Mäkelä	1eabad5dbe	Remove MYSQL_COMPRESSION. The MariaDB 10.1 page_compression is incompatible with the Oracle implementation that was introduced in MySQL 5.7 later. Remove the Oracle implementation. Also remove the remaining traces of MYSQL_ENCRYPTION. This will also remove traces of PUNCH_HOLE until it is implemented better. The only effective call to os_file_punch_hole() was in fil_node_create_low() to test if the operation is supported for the file. In other words, it looks like page_compression is not working in MariaDB 10.2, because no code equivalent to the 10.1 os_file_trim() is enabled.	9 years ago
Marko Mäkelä	70c11485d2	Remove MYSQL_ENCRYPTION. MariaDB will likely never support MySQL-style encryption for InnoDB, because we cannot link with the Oracle encryption plugin. This is preparation for merging MDEV-11623.	9 years ago
Sergei Petrunia	45f11a729c	MDEV-10773: ANALYZE FORMAT=JSON query_with_CTE crashes Added testcase	9 years ago
Marko Mäkelä	6a65de6cda	Correct a test broken in the merge `7c81f15ec3`	9 years ago
Marko Mäkelä	c849b7df61	MDEV-11785 Remove INFORMATION_SCHEMA.INNODB_TEMP_TABLE_INFO The INFORMATION_SCHEMA view INNODB_TEMP_TABLE_INFO was added to MySQL 5.7 as part of the work to implement temporary tables without any redo logging. The only use case of this view was SELECT COUNT(*) in some tests, to see how many temporary tables exist in InnoDB. The columns do not report much useful information. For example, the table name would not be the user-specified table name, but a generated #sql name. Also, the session that created the table is not identified.	9 years ago
Marko Mäkelä	bb109aeea3	MDEV-6076: Fix a broken assertion.	9 years ago
Marko Mäkelä	0349712937	MDEV-11623 follow-up: Adjust one more test. The function fsp_flags_try_adjust(), which is called on startup, is incrementing the Innodb_pages0_read counts for every affected file. Adjust the result of encryption.innodb_lotoftables accordingly.	9 years ago
Marko Mäkelä	7e3f3deb41	MDEV-11623 follow-up: Adjust tests. innodb.doublewrite: Similar to what was done to innodb.101_compatibility, add an explicit $_ parameter to the Perl unpack function. Also, fix some diagnostic messages in the Perl code. innodb.innodb-wl5522-debug: Adjust for the changed error codes and messages on fault injection.	9 years ago
Elena Stepanova	3a91dec124	101_compatibility test fails on CentOS 5 Apparently, in older versions of Perl unpack does not have a logic for using $_ as a default value for the second argument. Fixed by specifying it explicitly	9 years ago
Elena Stepanova	34c89d01d2	Updated list of unstable tests for 10.1.21	9 years ago
Marko Mäkelä	ab1e6fefd8	MDEV-11623 MariaDB 10.1 fails to start datadir created with MariaDB 10.0/MySQL 5.6 using innodb-page-size!=16K The storage format of FSP_SPACE_FLAGS was accidentally broken already in MariaDB 10.1.0. This fix is bringing the format in line with other MySQL and MariaDB release series. Please refer to the comments that were added to fsp0fsp.h for details. This is an INCOMPATIBLE CHANGE that affects users of page_compression and non-default innodb_page_size. Upgrading to this release will correct the flags in the data files. If you want to downgrade to earlier MariaDB 10.1.x, please refer to the test innodb.101_compatibility how to reset the FSP_SPACE_FLAGS in the files. NOTE: MariaDB 10.1.0 to 10.1.20 can misinterpret uncompressed data files with innodb_page_size=4k or 64k as compressed innodb_page_size=16k files, and then probably fail when trying to access the pages. See the comments in the function fsp_flags_convert_from_101() for detailed analysis. Move PAGE_COMPRESSION to FSP_SPACE_FLAGS bit position 16. In this way, compressed innodb_page_size=16k tablespaces will not be mistaken for uncompressed ones by MariaDB 10.1.0 to 10.1.20. Derive PAGE_COMPRESSION_LEVEL, ATOMIC_WRITES and DATA_DIR from the dict_table_t::flags when the table is available, in fil_space_for_table_exists_in_mem() or fil_open_single_table_tablespace(). During crash recovery, fil_load_single_table_tablespace() will use innodb_compression_level for the PAGE_COMPRESSION_LEVEL. FSP_FLAGS_MEM_MASK: A bitmap of the memory-only fil_space_t::flags that are not to be written to FSP_SPACE_FLAGS. Currently, these will include PAGE_COMPRESSION_LEVEL, ATOMIC_WRITES and DATA_DIR. Introduce the macro FSP_FLAGS_PAGE_SSIZE(). We only support one innodb_page_size for the whole instance. When creating a dummy tablespace for the redo log, use fil_space_t::flags=0. The flags are never written to the redo log files. Remove many FSP_FLAGS_SET_ macros. dict_tf_verify_flags(): Remove. This is basically only duplicating the logic of dict_tf_to_fsp_flags(), used in a debug assertion. fil_space_t::mark: Remove. This flag was not used for anything. fil_space_for_table_exists_in_mem(): Remove the unnecessary parameter mark_space, and add a parameter for table flags. Check that fil_space_t::flags match the table flags, and adjust the (memory-only) flags based on the table flags. fil_node_open_file(): Remove some redundant or unreachable conditions, do not use stderr for output, and avoid unnecessary server aborts. fil_user_tablespace_restore_page(): Convert the flags, so that the correct page_size will be used when restoring a page from the doublewrite buffer. fil_space_get_page_compressed(), fsp_flags_is_page_compressed(): Remove. It suffices to have fil_space_is_page_compressed(). FSP_FLAGS_WIDTH_DATA_DIR, FSP_FLAGS_WIDTH_PAGE_COMPRESSION_LEVEL, FSP_FLAGS_WIDTH_ATOMIC_WRITES: Remove, because these flags do not exist in the FSP_SPACE_FLAGS but only in memory. fsp_flags_try_adjust(): New function, to adjust the FSP_SPACE_FLAGS in page 0. Called by fil_open_single_table_tablespace(), fil_space_for_table_exists_in_mem(), innobase_start_or_create_for_mysql() except if --innodb-read-only is active. fsp_flags_is_valid(ulint): Reimplement from the scratch, with accurate comments. Do not display any details of detected inconsistencies, because the output could be confusing when dealing with MariaDB 10.1.x data files. fsp_flags_convert_from_101(ulint): Convert flags from buggy MariaDB 10.1.x format, or return ULINT_UNDEFINED if the flags cannot be in MariaDB 10.1.x format. fsp_flags_match(): Check the flags when probing files. Implemented based on fsp_flags_is_valid() and fsp_flags_convert_from_101(). dict_check_tablespaces_and_store_max_id(): Do not access the page after committing the mini-transaction. IMPORT TABLESPACE fixes: AbstractCallback::init(): Convert the flags. FetchIndexRootPages::operator(): Check that the tablespace flags match the table flags. Do not attempt to convert tablespace flags to table flags, because the conversion would necessarily be lossy. PageConverter::update_header(): Write back the correct flags. This takes care of the flags in IMPORT TABLESPACE.	9 years ago
Marko Mäkelä	a9d00db155	MDEV-11799 InnoDB can abort if the doublewrite buffer contains a bad and a good copy Clean up the InnoDB doublewrite buffer code. buf_dblwr_init_or_load_pages(): Do not add empty pages to the buffer. buf_dblwr_process(): Do consider changes to pages that are all zero. Do not abort when finding a corrupted copy of a page in the doublewrite buffer, because there could be multiple copies in the doublewrite buffer, and only one of them needs to be good.	9 years ago
Marko Mäkelä	9b99d9bebd	MDEV-8139: Disable a randomly failing test until the code is fixed.	9 years ago
Sergei Golubchik	5dfab33c4e	MDEV-11551 Server crashes in Field::is_real_null sometimes table->s->stored_fields is less than table->s->null_fields	9 years ago
Sergei Golubchik	7e2f9d092d	max_session_mem_used server variable	9 years ago
Sergei Golubchik	ab3388c3c4	bugfix: mutex order violation in embedded safe_mutex: Found wrong usage of mutex 'LOCK_thread_count' and 'LOCK_status' Mutex currently locked (in reverse order): LOCK_status sql/sql_class.h line 3873 LOCK_thread_count libmysqld/lib_sql.cc line 432	9 years ago
Sergei Golubchik	1282eb694c	cleanup: make malloc_size_cb_func always defined	9 years ago
Alexander Barkov	ebb8c9fb26	MDEV-11030 Assertion `precision > 0' failed in decimal_bin_size Fixing Item::decimal_precision() to return at least one digit. This fixes the problem reported in MDEV. Also, fixing Item_func_signed::fix_length_and_dec() to reserve space for at least one digit (plus one character for an optional sign). This is needed to have CONVERT(expr,SIGNED) and CONVERT(expr,UNSIGNED) create correct string fields when they appear in string context, e.g.: CREATE TABLE t1 AS SELECT CONCAT(CONVERT('',SIGNED));	9 years ago
Marko Mäkelä	1ba7234b21	Follow-up to MDEV-11713: Make more use of DBUG_LOG	9 years ago
Marko Mäkelä	7c81f15ec3	Merge 10.1 into bb-10.2-mdev-11782	9 years ago
Marko Mäkelä	2dc5d8bb7e	Improve an MDEV-9011 test of innodb_encrypt_log. Test crash recovery from an encrypted redo log with innodb_encrypt_log=0. Previously, we did a clean shutdown, so only the log checkpoint information would have been read from the redo log. With this change, we will be reading and applying encrypted redo log records. include/start_mysqld.inc: Observe $restart_parameters. encryption.innodb-log-encrypt: Remove some unnecessary statements, and instead of restarting the server and concurrently accessing the files while the server is running, kill the server, check the files, and finally start up the server. innodb.log_data_file_size: Use start_mysqld.inc with $restart_parameters.	9 years ago
Marko Mäkelä	fd0479ce59	Merge 10.1 into 10.2	9 years ago
Marko Mäkelä	4507f1e5d4	Remove an excessive copyright message.	9 years ago
Marko Mäkelä	5b5bce81e6	Merge 10.0 into 10.1	9 years ago
Marko Mäkelä	833fda8f1a	InnoDB: Enable UNIV_DEBUG_VALGRIND for cmake -DWITH_VALGRIND The symbol HAVE_VALGRIND_MEMCHECK_H was never defined. Instead, the symbol HAVE_VALGRIND should have been used.	9 years ago
Monty	5f0c31f928	MDEV-11597 Assertion when doing select from virtual column with impossible value - Changed error handlers interface so that they can change error level in the handler - Give warnings and errors when calculating virtual columns - On insert/update error is fatal in strict mode. - SELECT and DELETE will only give a warning if a virtual field generates an error - Added VCOL_UPDATE_FOR_DELETE and VCOL_UPDATE_INDEX_FOR_REPLACE to be able to easily detect in update_virtual_fields() if we should use an error handler to mask errors or not.	9 years ago
Monty	177c191ff4	MDEV-11606 Server crashes in mi_make_key / sort_key_read Problem was that one internal record buffer in MYISAM was not big enough to handle virtual fields. Fixed by extending buffer. Fixed test case to 79 characters	9 years ago
Monty	c9b3e4535b	MDEV-11737 Failing assertion: block->magic_n == MEM_BLOCK_MAGIC_N Issue was that the m_prebuilt array was reused without resetting a counter, which caused a memory overrun. Adjusted test case to 79 characters	9 years ago
Monty	6c5c98316f	Updated mysql-test/README with information about KB	9 years ago
Monty	135e144479	MDEV-11598 Assertion `!table \|\| (!table->read_set \|\| bitmap_is_set(table->read_set, field_index))' failed Found and fixed 2 problems: - Filesort addon fields didn't mark virtual columns properly - multi-range-read calculated vcol bitmap but was not using it. This caused wrong vcol field to be calculated on read, which caused the assert.	9 years ago
Monty	de22cd3fe5	Fixes for failures in test scripts and removal of some compiler warnings - rpl_gtid_stop_start: Random failure - sysvars_innodb,32bit: Fixes for 32 bit	9 years ago
Monty	bf95970ec0	Cleanups * Ensure that BUILD scripts cleans up things properly before new BUILD * MySQL -> MariaDB * Ignore wrong valgrind memleak	9 years ago
Sergei Golubchik	53ae72e2ee	mtr uses dgcov for --gcov option	9 years ago
Sergei Golubchik	00f462cf1b	dgcov: import, rewrite to work with cmake and git	9 years ago
Monty	7567cf5aef	Fixes for using ssl with BUILD scripts. Needed as GNUTLS can't be used by MariaDB on many systems, like OpenSuse 13.2	9 years ago
Monty	ea1b25046c	New simpler bugfix for UPDATE and virtual BLOBs When updating a table with virtual BLOB columns, the following might happen: - an old record is read from the table, it has no virtual blob values - update_virtual_fields() is run, vcol blob gets its value into the record. But only a pointer to the value is in the table->record[0], the value is in Field_blob::value String (but it doesn't have to be! it can be in the record, if the column is just a copy of another columns: ... b VARCHAR, c BLOB AS (b) ...) - store_record(table,record[1]), old record now is in record[1] - fill_record() prepares new values in record[0], vcol blob is updated, new value replaces the old one in the Field_blob::value - now both record[1] and record[0] have a pointer that points to the new vcol blob value. Or record[1] has a pointer to nowhere if Field_blob::value had to realloc. To fix this I have introduced a new String object 'read_value' in Field_blob. When updating virtual columns when a row has been read, the allocated value is stored in 'read_value' instead of 'value'. The allocated blobs for the new row is stored in 'value' as before. I also made, as a safety precaution, the insert delayed handling of blobs more general by using value to store strings instead of the record. This ensures that virtual functions on delayed insert should work in as in the case of normal insert. Triggers are now properly updating the read, write and vcol maps for used fields. This means that we don't need VCOL_UPDATE_FOR_READ_WRITE anymore and there is no need for any other special handling of triggers in update_virtual_fields(). To be able to test how many times virtual fields are invoked, I also relaxed rules that one can use local (@) variables in DEFAULT and non persistent virtual field expressions.	9 years ago

1 2 3 4 5 ...

176052 Commits (d75d8631ed2d6af730931ea7079ec7e512e61796) All Branches Search

176052 Commits (d75d8631ed2d6af730931ea7079ec7e512e61796)

All Branches