mariadb

Tree: 45ed9dd957

Author	SHA1	Message	Date
Marko Mäkelä	45ed9dd957	MDEV-23855: Remove fil_system.LRU and reduce fil_system.mutex contention Also fixes MDEV-23929: innodb_flush_neighbors is not being ignored for system tablespace on SSD When the maximum configured number of file is exceeded, InnoDB will close data files. We used to maintain a fil_system.LRU list and a counter fil_node_t::n_pending to achieve this, at the huge cost of multiple fil_system.mutex operations per I/O operation. fil_node_open_file_low(): Implement a FIFO replacement policy: The last opened file will be moved to the end of fil_system.space_list, and files will be closed from the start of the list. However, we will not move tablespaces in fil_system.space_list while i_s_tablespaces_encryption_fill_table() is executing (producing output for INFORMATION_SCHEMA.INNODB_TABLESPACES_ENCRYPTION) because it may cause information of some tablespaces to go missing. We also avoid this in mariabackup --backup because datafiles_iter_next() assumes that the ordering is not changed. IORequest: Fold more parameters to IORequest::type. fil_space_t::io(): Replaces fil_io(). fil_space_t::flush(): Replaces fil_flush(). OS_AIO_IBUF: Remove. We will always issue synchronous reads of the change buffer pages in buf_read_page_low(). We will always ignore some errors for background reads. This should reduce fil_system.mutex contention a little. fil_node_t::complete_write(): Replaces fil_node_t::complete_io(). On both read and write completion, fil_space_t::release_for_io() will have to be called. fil_space_t::io(): Do not acquire fil_system.mutex in the normal code path. xb_delta_open_matching_space(): Do not try to open the system tablespace which was already opened. This fixes a file sharing violation in mariabackup --prepare --incremental. Reviewed by: Vladislav Vaintroub	5 years ago
Thirunarayanan Balathandayuthapani	c0f47a4a58	MDEV-12026: Implement innodb_checksum_algorithm=full_crc32 MariaDB data-at-rest encryption (innodb_encrypt_tables) had repurposed the same unused data field that was repurposed in MySQL 5.7 (and MariaDB 10.2) for the Split Sequence Number (SSN) field of SPATIAL INDEX. Because of this, MariaDB was unable to support encryption on SPATIAL INDEX pages. Furthermore, InnoDB page checksums skipped some bytes, and there are multiple variations and checksum algorithms. By default, InnoDB accepts all variations of all algorithms that ever existed. This unnecessarily weakens the page checksums. We hereby introduce two more innodb_checksum_algorithm variants (full_crc32, strict_full_crc32) that are special in a way: When either setting is active, newly created data files will carry a flag (fil_space_t::full_crc32()) that indicates that all pages of the file will use a full CRC-32C checksum over the entire page contents (excluding the bytes where the checksum is stored, at the very end of the page). Such files will always use that checksum, no matter what the parameter innodb_checksum_algorithm is assigned to. For old files, the old checksum algorithms will continue to be used. The value strict_full_crc32 will be equivalent to strict_crc32 and the value full_crc32 will be equivalent to crc32. ROW_FORMAT=COMPRESSED tables will only use the old format. These tables do not support new features, such as larger innodb_page_size or instant ADD/DROP COLUMN. They may be deprecated in the future. We do not want an unnecessary file format change for them. The new full_crc32() format also cleans up the MariaDB tablespace flags. We will reserve flags to store the page_compressed compression algorithm, and to store the compressed payload length, so that checksum can be computed over the compressed (and possibly encrypted) stream and can be validated without decrypting or decompressing the page. In the full_crc32 format, there no longer are separate before-encryption and after-encryption checksums for pages. The single checksum is computed on the page contents that is written to the file. We do not make the new algorithm the default for two reasons. First, MariaDB 10.4.2 was a beta release, and the default values of parameters should not change after beta. Second, we did not yet implement the full_crc32 format for page_compressed pages. This will be fixed in MDEV-18644. This is joint work with Marko Mäkelä.	7 years ago
Marko Mäkelä	e82e216e37	MDEV-17849 Undo tablespace truncation recovery fails to shrink file fil_space_t::add(): Replaces fil_node_create(), fil_node_create_low(). Let the caller pass fil_node_t::handle, to avoid having to close and re-open files. fil_node_t::read_page0(): Refactored from fil_node_open_file(). Read the first page of a data file. fil_node_open_file(): Open the file only once. srv_undo_tablespace_open(): Set the file handle for the opened undo tablespace. This should ensure that ut_ad(file->is_open()) no longer fails in recv_add_trim(). xtrabackup_backup_func(): Remove some dead code. xb_fil_cur_open(): Open files only if needed. Undo tablespaces should already have been opened.	7 years ago
Marko Mäkelä	ff88e4bb8a	Remove many redundant #include from InnoDB	7 years ago
Jan Lindström	eca238aea7	MDEV-13557: Startup failure, unable to decrypt ibdata1 Fixes also MDEV-13488: InnoDB writes CRYPT_INFO even though encryption is not enabled. Fixes also MDEV-13093: Leak of Datafile::m_crypt_info on shutdown after failed startup. Problem was that we created encryption metadata (crypt_data) for system tablespace even when no encryption was enabled and too early. System tablespace can be encrypted only using key rotation. Test innodb-key-rotation-disable, innodb_encryption, innodb_lotoftables require adjustment because INFORMATION_SCHEMA INNODB_TABLESPACES_ENCRYPTION contain row only if tablespace really has encryption metadata. xb_load_single_table_tablespace(): Do not call fil_space_destroy_crypt_data() any more, because Datafile::m_crypt_data has been removed. fil_crypt_realloc_iops(): Avoid divide by zero. fil_crypt_set_thread_cnt(): Set fil_crypt_threads_event if encryption threads exist. This is required to find tablespaces requiring key rotation if no other changes happen. fil_crypt_find_space_to_rotate(): Decrease the amount of time waiting when nothing happens to better enable key rotation on startup. fil_ibd_open(), fil_ibd_load(): Load possible crypt_data from first page. class Datafile, class SysTablespace : remove m_crypt_info field. Datafile::get_first_page(): Return a pointer to first page buffer. fsp_header_init(): Write encryption metadata to page 0 only if tablespace is encrypted or encryption is disabled by table option. i_s_dict_fill_tablespaces_encryption(): Skip tablespaces that do not contain encryption metadata. This is required to avoid too early wait condition trigger in encrypted -> unencrypted state transfer.	8 years ago
Marko Mäkelä	61847b9d80	Tablespace: Add iterator, const_iterator, begin(), end()	8 years ago
Marko Mäkelä	e903d458bb	Clean up InnoDB shutdown Tablespace::shutdown(): Clear m_path. This was moved from Tablespace::~Tablespace(). LatchDebug::shutdown(): Remove a redundant condition.	8 years ago
Marko Mäkelä	494e4b99a4	Remove MYSQL_TABLESPACES. MySQL 5.7 introduced partial support for user-created shared tablespaces (for example, import and export are not supported). MariaDB Server does not support tablespaces at this point of time. Let us remove most InnoDB code and data structures that is related to shared tablespaces.	9 years ago
Sergei Golubchik	ed008a74cf	Make atomic writes general - Atomic writes are enabled by default - Automatically detect if device supports atomic write and use it if atomic writes are enabled - Remove ATOMIC WRITE options from CREATE TABLE - Atomic write is a device option, not a table options as the table may crash if the media changes - Add support for SHANNON SSD cards	9 years ago
Marko Mäkelä	63574f1275	MDEV-11690 Remove UNIV_HOTBACKUP The InnoDB source code contains quite a few references to a closed-source hot backup tool which was originally called InnoDB Hot Backup (ibbackup) and later incorporated in MySQL Enterprise Backup. The open source backup tool XtraBackup uses the full database for recovery. So, the references to UNIV_HOTBACKUP are only cluttering the source code.	9 years ago
Marko Mäkelä	d4342702bf	Remove dead references to NO_FALLOCATE.	9 years ago
Marko Mäkelä	8375a2c1ce	MDEV-11585 Hard-code the shared InnoDB temporary tablespace ID at -1 MySQL 5.7 supports only one shared temporary tablespace. MariaDB 10.2 does not support any other shared InnoDB tablespaces than the two predefined tablespaces: the persistent InnoDB system tablespace (default file name ibdata1) and the temporary tablespace (default file name ibtmp1). InnoDB is unnecessarily allocating a tablespace ID for the predefined temporary tablespace on every startup, and it is in several places testing whether a tablespace ID matches this dynamically generated ID. We should use a compile-time constant to reduce code size and to avoid unnecessary updates to the DICT_HDR page at every startup. Using a hard-coded tablespace ID will should make it easier to remove the TEMPORARY flag from FSP_SPACE_FLAGS in MDEV-11202.	9 years ago
Jan Lindström	fec844aca8	Merge InnoDB 5.7 from mysql-5.7.14. Contains also: MDEV-10549 mysqld: sql/handler.cc:2692: int handler::ha_index_first(uchar*): Assertion `table_share->tmp_table != NO_TMP_TABLE \|\| m_lock_type != 2' failed. (branch bb-10.2-jan) Unlike MySQL, InnoDB still uses THR_LOCK in MariaDB MDEV-10548 Some of the debug sync waits do not work with InnoDB 5.7 (branch bb-10.2-jan) enable tests that were fixed in MDEV-10549 MDEV-10548 Some of the debug sync waits do not work with InnoDB 5.7 (branch bb-10.2-jan) fix main.innodb_mysql_sync - re-enable online alter for partitioned innodb tables	9 years ago
Jan Lindström	2e814d4702	Merge InnoDB 5.7 from mysql-5.7.9. Contains also MDEV-10547: Test multi_update_innodb fails with InnoDB 5.7 The failure happened because 5.7 has changed the signature of the bool handler::primary_key_is_clustered() const virtual function ("const" was added). InnoDB was using the old signature which caused the function not to be used. MDEV-10550: Parallel replication lock waits/deadlock handling does not work with InnoDB 5.7 Fixed mutexing problem on lock_trx_handle_wait. Note that rpl_parallel and rpl_optimistic_parallel tests still fail. MDEV-10156 : Group commit tests fail on 10.2 InnoDB (branch bb-10.2-jan) Reason: incorrect merge MDEV-10550: Parallel replication can't sync with master in InnoDB 5.7 (branch bb-10.2-jan) Reason: incorrect merge	9 years ago

19 Commits (45ed9dd957eebc7fc84feb2509f4aa6baa908a95)