mariadb

Commit Graph

Author	SHA1	Message	Date
Marko Mäkelä	a90100d756	Replace univ_page_size and UNIV_PAGE_SIZE Try to use one variable (srv_page_size) for innodb_page_size. Also, replace UNIV_PAGE_SIZE_SHIFT with srv_page_size_shift.	8 years ago
Marko Mäkelä	ba19764209	Fix most -Wsign-conversion in InnoDB Change innodb_buffer_pool_size, innodb_fill_factor to unsigned.	8 years ago
Marko Mäkelä	8861d442d6	Correct member function comments	8 years ago
Marko Mäkelä	30553aa37b	MDEV-15914: Fast path for rw_trx_hash::find() rw_trx_hash_t::find(): Implement a fast path for looking up the current transaction. This helps transactions that visit rows modified by themselves.	8 years ago
Sergey Vojtovich	e6a9ce2759	MDEV-15773 - Simplified away trx_sys_t::m_views Use trx_sys_t::trx_list instead.	8 years ago
Sergey Vojtovich	0993d6b81b	MDEV-15773 - trx_sys.mysql_trx_list -> trx_sys.trx_list Replaced "list of transactions created for MySQL" with "list of all transactions". This simplifies code and allows further removal of trx_sys.m_views.	8 years ago
Marko Mäkelä	67f6d40bd9	MDEV-15443 Properly read wsrep XID and binlog position from rollback segment headers The problem is a regression caused by MDEV-15158. If some transactions were committed with wsrep_on=0, a rollback segment header having the highest trx_id assigned might store undefined wsrep XID. When reading the wsrep checkpoint from InnodB, the undefined wsrep XID might be returned instead of the highest valid one. Similarly, if the binary log is intermittently disabled or enabled while InnoDB transactions are being committed, the latest updated rollback segment header page might not contain the latest binlog metadata. Therefore, the MDEV-15158 logic to rely on TRX_RSEG_MAX_TRX_ID for determining the most recent WSREP XID or binlog position is invalid. We must choose the maximum entries among the rollback segment header pages. This fix is based on code submitted by Teemu Ollakka from Codership and by Thirunarayanan Balathandayuthapani from MariaDB Corporation. trx_purge_add_undo_to_history(): Only write TRX_RSEG_MAX_TRX_ID when it was used to be written before MDEV-15158. wsrep_seqno: Renamed from trx_sys_cur_xid_seqno. wsrep_uuid: Renamed from trx_sys_cur_xid_uuid, and enable in non-debug builds. read_wsrep_xid_uuid(): Make non-debug, and remove the memcpy(). trx_rseg_update_wsrep_checkpoint(): Correctly compare and copy the entire UUID in the debug check. In case of UUID mismatch, write the WSREP XID to all 128 rollback segment headers in a single mini-transaction. trx_rseg_read_wsrep_checkpoint(rseg_header, xid): Make static. In case the information is absent, do not overwrite xid. trx_rseg_read_wsrep_checkpoint(xid): Determine the maximum WSREP XID. trx_rseg_mem_restore(): Remove the parameter max_rseg_trx_id. Determine the latest binlog file and position by comparing file names and offsets. Declare trx_sys.recovered_binlog_offset as an unsigned type.	8 years ago
Sergey Vojtovich	131d9a5d0c	Allocate lock_sys statically There is only one lock_sys. Allocate it statically in order to avoid dereferencing a pointer whenever accessing it. Also, align some members to their own cache line in order to avoid false sharing. lock_sys_t::create(): The deferred constructor. lock_sys_t::close(): The early destructor.	8 years ago
Marko Mäkelä	fb335b48b5	Allocate purge_sys statically There is only one purge_sys. Allocate it statically in order to avoid dereferencing a pointer whenever accessing it. Also, align some members to their own cache line in order to avoid false sharing. purge_sys_t::create(): The deferred constructor. purge_sys_t::close(): The early destructor. undo::Truncate::create(): The deferred constructor. Because purge_sys.undo_trunc is constructed before the start-up parameters are parsed, the normal constructor would copy a wrong value of srv_purge_rseg_truncate_frequency. TrxUndoRsegsIterator: Do not forward-declare an inline constructor, because the static construction of purge_sys.rseg_iter would not have access to it.	8 years ago
Marko Mäkelä	6ae7fa6878	Simplify TrxUndoRsegs Construct directly from trx_rseg_t&. push_back(), size(): Remove.	8 years ago
Marko Mäkelä	947efe17ed	MDEV-15158 On commit, do not write to the TRX_SYS page This is based on a prototype by Thirunarayanan Balathandayuthapani <thiru@mariadb.com>. Binlog and Galera write-set replication information was written into TRX_SYS page on each commit. Instead of writing to the TRX_SYS during normal operation, InnoDB can make use of rollback segment header pages, which are already being written to during a commit. The following list of fields in rollback segment header page are added: TRX_RSEG_BINLOG_OFFSET TRX_RSEG_BINLOG_NAME (NUL-terminated; empty name = not present) TRX_RSEG_WSREP_XID_FORMAT (0=not present; 1=present) TRX_RSEG_WSREP_XID_GTRID TRX_RSEG_WSREP_XID_BQUAL TRX_RSEG_WSREP_XID_DATA trx_sys_t: Introduce the fields recovered_binlog_filename, recovered_binlog_offset, recovered_wsrep_xid. To facilitate upgrade from older mysql or mariaDB versions, we will read the information in TRX_SYS page. It will be overridden by the information that we find in rollback segment header pages. Mariabackup --prepare will read the metadata from the rollback segment header pages via trx_rseg_array_init(). It will still not read any undo log pages or recover any transactions.	8 years ago
Marko Mäkelä	cd63f43c40	Fix the Windows build my_atomic_load32() expects a pointer to non-const on Windows.	8 years ago
Marko Mäkelä	60a68fdf71	Clarify the access to trx_sys.rseg_history_len trx_sys_t::rseg_history_len: Make private, and clarify the documentation. trx_sys_t::history_size(): Read rseg_history_len. trx_sys_t::history_insert(), trx_sys_t::history_remove(), trx_sys_t::history_add(): Update rseg_history_len.	8 years ago
Sergey Vojtovich	b782971c58	MDEV-15246 - premature history data deletion This is regression after `bc7a1dc1fb` of MDEV-15104 - Optimise MVCC snapshot. Aforementioned revision removes mutex lock around ReadView creation, which allows views to be created concurrently. Effectively it invalidates "oldest view" approach: no single view can be considered oldest anymore. Instead we have to iterate trx_sys.m_views to find min(m_low_limit_no), min(m_low_limit_id) and all transaction ids below min(m_low_limit_id), which would form oldest view. Second regression comes from `c0d5d7c0ef` of MDEV-15104 - Optimise MVCC snapshot. It removes mutex protection around trx->no assignment, which opens up a gap between m_max_trx_id increment and transaction serialisation number becoming visible through rw_trx_hash. While we're in this gap concurrent thread may come and do MVCC snapshot without seeing allocated but not yet assigned serialisation number. Then at some point purge thread may clone this view. As a result it won't see newly allocated serialisation number and may remove "unnecessary" history data of this transaction from rollback segments.	8 years ago
Sergey Vojtovich	bc7a1dc1fb	MDEV-15104 - Optimise MVCC snapshot With trx_sys_t::rw_trx_ids removal, MVCC snapshot overhead became slightly higher. That is instead of copying an array we now have to iterate LF_HASH. All this done under trx_sys.mutex protection. This patch moves MVCC snapshot out of trx_sys.mutex. Clean-ups: Removed MVCC: doesn't make too much sense to keep it in a separate class anymore. Refactored ReadView so that it now calls register()/deregister() routines (it was vice versa before). ReadView doesn't have friends anymore. :( Even less trx_sys.mutex references.	8 years ago
Sergey Vojtovich	c0d5d7c0ef	MDEV-15104 - Remove trx_sys_t::serialisation_list serialisation_list was supposed to instantly give minimum registered transaction serialisation number. However maintaining and accessing this list requires global mutex protection. Since we already take MVCC snapshot by iterating trx_sys_t::rw_trx_hash, it is cheap to integrate minimum registered transaction lookup into this iteration.	8 years ago
Sergey Vojtovich	53cc9aa5be	MDEV-15104 - Remove trx_sys_t::rw_trx_ids Take snapshot of registered read-write transaction identifiers directly from rw_trx_hash. It immediately saves one trx_sys.mutex lock, reduces size of another critical section protected by this mutex, and makes further optimisations like removing trx_sys_t::serialisation_list possible. Downside of this approach is bigger overhead for view opening, because iterating LF_HASH is more expensive compared to taking snapshot of an array. However for low concurrency overhead difference is negligible, while for high concurrency mutex is much bigger evil. Currently we still take trx_sys.mutex to serialise ReadView creation. This is required to keep serialisation_list ordered by trx->no as well as not to let purge thread to create more recent snapshot while another thread gets suspended during creation of older snapshot. This will become completely mutex free along with serialisation_list removal. Compared to previous implementation removing element from rw_trx_hash and serialisation_list is not atomic. We disregard all possible bad consequences (if there're any) since it will be solved along with serialisation_list removal.	8 years ago
Marko Mäkelä	c7d0448797	MDEV-15132 Avoid accessing the TRX_SYS page InnoDB maintains an internal persistent sequence of transaction identifiers. This sequence is used for assigning both transaction start identifiers (DB_TRX_ID=trx->id) and end identifiers (trx->no) as well as end identifiers for the mysql.transaction_registry table that was introduced in MDEV-12894. TRX_SYS_TRX_ID_WRITE_MARGIN: Remove. After this many updates of the sequence we used to update the TRX_SYS page. We can avoid accessing the TRX_SYS page if we modify the InnoDB startup so that resurrecting the sequence from other pages of the transaction system. TRX_SYS_TRX_ID_STORE: Deprecate. The field only exists for the purpose of upgrading from an earlier version of MySQL or MariaDB. Starting with this fix, MariaDB will rely on the fields TRX_UNDO_TRX_ID, TRX_UNDO_TRX_NO in the undo log header page of each non-committed transaction, and on the new field TRX_RSEG_MAX_TRX_ID in rollback segment header pages. Because of this change, setting innodb_force_recovery=5 or 6 may cause the system to recover with trx_sys.get_max_trx_id()==0. We must adjust checks for invalid DB_TRX_ID and PAGE_MAX_TRX_ID accordingly. We will change the startup and shutdown messages to display the trx_sys.get_max_trx_id() in addition to the log sequence number. trx_sys_t::flush_max_trx_id(): Remove. trx_undo_mem_create_at_db_start(), trx_undo_lists_init(): Add an output parameter max_trx_id, to be updated from TRX_UNDO_TRX_ID, TRX_UNDO_TRX_NO. TRX_RSEG_MAX_TRX_ID: New field, for persisting trx_sys.get_max_trx_id() at the time of the latest transaction commit. Startup is not reading the undo log pages of committed transactions. We want to avoid additional page accesses on startup, as well as trouble when all undo logs have been emptied. On startup, we will simply determine the maximum value from all pages that are being read anyway. TRX_RSEG_FORMAT: Redefined from TRX_RSEG_MAX_SIZE. Old versions of InnoDB wrote uninitialized garbage to unused data fields. Because of this, we cannot simply introduce a new field in the rollback segment pages and expect it to be always zero, like it would if the database was created by a recent enough InnoDB version. Luckily, it looks like the field TRX_RSEG_MAX_SIZE was always written as 0xfffffffe. We will indicate a new subformat of the page by writing 0 to this field. This has the nice side effect that after a downgrade to older versions of InnoDB, transactions should fail to allocate any undo log, that is, writes will be blocked. So, there is no problem of getting corrupted transaction identifiers after downgrading. trx_rseg_t::max_size: Remove. trx_rseg_header_create(): Remove the parameter max_size=ULINT_MAX. trx_purge_add_undo_to_history(): Update TRX_RSEG_MAX_SIZE (and TRX_RSEG_FORMAT if needed). This is invoked on transaction commit. trx_rseg_mem_restore(): If TRX_RSEG_FORMAT contains 0, read TRX_RSEG_MAX_SIZE. trx_rseg_array_init(): Invoke trx_sys.init_max_trx_id(max_trx_id + 1) where max_trx_id was the maximum that was encountered in the rollback segment pages and the undo log pages of recovered active, XA PREPARE, or some committed transactions. (See trx_purge_add_undo_to_history() which invokes trx_rsegf_set_nth_undo(..., FIL_NULL, ...); not all committed transactions will be immediately detached from the rollback segment header.)	8 years ago
Marko Mäkelä	6058f92f5c	Simplify undo log access during InnoDB startup trx_rseg_mem_restore(): Update the max_trx_id from the undo log pages. trx_sys_init_at_db_start(): Remove; merge with trx_lists_init_at_db_start(). trx_undo_lists_init(): Move to the only calling module, trx0rseg.cc. trx_undo_mem_create_at_db_start(): Declare globally. Return the number of pages.	8 years ago
Marko Mäkelä	8d1d38f953	Simplify access to the TRX_SYS page trx_sysf_t: Remove. trx_sysf_get(): Return the TRX_SYS page, not a pointer within it. trx_sysf_rseg_get_space(), trx_sysf_rseg_get_page_no(): Remove a parameter, and merge the declaration and definition. Take the TRX_SYS page as a parameter. TRX_SYS_N_RSEGS: Correct the comment. trx_sysf_rseg_find_free(), trx_sys_update_mysql_binlog_offset(), trx_sys_update_wsrep_checkpoint(): Take the TRX_SYS page as a parameter. trx_rseg_header_create(): Add a parameter for the TRX_SYS page. trx_sysf_rseg_set_space(), trx_sysf_rseg_set_page_no(): Remove; merge to the only caller, trx_rseg_header_create().	8 years ago
Sergey Vojtovich	55277e8840	MDEV-15059 - Misc small InnoDB scalability fixes Form better trx_sys API.	8 years ago
Sergey Vojtovich	4dc30f3c17	MDEV-15019 - InnoDB: store ReadView on trx This will allow us to reduce critical section protected by trx_sys.mutex: - no need to maintain global m_free list - eliminate if (trx->read_view == NULL) condition. On x86_64 sizeof(Readview) is 144 mostly due to padding, sizeof(trx_t) with ReadView is 1200. Also don't close ReadView for read-write transactions, just mark it closed similarly to read-only. Clean-up: removed n_prepared_recovered_trx and n_prepared_trx, which accidentally re-appeared after some rebase.	8 years ago
Sergey Vojtovich	64048bafe0	Removed purge_trx_id_age and purge_view_trx_id_age These were unused status variables available in debug builds only. Also removed trx_sys.rw_max_trx_id: not used anymore.	8 years ago
Sergey Vojtovich	db5bb785f9	Allocate trx_sys.mvcc at link time trx_sys.mvcc was allocated dynamically for no good reason.	8 years ago
Marko Mäkelä	f8882cce93	Replace trx_sys_t* trx_sys with trx_sys_t trx_sys There is only one transaction system object in InnoDB. Allocate the storage for it at link time, not at runtime. lock_rec_fetch_page(): Use the correct fetch mode BUF_GET. Pages may never be deallocated from a tablespace while record locks are pointing to them.	8 years ago
Sergey Vojtovich	7078203389	MDEV-14756 - Remove trx_sys_t::rw_trx_list Use atomic operations when accessing trx_sys_t::max_trx_id. We can't yet move trx_sys_t::get_new_trx_id() out of mutex because it must be updated atomically along with trx_sys_t::rw_trx_ids.	8 years ago
Sergey Vojtovich	c6d2842d9a	MDEV-14756 - Remove trx_sys_t::rw_trx_list Remove rw_trx_list.	8 years ago
Sergey Vojtovich	886af392d3	MDEV-14756 - Remove trx_sys_t::rw_trx_list Let trx_rollback_recovered() iterate rw_trx_hash instead of rw_trx_list.	8 years ago
Sergey Vojtovich	02270b44d0	MDEV-14756 - Remove trx_sys_t::rw_trx_list Let lock_validate_table_locks(), lock_rec_other_trx_holds_expl(), lock_table_locks_lookup(), trx_recover_for_mysql(), trx_get_trx_by_xid(), trx_roll_must_shutdown(), fetch_data_into_cache() iterate rw_trx_hash instead of rw_trx_list.	8 years ago
Sergey Vojtovich	d8c0caad32	MDEV-14756 - Remove trx_sys_t::rw_trx_list Removed trx_sys_validate_trx_list(): with rw_trx_hash elements are not required to be ordered by transaction id. Transaction state is now guarded by asserts in rw_trx_hash_t.	8 years ago
Sergey Vojtovich	900b07908b	MDEV-14756 - Remove trx_sys_t::rw_trx_list Removed trx_sys_t::n_prepared_recovered_trx: never used. Removed trx_sys_t::n_prepared_trx: used only at shutdown, we can perfectly get this value from rw_trx_hash.	8 years ago
Sergey Vojtovich	a0b385ea2b	MDEV-14756 - Remove trx_sys_t::rw_trx_list Determine minimum transaction id by iterating rw_trx_hash, not rw_trx_list. It is more expensive than previous implementation since it does linear search, especially if there're many concurrent transactions running. But in such case mutex is much bigger evil. And since it doesn't require trx_sys->mutex protection it scales better. For low concurrency performance difference is neglible.	8 years ago
Sergey Vojtovich	868c77df3e	MDEV-14756 - Remove trx_sys_t::rw_trx_list Replaced UT_LIST_GET_LEN(trx_sys->rw_trx_list) with trx_sys->rw_trx_hash.size(). Moved freeing of trx objects at shutdown to rw_trx_hash destructor. Small clean-up in trx_rollback_recovered().	8 years ago
Sergey Vojtovich	0ca2ea1a65	MDEV-14638 - Replace trx_sys_t::rw_trx_set with LF_HASH trx reference counter was updated under mutex and read without any protection. This is both slow and unsafe. Use atomic operations for reference counter accesses.	8 years ago
Sergey Vojtovich	380069c235	MDEV-14638 - Replace trx_sys_t::rw_trx_set with LF_HASH trx_sys_t::rw_trx_set is implemented as std::set, which does a few quite expensive operations under trx_sys_t::mutex protection: e.g. malloc/free when adding/removing elements. Traversing b-tree is not that cheap either. This has negative scalability impact, which is especially visible when running oltp_update_index.lua benchmark on a ramdisk. To reduce trx_sys_t::mutex contention std::set is replaced with LF_HASH. None of LF_HASH operations require trx_sys_t::mutex (nor any other global mutex) protection. Another interesting issue observed with std::set is reproducible ~2% performance decline after benchmark is ran for ~60 seconds. With LF_HASH results are stable. All in all this patch optimises away one of three trx_sys->mutex locks per oltp_update_index.lua query. The other two critical sections became smaller. Relevant clean-ups: Replaced rw_trx_set iteration at startup with local set. The latter is needed because values inserted to rw_trx_list must be ordered by trx->id. Removed redundant conditions from trx_reference(): it is (and even was) never called with transactions that have trx->state == TRX_STATE_COMMITTED_IN_MEMORY. do_ref_count doesn't (and probably even didn't) make any sense: now it is called only when reference counter increment is actually requested. Moved condition out of mutex in trx_erase_lists(). trx_rw_is_active(), trx_rw_is_active_low() and trx_get_rw_trx_by_id() were greatly simplified and replaced by appropriate trx_rw_hash_t methods. Compared to rw_trx_set, rw_trx_hash holds transactions only in PREPARED or ACTIVE states. Transactions in COMMITTED state were required to be found at InnoDB startup only. They are now looked up in the local set. Removed unused trx_assert_recovered(). Removed unused innobase_get_trx() declaration. Removed rather semantically incorrect trx_sys_rw_trx_add(). Moved information printout from trx_sys_init_at_db_start() to trx_lists_init_at_db_start().	8 years ago
Marko Mäkelä	ff81faf670	MDEV-13654 Various crashes due to DB_TRX_ID mismatch in table-rebuilding ALTER TABLE…LOCK=NONE After MDEV-12288 and MDEV-13536, the DB_TRX_ID of old clustered index records for which no history is available should be reset to 0. This caused crashes in online table-rebuilding ALTER, because the row_log_table_apply() is built on the assumption that the PRIMARY KEY together with DB_TRX_ID,DB_ROLL_PTR identifies the record. Both when copying the old table and when writing log about changes to the old table, we must map "old" DB_TRX_ID to 0. "old" here is simply "older than the trx_id of the ALTER TABLE transaction", because the MDL_EXCLUSIVE (and exclusive InnoDB table lock) in ha_innobase::prepare_inplace_alter_table() forces any transactions accessing the table to commit or rollback. So, we know that we can safely reset any DB_TRX_ID in the table that is older than the transaction ID of the ALTER TABLE, because the undo log history would be lost in a table-rebuilding ALTER. Note: After a table-rebuilding online ALTER TABLE, the rebuilt table may end up containing some nonzero DB_TRX_ID columns. The apply logic identifies the rows by the combination of PRIMARY KEY and DB_TRX_ID. These nonzero DB_TRX_ID would necessarily refer to concurrent DML operations that were started during ha_innobase::inplace_alter_table(). row_log_allocate(): Add a parameter for the ALTER TABLE transaction. row_log_t::min_trx: The ALTER TABLE transaction ID. trx_id_check(): A debug function to check that DB_TRX_ID makes sense (is either 0 or bigger than the ALTER TABLE transaction ID). reset_trx_id[]: The reset DB_TRX_ID,DB_ROLL_PTR columns. row_log_table_delete(), row_log_table_get_pk(): Reset the DB_TRX_ID,DB_ROLL_PTR when they precede the ALTER TABLE transaction. row_log_table_apply_delete(), row_log_table_apply_update(): Assert trx_id_check(). row_merge_insert_index_tuples(): Remove the unused parameter trx_id. row_merge_read_clustered_index(): In a table-rebuilding ALTER, reset the DB_TRX_ID,DB_ROLL_PTR when they precede the ALTER TABLE transaction. Assert trx_id_check() on clustered index records that are being buffered.	8 years ago
Jan Lindström	eba0120d8f	Fix test failures on embedded server. Problem was incorrect definition of wsrep_recovery, trx_sys_update_wsrep_checkpoint and trx_sys_read_wsrep_checkpoint functions causing innodb_plugin not to load as there was undefined symbols.	8 years ago
Jan Lindström	b29f26d774	Fix test failures on embedded server. Problem was incorrect definition of wsrep_recovery, trx_sys_update_wsrep_checkpoint and trx_sys_read_wsrep_checkpoint functions causing innodb_plugin not to load as there was undefined symbols.	8 years ago
Jan Lindström	391b1af0fb	MDEV-13471: Test failure on innodb.log_file_size,4k Problem was that 4k page size is not really supported in Galera. For reference see: codership/galera#398 Page size 4k is problematic because WSREP XID info location that was set to constant UNIV_PAGE_SIZE - 3500 and that is conflicting with rseg undo slots location if there is lot of undo tablespaces. Undo tablespace identifiers and page numbers require at least 126*8=1024 bytes starting from offset 56. Therefore, WSREP XID startig from offset 596 would overwrite several space_id,page_no pairs starting from 72th undo log tablespace space_id,page_no pair at offset 594. This will cause InnoDB startup failure seen as [ERROR] InnoDB: Unable to open undo tablespace './undo30579'. Originally, the undo tablespace ID would always be between 0 and 127. Starting with MySQL 5.6.36 which introduced Bug #25551311 BACKPORT BUG #23517560 REMOVE SPACE_ID RESTRICTION FOR UNDO TABLESPACES (merged to MariaDB 10.0.31) it is possible for an undo tablespace ID to be 0x7773. But in this case, the page number should be 3, not 0x72650003. This is just the first collision. The WSREP XID data would overwrite subsequent slots. trx0sys.h trx0sys.cc Code formatting and add comments.	8 years ago
Marko Mäkelä	591edccc93	Simplify access to the binlog offset in InnoDB trx_sys_print_mysql_binlog_offset(): Use 64-bit arithmetics and ib::info(). TRX_SYS_MYSQL_LOG_OFFSET: Replaces TRX_SYS_MYSQL_LOG_OFFSET_HIGH, TRX_SYS_MYSQL_LOG_OFFSET_LOW. trx_sys_update_mysql_binlog_offset(): Remove the constant parameter field=TRX_SYS_MYSQL_LOG_INFO. Use 64-bit arithmetics.	8 years ago
Marko Mäkelä	aea0e125d2	trx_sys_read_wsrep_checkpoint(): Return whether a checkpoint is present	8 years ago
Marko Mäkelä	0c92794db3	Remove deprecated InnoDB file format parameters The following options will be removed: innodb_file_format innodb_file_format_check innodb_file_format_max innodb_large_prefix They have been deprecated in MySQL 5.7.7 (and MariaDB 10.2.2) in WL#7703. The file_format column in two INFORMATION_SCHEMA tables will be removed: innodb_sys_tablespaces innodb_sys_tables Code to update the file format tag at the end of page 0:5 (TRX_SYS_PAGE in the InnoDB system tablespace) will be removed. When initializing a new database, the bytes will remain 0. All references to the Barracuda file format will be removed. Some references to the Antelope file format (meaning ROW_FORMAT=REDUNDANT or ROW_FORMAT=COMPACT) will remain. This basically ports WL#7704 from MySQL 8.0.0 to MariaDB 10.3.1: commit `4a69dc2a95` Author: Marko Mäkelä <marko.makela@oracle.com> Date: Wed Mar 11 22:19:49 2015 +0200	9 years ago
Marko Mäkelä	206ecb79a5	Follow-up to MDEV-12289: Support innodb_undo_tablespaces=127 MySQL 5.7 reduced the maximum number of innodb_undo_tablespaces from 126 to 95 when it reserved 32 persistent rollback segments for the temporary undo logs. Since MDEV-12289 restored all 128 persistent rollback segments for persistent undo logs, the reasonable maximum value of innodb_undo_tablespaces is 127 (not 126 or 95). This is because out of the 128 rollback segments, the first one will always be created in the system tablespace and the remaining ones can be created in dedicated undo tablespaces.	9 years ago
Marko Mäkelä	23ea4360fd	Remove TRX_SYS_OLD_N_RSEGS	9 years ago
Marko Mäkelä	124bae082b	MDEV-12289 Keep 128 persistent rollback segments for compatibility and performance InnoDB divides the allocation of undo logs into rollback segments. The DB_ROLL_PTR system column of clustered indexes can address up to 128 rollback segments (TRX_SYS_N_RSEGS). Originally, InnoDB only created one rollback segment. In MySQL 5.5 or in the InnoDB Plugin for MySQL 5.1, all 128 rollback segments were created. MySQL 5.7 hard-codes the rollback segment IDs 1..32 for temporary undo logs. On upgrade, unless a slow shutdown (innodb_fast_shutdown=0) was performed on the old server instance, these rollback segments could be in use by transactions that are in XA PREPARE state or transactions that were left behind by a server kill followed by a normal shutdown immediately after restart. Persistent tables cannot refer to temporary undo logs or vice versa. Therefore, we should keep two distinct sets of rollback segments: one for persistent tables and another for temporary tables. In this way, all 128 rollback segments will be available for both types of tables, which could improve performance. Also, MariaDB 10.2 will remain more compatible than MySQL 5.7 with data files from earlier versions of MySQL or MariaDB. trx_sys_t::temp_rsegs[TRX_SYS_N_RSEGS]: A new array of temporary rollback segments. The trx_sys_t::rseg_array[TRX_SYS_N_RSEGS] will be solely for persistent undo logs. srv_tmp_undo_logs. Remove. Use the constant TRX_SYS_N_RSEGS. srv_available_undo_logs: Change the type to ulong. trx_rseg_get_on_id(): Remove. Instead, let the callers refer to trx_sys directly. trx_rseg_create(), trx_sysf_rseg_find_free(): Remove unneeded parameters. These functions only deal with persistent undo logs. trx_temp_rseg_create(): New function, to create all temporary rollback segments at server startup. trx_rseg_t::is_persistent(): Determine if the rollback segment is for persistent tables. trx_sys_is_noredo_rseg_slot(): Remove. The callers must know based on context (such as table handle) whether the DB_ROLL_PTR is referring to a persistent undo log. trx_sys_create_rsegs(): Remove all parameters, which were always passed as global variables. Instead, modify the global variables directly. enum trx_rseg_type_t: Remove. trx_t::get_temp_rseg(): A method to ensure that a temporary rollback segment has been assigned for the transaction. trx_t::assign_temp_rseg(): Replaces trx_assign_rseg(). trx_purge_free_segment(), trx_purge_truncate_rseg_history(): Remove the redundant variable noredo=false. Temporary undo logs are discarded immediately at transaction commit or rollback, not lazily by purge. trx_purge_mark_undo_for_truncate(): Remove references to the temporary rollback segments. trx_purge_mark_undo_for_truncate(): Remove a check for temporary rollback segments. Only the dedicated persistent undo log tablespaces can be truncated. trx_undo_get_undo_rec_low(), trx_undo_get_undo_rec(): Add the parameter is_temp. trx_rseg_mem_restore(): Split from trx_rseg_mem_create(). Initialize the undo log and the rollback segment from the file data structures. trx_sysf_get_n_rseg_slots(): Renamed from trx_sysf_used_slots_for_redo_rseg(). Count the persistent rollback segment headers that have been initialized. trx_sys_close(): Also free trx_sys->temp_rsegs[]. get_next_redo_rseg(): Merged to trx_assign_rseg_low(). trx_assign_rseg_low(): Remove the parameters and access the global variables directly. Revert to simple round-robin, now that the whole trx_sys->rseg_array[] is for persistent undo log again. get_next_noredo_rseg(): Moved to trx_t::assign_temp_rseg(). srv_undo_tablespaces_init(): Remove some parameters and use the global variables directly. Clarify some error messages. Adjust the test innodb.log_file. Apparently, before these changes, InnoDB somehow ignored missing dedicated undo tablespace files that are pointed by the TRX_SYS header page, possibly losing part of essential transaction system state.	9 years ago
Marko Mäkelä	97acc4a1c3	MDEV-12270 Port MySQL 8.0 Bug#21141390 REMOVE UNUSED FUNCTIONS AND CONVERT GLOBAL SYMBOLS TO STATIC InnoDB defines some functions that are not called at all. Other functions are called, but only from the same compilation unit. Remove some function declarations and definitions, and add 'static' keywords. Some symbols must be kept for separately compiled tools, such as innochecksum.	11 years ago
Marko Mäkelä	4e1116b2c6	MDEV-12271 Port MySQL 8.0 Bug#23150562 REMOVE UNIV_MUST_NOT_INLINE AND UNIV_NONINL Also, remove empty .ic files that were not removed by my MySQL commit. Problem: InnoDB used to support a compilation mode that allowed to choose whether the function definitions in .ic files are to be inlined or not. This stopped making sense when InnoDB moved to C++ in MySQL 5.6 (and ha_innodb.cc started to #include .ic files), and more so in MySQL 5.7 when inline methods and functions were introduced in .h files. Solution: Remove all references to UNIV_NONINL and UNIV_MUST_NOT_INLINE from all files, assuming that the symbols are never defined. Remove the files fut0fut.cc and ut0byte.cc which only mattered when UNIV_NONINL was defined.	10 years ago
Marko Mäkelä	7a30d86e9d	Simplify InnoDB startup. InnoDB needs to collect transactions from the persistent data files in trx_rseg_array_init() before trx_lists_init_at_db_start() is executed. But there is no need to create purge_sys->purge_queue separately from purge_sys. trx_sys_init_at_db_start(): Change the return type to void. Remove the direct call to trx_rseg_array_init(). It will be called by trx_lists_init_at_db_start(), which we are calling. Initialize the purge system read view. trx_lists_init_at_db_start(): Call trx_purge_sys_create(), which will invoke trx_rseg_array_init() to read the undo log segments. trx_purge_sys_create(): Remove the parameters. Do not initialize the purge system read view, because trx_sys->rw_trx_list has not been recovered yet. The purge_sys->view will be initialized at the end of trx_sys_init_at_db_start(). trx_rseg_array_init(): Remove the parameter. Use purge_sys->purge_queue directly. innobase_start_or_create_for_mysql(): Remove the local variable purge_queue. Do not call trx_purge_sys_create(), because it will be called by trx_sys_init_at_db_start().	9 years ago
Marko Mäkelä	15bdfeeba8	Remove trx_sys_t::pending_purge_rseg_array. In MySQL 5.7, there is some redundant code for supposedly handling an upgrade from an earlier version of InnoDB. An upgrade of InnoDB between major versions should include a slow shutdown (innodb_fast_shutdown=0) of the previous version. A comment in trx_lists_init_at_db_start() confused clean shutdown and slow shutdown. A clean shutdown does not necessarily guarantee that there are no active transactions. A slow shutdown guarantees that. Because there was no code to handle rollback of recovered transactions that happened to use the rollback segment slots that MySQL 5.7.2 repurposed for temporary undo logs, the upgrade is not working in all cases, and we may as well remove the code to handle purging. trx_sys_t::pending_purge_rseg_array: Remove. trx_undo_get_undo_rec_low(): Define as static. Remove the parameter is_redo_rseg. trx_undo_get_undo_rec(), trx_rseg_get_on_id(): Remove the parameter is_redo_rseg. trx_rseg_mem_free(): Remove the second parameter. trx_sys_get_nth_rseg(): Replace with trx_rseg_get_on_id(). trx_rseg_schedule_pending_purge(): Remove.	9 years ago
Marko Mäkelä	63574f1275	MDEV-11690 Remove UNIV_HOTBACKUP The InnoDB source code contains quite a few references to a closed-source hot backup tool which was originally called InnoDB Hot Backup (ibbackup) and later incorporated in MySQL Enterprise Backup. The open source backup tool XtraBackup uses the full database for recovery. So, the references to UNIV_HOTBACKUP are only cluttering the source code.	9 years ago

1 2

83 Commits (a90100d756eb046814f5bba54d522f1574c131ba)