mariadb

Tree: dc7bc64483

Author	SHA1	Message	Date
Marko Mäkelä	41bb3537ba	Bug#12704861 Corruption after a crash during BLOB update The fix of Bug#12612184 broke crash recovery. When a record that contains off-page columns (BLOBs) is updated, we must first write redo log about the BLOB page writes, and only after that write the redo log about the B-tree changes. The buggy fix would log the B-tree changes first, meaning that after recovery, we could end up having a record that contains a null BLOB pointer. Because we will be redo logging the writes off the off-page columns before the B-tree changes, we must make sure that the pages chosen for the off-page columns are free both before and after the B-tree changes. In this way, the worst thing that can happen in crash recovery is that the BLOBs are written to free pages, but the B-tree changes are not applied. The BLOB pages would correctly remain free in this case. To achieve this, we must allocate the BLOB pages in the mini-transaction of the B-tree operation. A further quirk is that BLOB pages are allocated from the same file segment as leaf pages. Because of this, we must temporarily "hide" any leaf pages that were freed during the B-tree operation by "fake allocating" them prior to writing the BLOBs, and freeing them again before the mtr_commit() of the B-tree operation, in btr_mark_freed_leaves(). btr_cur_mtr_commit_and_start(): Remove this faulty function that was introduced in the Bug#12612184 fix. The problem that this function was trying to address was that when we did mtr_commit() the BLOB writes before the mtr_commit() of the update, the new BLOB pages could have overwritten clustered index B-tree leaf pages that were freed during the update. If recovery applied the redo log of the BLOB writes but did not see the log of the record update, the index tree would be corrupted. The correct solution is to make the freed clustered index pages unavailable to the BLOB allocation. This function is also a likely culprit of InnoDB hangs that were observed when testing the Bug#12612184 fix. btr_mark_freed_leaves(): Mark all freed clustered index leaf pages of a mini-transaction allocated (nonfree=TRUE) before storing the BLOBs, or freed (nonfree=FALSE) before committing the mini-transaction. btr_freed_leaves_validate(): A debug function for checking that all clustered index leaf pages that have been marked free in the mini-transaction are consistent (have not been zeroed out). btr_page_alloc_low(): Refactored from btr_page_alloc(). Return the number of the allocated page, or FIL_NULL if out of space. Add the parameter "mtr_t* init_mtr" for specifying the mini-transaction where the page should be initialized, or if this is a "fake allocation" (init_mtr=NULL) by btr_mark_freed_leaves(nonfree=TRUE). btr_page_alloc(): Add the parameter init_mtr, allowing the page to be initialized and X-latched in a different mini-transaction than the one that is used for the allocation. Invoke btr_page_alloc_low(). If a clustered index leaf page was previously freed in mtr, remove it from the memo of previously freed pages. btr_page_free(): Assert that the page is a B-tree page and it has been X-latched by the mini-transaction. If the freed page was a leaf page of a clustered index, link it by a MTR_MEMO_FREE_CLUST_LEAF marker to the mini-transaction. btr_store_big_rec_extern_fields_func(): Add the parameter alloc_mtr, which is NULL (old behaviour in inserts) and the same as local_mtr in updates. If alloc_mtr!=NULL, the BLOB pages will be allocated from it instead of the mini-transaction that is used for writing the BLOBs. fsp_alloc_from_free_frag(): Refactored from fsp_alloc_free_page(). Allocate the specified page from a partially free extent. fseg_alloc_free_page_low(), fseg_alloc_free_page_general(): Add the parameter "mtr_t* init_mtr" for specifying the mini-transaction where the page should be initialized, or NULL if this is a "fake allocation" that prevents the reuse of a previously freed B-tree page for BLOB storage. If init_mtr==NULL, try harder to reallocate the specified page and assert that it succeeded. fsp_alloc_free_page(): Add the parameter "mtr_t* init_mtr" for specifying the mini-transaction where the page should be initialized. Do not allow init_mtr == NULL, because this function is never to be used for "fake allocations". mtr_t: Add the operation MTR_MEMO_FREE_CLUST_LEAF and the flag mtr->freed_clust_leaf for quickly determining if any MTR_MEMO_FREE_CLUST_LEAF operations have been posted. row_ins_index_entry_low(): When columns are being made off-page in insert-by-update, invoke btr_mark_freed_leaves(nonfree=TRUE) and pass the mini-transaction as the alloc_mtr to btr_store_big_rec_extern_fields(). Finally, invoke btr_mark_freed_leaves(nonfree=FALSE) to avoid leaking pages. row_build(): Correct a comment, and add a debug assertion that a record that contains NULL BLOB pointers must be a fresh insert. row_upd_clust_rec(): When columns are being moved off-page, invoke btr_mark_freed_leaves(nonfree=TRUE) and pass the mini-transaction as the alloc_mtr to btr_store_big_rec_extern_fields(). Finally, invoke btr_mark_freed_leaves(nonfree=FALSE) to avoid leaking pages. buf_reset_check_index_page_at_flush(): Remove. The function fsp_init_file_page_low() already sets bpage->check_index_page_at_flush=FALSE. There is a known issue in tablespace extension. If the request to allocate a BLOB page leads to the tablespace being extended, crash recovery could see BLOB writes to pages that are off the tablespace file bounds. This should trigger an assertion failure in fil_io() at crash recovery. The safe thing would be to write redo log about the tablespace extension to the mini-transaction of the BLOB write, not to the mini-transaction of the record update. However, there is no redo log record for file extension in the current redo log format. rb:693 approved by Sunny Bains	14 years ago
Sergey Vojtovich	a47cb8f055	Applying InnoDB snapshot Detailed revision comments: r6891 \| vdimov \| 2010-03-26 16:19:01 +0200 (Fri, 26 Mar 2010) \| 5 lines Non-functional change: update copyright year to 2010 of the files that have been modified after 2010-01-01 according to svn. for f in $(svn log -v -r{2010-01-01}:HEAD \|grep "^ M " \|cut -b 16- \|sort -u) ; do sed -i "" -E 's/(Copyright $c$ [0-9]{4},) [0-9]{4}, (.*Innobase Oy.+All Rights Reserved)/\1 2010, \2/' $f ; done	16 years ago
Sergey Vojtovich	15568b946e	Applying InnoDB snapshot Detailed revision comments: r6801 \| marko \| 2010-03-11 13:34:28 +0200 (Thu, 11 Mar 2010) \| 2 lines branches/zip: mtr_memo_contains(): Relax the assertion of r6800, allowing mtr->state == MTR_COMMITTING.	16 years ago
Sergey Vojtovich	edba051bee	Applying InnoDB snapshot Detailed revision comments: r6800 \| marko \| 2010-03-11 12:02:57 +0200 (Thu, 11 Mar 2010) \| 1 line branches/zip: Add ut_ad(mtr->state == MTR_ACTIVE) to various places.	16 years ago
Satya B	1b9fe0bbac	Adding innodb_plugin-1.0.4 as storage/innodb_plugin.	17 years ago
tsmith/tim@siva.hindu.god	032f356f7c	Apply InnoDB snapshot ss923 Fixes: - Bug #18077: InnoDB uses full explicit table locks in stored FUNCTION	19 years ago
aivanov@mysql.com	d33b523271	Applied innodb-5.1-ss269 snapshot. Fixed BUGS: #3300: "UPDATE statement with no index column in where condition locks all rows" Implement semi-consistent read to reduce lock conflicts at the cost of breaking serializability. ha_innobase::unlock_row(): reset the "did semi consistent read" flag ha_innobase::was_semi_consistent_read(), ha_innobase::try_semi_consistent_read(): new methods row_prebuilt_t, row_create_prebuilt(): add field row_read_type for keeping track of semi-consistent reads row_vers_build_for_semi_consistent_read(), row_sel_build_committed_vers_for_mysql(): new functions row_search_for_mysql(): implement semi-consistent reads #9802: "Foreign key checks disallow alter table". Added test cases. #12456: "Cursor shows incorrect data - DML does not affect, probably caching" This patch implements a high-granularity read view to be used with cursors. In this high-granularity consistent read view modifications done by the creating transaction after the cursor is created or future transactions are not visible. But those modifications that transaction did before the cursor was created are visible. #12701: "Support >4GB buffer pool and log files on 64-bit Windows" Do not call os_file_create_tmpfile() at runtime. Instead, create all tempfiles at startup and guard access to them with mutexes. #13778: "If FOREIGN_KEY_CHECKS=0, one can create inconsistent FOREIGN KEYs". When FOREIGN_KEY_CHECKS=0 we still need to check that datatypes between foreign key references are compatible. #14189: "VARBINARY and BINARY variables: trailing space ignored with InnoDB" innobase_init(): Assert that DATA_MYSQL_BINARY_CHARSET_COLL == my_charset_bin.number. dtype_get_pad_char(): Do not pad VARBINARY or BINARY columns. row_ins_cascade_calc_update_vec(): Refuse ON UPDATE CASCADE when trying to change the length of a VARBINARY column that refers to or is referenced by a BINARY column. BINARY columns are no longer padded on comparison, and thus they cannot be padded on storage either. #14747: "Race condition can cause btr_search_drop_page_hash_index() to crash" Note that buf_block_t::index should be protected by btr_search_latch or an s-latch or x-latch on the index page. btr_search_drop_page_hash_index(): Read block->index while holding btr_search_latch and use the cached value in the loop. Remove some redundant assertions. #15108: "mysqld crashes when innodb_log_file_size is set > 4G" #15308: "Problem of Order with Enum Column in Primary Key" #15550: "mysqld crashes in printing a FOREIGN KEY error in InnoDB" row_ins_foreign_report_add_err(): When printing the parent record, use the index in the parent table rather than the index in the child table. #15653: "Slow inserts to InnoDB if many thousands of .ibd files" Keep track on unflushed modifications to file spaces. When there are tens of thousands of file spaces, flushing all files in fil_flush_file_spaces() would be very slow. fil_flush_file_spaces(): Only flush unflushed file spaces. fil_space_t, fil_system_t: Add a list of unflushed spaces. #15991: "innodb-file-per-table + symlink database + rename = cr" os_file_handle_error(): Map the error codes EXDEV, ENOTDIR, and EISDIR to the new code OS_FILE_PATH_ERROR. Treat this code as OS_FILE_PATH_ERROR. This fixes the crash on RENAME TABLE when the .ibd file is a symbolic link to a different file system. #16157: "InnoDB crashes when main location settings are empty" This patch is from Heikki. #16298: "InnoDB segfaults in INSERTs in upgrade of 4.0 -> 5.0 tables with VARCHAR BINARY" dict_load_columns(): Set the charset-collation code DATA_MYSQL_BINARY_CHARSET_COLL for those binary string columns that lack a charset-collation code, i.e., the tables were created with an older version of MySQL/InnoDB than 4.1.2. #16229: "MySQL/InnoDB uses full explicit table locks in trigger processing" Take a InnoDB table lock only if user has explicitly requested a table lock. Added some additional comments to store_lock() and external_lock(). #16387: "InnoDB crash when dropping a foreign key <table>_ibfk_0" Do not mistake TABLENAME_ibfk_0 for auto-generated id. dict_table_get_highest_foreign_id(): Ignore foreign constraint identifiers starting with the pattern TABLENAME_ibfk_0. #16582: "InnoDB: Error in an adaptive hash index pointer to page" Account for a race condition when dropping the adaptive hash index for a B-tree page. btr_search_drop_page_hash_index(): Retry the operation if a hash index with different parameters was built meanwhile. Add diagnostics for the case that hash node pointers to the page remain. btr_search_info_update_hash(), btr_search_info_update_slow(): Document the parameter "info" as in/out. #16814: "SHOW INNODB STATUS format error in LATEST FOREIGN KEY ERROR section" Add a missing newline to the LAST FOREIGN KEY ERROR section in SHOW INNODB STATUS output. dict_foreign_error_report(): Always print a newline after invoking dict_print_info_on_foreign_key_in_create_format(). #16827: "Better InnoDB error message if ibdata files omitted from my.cnf" #17126: "CHECK TABLE on InnoDB causes a short hang during check of adaptive hash" CHECK TABLE blocking other queries, by releasing the btr_search_latch periodically during the adaptive hash table validation. #17405: "Valgrind: conditional jump or move depends on unititialised values" buf_block_init(): Reset magic_n, buf_fix_count and io_fix to avoid testing uninitialized variables.	20 years ago
brian@zim.(none)	2a7c71e309	Changes to create storage directory for storage engines.	21 years ago
marko@hundin.mysql.fi	a5cca9df64	InnoDB cleanup: eliminate IB__FILE__	22 years ago
heikki@donna.mysql.fi	1d7747aa25	ut0mem.c Merge changes in InnoDB-3.23.43b ut0ut.c Merge changes in InnoDB-3.23.43b trx0purge.c Merge changes in InnoDB-3.23.43b trx0rec.c Merge changes in InnoDB-3.23.43b trx0trx.c Merge changes in InnoDB-3.23.43b trx0undo.c Merge changes in InnoDB-3.23.43b thr0loc.c Merge changes in InnoDB-3.23.43b sync0arr.c Merge changes in InnoDB-3.23.43b sync0rw.c Merge changes in InnoDB-3.23.43b sync0sync.c Merge changes in InnoDB-3.23.43b srv0srv.c Merge changes in InnoDB-3.23.43b srv0start.c Merge changes in InnoDB-3.23.43b row0ins.c Merge changes in InnoDB-3.23.43b row0mysql.c Merge changes in InnoDB-3.23.43b row0purge.c Merge changes in InnoDB-3.23.43b row0sel.c Merge changes in InnoDB-3.23.43b row0umod.c Merge changes in InnoDB-3.23.43b row0upd.c Merge changes in InnoDB-3.23.43b row0vers.c Merge changes in InnoDB-3.23.43b rem0cmp.c Merge changes in InnoDB-3.23.43b que0que.c Merge changes in InnoDB-3.23.43b pars0opt.c Merge changes in InnoDB-3.23.43b pars0pars.c Merge changes in InnoDB-3.23.43b lexyy.c Merge changes in InnoDB-3.23.43b pars0grm.c Merge changes in InnoDB-3.23.43b page0page.c Merge changes in InnoDB-3.23.43b os0file.c Merge changes in InnoDB-3.23.43b mtr0log.c Merge changes in InnoDB-3.23.43b mem0pool.c Merge changes in InnoDB-3.23.43b log0log.c Merge changes in InnoDB-3.23.43b log0recv.c Merge changes in InnoDB-3.23.43b lock0lock.c Merge changes in InnoDB-3.23.43b ibuf0ibuf.c Merge changes in InnoDB-3.23.43b fil0fil.c Merge changes in InnoDB-3.23.43b dict0crea.c Merge changes in InnoDB-3.23.43b dict0dict.c Merge changes in InnoDB-3.23.43b dict0load.c Merge changes in InnoDB-3.23.43b dict0mem.c Merge changes in InnoDB-3.23.43b data0data.c Merge changes in InnoDB-3.23.43b data0type.c Merge changes in InnoDB-3.23.43b buf0buf.c Merge changes in InnoDB-3.23.43b buf0lru.c Merge changes in InnoDB-3.23.43b btr0btr.c Merge changes in InnoDB-3.23.43b btr0cur.c Merge changes in InnoDB-3.23.43b btr0pcur.c Merge changes in InnoDB-3.23.43b btr0sea.c Merge changes in InnoDB-3.23.43b data0type.ic Merge changes in InnoDB-3.23.43b dict0dict.ic Merge changes in InnoDB-3.23.43b mtr0mtr.ic Merge changes in InnoDB-3.23.43b row0upd.ic Merge changes in InnoDB-3.23.43b sync0ipm.ic Merge changes in InnoDB-3.23.43b sync0rw.ic Merge changes in InnoDB-3.23.43b sync0sync.ic Merge changes in InnoDB-3.23.43b trx0rseg.ic Merge changes in InnoDB-3.23.43b btr0pcur.ic Merge changes in InnoDB-3.23.43b buf0buf.ic Merge changes in InnoDB-3.23.43b data0data.ic Merge changes in InnoDB-3.23.43b row0upd.h Merge changes in InnoDB-3.23.43b srv0srv.h Merge changes in InnoDB-3.23.43b sync0arr.h Merge changes in InnoDB-3.23.43b sync0rw.h Merge changes in InnoDB-3.23.43b sync0sync.h Merge changes in InnoDB-3.23.43b trx0trx.h Merge changes in InnoDB-3.23.43b ut0mem.h Merge changes in InnoDB-3.23.43b data0data.h Merge changes in InnoDB-3.23.43b data0type.h Merge changes in InnoDB-3.23.43b db0err.h Merge changes in InnoDB-3.23.43b dict0crea.h Merge changes in InnoDB-3.23.43b dict0dict.h Merge changes in InnoDB-3.23.43b dict0load.h Merge changes in InnoDB-3.23.43b dict0mem.h Merge changes in InnoDB-3.23.43b dict0types.h Merge changes in InnoDB-3.23.43b fil0fil.h Merge changes in InnoDB-3.23.43b ibuf0ibuf.h Merge changes in InnoDB-3.23.43b lock0lock.h Merge changes in InnoDB-3.23.43b log0log.h Merge changes in InnoDB-3.23.43b mtr0mtr.h Merge changes in InnoDB-3.23.43b rem0cmp.h Merge changes in InnoDB-3.23.43b row0ins.h Merge changes in InnoDB-3.23.43b row0mysql.h Merge changes in InnoDB-3.23.43b btr0cur.h Merge changes in InnoDB-3.23.43b btr0pcur.h Merge changes in InnoDB-3.23.43b btr0sea.h Merge changes in InnoDB-3.23.43b buf0buf.h Merge changes in InnoDB-3.23.43b sql_table.cc Merge changes in InnoDB-3.23.43b sql_db.cc Merge changes in InnoDB-3.23.43b ha_innobase.cc Merge changes in InnoDB-3.23.43b handler.cc Merge changes in InnoDB-3.23.43b ha_innobase.h Merge changes in InnoDB-3.23.43b handler.h Merge changes in InnoDB-3.23.43b	24 years ago
monty@donna.mysql.com	c533308a15	Added Innobase to source distribution	25 years ago

5 Commits (dc7bc64483c50d9c4588f7e9b14a663b0a5f026a)