mariadb

Tree: e1df69f75a

Author	SHA1	Message	Date
Marko Mäkelä	2c67d5066d	Revert revno:3452.71.32 (Bug#12612184 fix). Bug#12612184 RACE CONDITION AFTER BTR_CUR_PESSIMISTIC_UPDATE() The fix introduced potentially more severe crash recovery problems than the bug causes. Revert the fix for now.	14 years ago
Marko Mäkelä	91b5e9352a	Revert most of revno 3560.9.1 (Bug#12704861) This was an attempt to address problems with the Bug#12612184 fix. Even with this follow-up fix, crash recovery can be broken. Let us fix the bug later.	14 years ago
Marko Mäkelä	41f229cd9e	Bug#12704861 Corruption after a crash during BLOB update The fix of Bug#12612184 broke crash recovery. When a record that contains off-page columns (BLOBs) is updated, we must first write redo log about the BLOB page writes, and only after that write the redo log about the B-tree changes. The buggy fix would log the B-tree changes first, meaning that after recovery, we could end up having a record that contains a null BLOB pointer. Because we will be redo logging the writes off the off-page columns before the B-tree changes, we must make sure that the pages chosen for the off-page columns are free both before and after the B-tree changes. In this way, the worst thing that can happen in crash recovery is that the BLOBs are written to free pages, but the B-tree changes are not applied. The BLOB pages would correctly remain free in this case. To achieve this, we must allocate the BLOB pages in the mini-transaction of the B-tree operation. A further quirk is that BLOB pages are allocated from the same file segment as leaf pages. Because of this, we must temporarily "hide" any leaf pages that were freed during the B-tree operation by "fake allocating" them prior to writing the BLOBs, and freeing them again before the mtr_commit() of the B-tree operation, in btr_mark_freed_leaves(). btr_cur_mtr_commit_and_start(): Remove this faulty function that was introduced in the Bug#12612184 fix. The problem that this function was trying to address was that when we did mtr_commit() the BLOB writes before the mtr_commit() of the update, the new BLOB pages could have overwritten clustered index B-tree leaf pages that were freed during the update. If recovery applied the redo log of the BLOB writes but did not see the log of the record update, the index tree would be corrupted. The correct solution is to make the freed clustered index pages unavailable to the BLOB allocation. This function is also a likely culprit of InnoDB hangs that were observed when testing the Bug#12612184 fix. btr_mark_freed_leaves(): Mark all freed clustered index leaf pages of a mini-transaction allocated (nonfree=TRUE) before storing the BLOBs, or freed (nonfree=FALSE) before committing the mini-transaction. btr_freed_leaves_validate(): A debug function for checking that all clustered index leaf pages that have been marked free in the mini-transaction are consistent (have not been zeroed out). btr_page_alloc_low(): Refactored from btr_page_alloc(). Return the number of the allocated page, or FIL_NULL if out of space. Add the parameter "mtr_t* init_mtr" for specifying the mini-transaction where the page should be initialized, or if this is a "fake allocation" (init_mtr=NULL) by btr_mark_freed_leaves(nonfree=TRUE). btr_page_alloc(): Add the parameter init_mtr, allowing the page to be initialized and X-latched in a different mini-transaction than the one that is used for the allocation. Invoke btr_page_alloc_low(). If a clustered index leaf page was previously freed in mtr, remove it from the memo of previously freed pages. btr_page_free(): Assert that the page is a B-tree page and it has been X-latched by the mini-transaction. If the freed page was a leaf page of a clustered index, link it by a MTR_MEMO_FREE_CLUST_LEAF marker to the mini-transaction. btr_store_big_rec_extern_fields_func(): Add the parameter alloc_mtr, which is NULL (old behaviour in inserts) and the same as local_mtr in updates. If alloc_mtr!=NULL, the BLOB pages will be allocated from it instead of the mini-transaction that is used for writing the BLOBs. fsp_alloc_from_free_frag(): Refactored from fsp_alloc_free_page(). Allocate the specified page from a partially free extent. fseg_alloc_free_page_low(), fseg_alloc_free_page_general(): Add the parameter "mtr_t* init_mtr" for specifying the mini-transaction where the page should be initialized, or NULL if this is a "fake allocation" that prevents the reuse of a previously freed B-tree page for BLOB storage. If init_mtr==NULL, try harder to reallocate the specified page and assert that it succeeded. fsp_alloc_free_page(): Add the parameter "mtr_t* init_mtr" for specifying the mini-transaction where the page should be initialized. Do not allow init_mtr == NULL, because this function is never to be used for "fake allocations". mtr_t: Add the operation MTR_MEMO_FREE_CLUST_LEAF and the flag mtr->freed_clust_leaf for quickly determining if any MTR_MEMO_FREE_CLUST_LEAF operations have been posted. row_ins_index_entry_low(): When columns are being made off-page in insert-by-update, invoke btr_mark_freed_leaves(nonfree=TRUE) and pass the mini-transaction as the alloc_mtr to btr_store_big_rec_extern_fields(). Finally, invoke btr_mark_freed_leaves(nonfree=FALSE) to avoid leaking pages. row_build(): Correct a comment, and add a debug assertion that a record that contains NULL BLOB pointers must be a fresh insert. row_upd_clust_rec(): When columns are being moved off-page, invoke btr_mark_freed_leaves(nonfree=TRUE) and pass the mini-transaction as the alloc_mtr to btr_store_big_rec_extern_fields(). Finally, invoke btr_mark_freed_leaves(nonfree=FALSE) to avoid leaking pages. buf_reset_check_index_page_at_flush(): Remove. The function fsp_init_file_page_low() already sets bpage->check_index_page_at_flush=FALSE. There is a known issue in tablespace extension. If the request to allocate a BLOB page leads to the tablespace being extended, crash recovery could see BLOB writes to pages that are off the tablespace file bounds. This should trigger an assertion failure in fil_io() at crash recovery. The safe thing would be to write redo log about the tablespace extension to the mini-transaction of the BLOB write, not to the mini-transaction of the record update. However, there is no redo log record for file extension in the current redo log format. rb:693 approved by Sunny Bains	14 years ago
Marko Mäkelä	5b4ceba58d	Bug#12612184 Race condition after btr_cur_pessimistic_update() btr_cur_compress_if_useful(), btr_compress(): Add the parameter ibool adjust. If adjust=TRUE, adjust the cursor position after compressing the page. btr_lift_page_up(): Return a pointer to the father page. BTR_KEEP_POS_FLAG: A new flag for btr_cur_pessimistic_update(). btr_cur_pessimistic_update(): If big_rec != NULL and flags & BTR_KEEP_POS_FLAG, keep the cursor positioned on the updated record. Also, do not release the index tree x-lock if big_rec != NULL. btr_cur_mtr_commit_and_start(): Commits and restarts a mini-transaction so that it will retain an x-lock on index->lock and the page of the cursor. This is invoked when btr_cur_pessimistic_update() returns big_rec != NULL. In all callers of btr_cur_pessimistic_update() that do not pass BTR_KEEP_POS_FLAG, assert that big_rec == NULL. btr_cur_compress(): Unused function [in the built-in MySQL 5.1], remove. page_rec_get_nth(): Return the nth record on the page (an inverse function of page_rec_get_n_recs_before()). Refactored from page_get_middle_rec(). page_get_middle_rec(): Invoke page_rec_get_nth(). page_cur_insert_rec_zip_reorg(): Make use of the page directory shortcuts in page_rec_get_nth() instead of scanning the whole list of records. row_ins_clust_index_entry_by_modify(): Pass BTR_KEEP_POS_FLAG to btr_cur_pessimistic_update(). row_ins_index_entry_low(): If row_ins_clust_index_entry_by_modify() returns a big_rec, invoke btr_cur_mtr_commit_and_start() in order to commit and start the mini-transaction without releasing the x-locks on index->lock and the cursor page, and write the big_rec. Releasing the page latch in mtr_commit() caused a race condition. row_upd_clust_rec(): Pass BTR_KEEP_POS_FLAG to btr_cur_pessimistic_update(). If it returns a big_rec, invoke btr_cur_mtr_commit_and_start() in order to commit and start the mini-transaction without releasing the x-locks on index->lock and the cursor page, and write the big_rec. Releasing the page latch in mtr_commit() caused a race condition. sync_thread_add_level(): Add the parameter ibool relock. When TRUE, bypass the latching order rules. rw_lock_add_debug_info(): For nested X-lock requests, pass relock=TRUE to sync_thread_add_level(). rb:678 approved by Jimmy Yang	15 years ago
Marko Mäkelä	5adf2313f7	Bug #55284 diagnostics: Introduce UNIV_BLOB_LIGHT_DEBUG, enabled by UNIV_DEBUG btr_rec_get_field_ref_offs(), btr_rec_get_field_ref(): New functions. Get the pointer to an externally stored field. btr_cur_set_ownership_of_extern_field(): Assert that the BLOB has not already been disowned. btr_store_big_rec_extern_fields(): Rename to btr_store_big_rec_extern_fields_func() and add the debug parameter update_in_place. All pointers to externally stored columns in the record must either be zero or they must be pointers to inherited columns, owned by this record or an earlier record version. For any BLOB that is stored, the BLOB pointer must previously have been zero. When the function completes, all BLOB pointers must be nonzero and owned by the record. rb://549 approved by Jimmy Yang	15 years ago
Jimmy Yang	9cd4d49840	Fix Bug#30423 "InnoDBs treatment of NULL in index stats causes bad "rows examined" estimates". This change implements "innodb_stats_method" with options of "nulls_equal", "nulls_unequal" and "null_ignored". rb://553 approved by Marko	15 years ago
Marko Mäkelä	721a890e48	Bug #55284 Double BLOB free due to lock wait while updating PRIMARY KEY This bug fix requires that Bug #58912 be fixed as well (bzr revision id marko.makela@oracle.com-20101221093919-mcmmgd4zpse9567d). Otherwise, another double BLOB free could occur when InnoDB would try to perform an update-in-place as delete-and-insert-by-update-in-place. row_upd_clust_rec_by_insert(): Do not disown the externally stored columns from the old record (btr_cur_mark_extern_inherited_fields()) until after checking the foreign key constraints and successfully inserting the updated record. If a lock wait timeout occurs between the delete-marking of the old record and the insertion of the updated record, mark the columns inherited before retrying the insert. Distinguish the state UPD_NODE_INSERT_BLOB from UPD_NODE_INSERT_CLUSTERED. btr_cur_del_mark_set_clust_rec(): Replace the cursor with block,rec,index,offsets so that the offsets need not be recalculated. Assert that rec is on a clustered index leaf page. btr_cur_disown_inherited_fields(): Renamed from btr_cur_mark_extern_inherited_fields(). Use upd_get_field_by_field_no(). Assert that there are externally stored columns. Assert that a mini-transaction is passed. Remove the return status. (The only caller, row_upd_clust_rec_by_insert(), will have determined that some fields have changed ownership.) btr_cur_mark_dtuple_inherited_extern(): Rename to row_upd_clust_rec_by_insert_inherit_func() and declare as static. Add the debug parameters rec, offsets. When rec is given, assert that the off-page columns match those in the inesrt tuple and that the off-page columns are owned by the record. Assert that the non-updated off-page columns in the insert tuple are owned, and mark them inherited. row_upd_clust_rec_by_insert_inherit(): A wrapper macro for row_upd_clust_rec_by_insert_inherit_func(). row_undo_mod_upd_exist_sec(): Adjust a comment about row_upd_clust_rec_by_insert(). rb:508 approved by Jimmy Yang	15 years ago
Marko Mäkelä	eab53d4e80	Non-functional change: Remove bogus const qualifiers and make some function comments more accurate.	15 years ago
Marko Mäkelä	c38071ec18	Bug #56680 wrong InnoDB results from a case-insensitive covering index row_search_for_mysql(): When a secondary index record might not be visible in the current transaction's read view and we consult the clustered index and optionally some undo log records, return the relevant columns of the clustered index record to MySQL instead of the secondary index record. ibuf_insert_to_index_page_low(): New function, refactored from ibuf_insert_to_index_page(). ibuf_insert_to_index_page(): When we are inserting a record in place of a delete-marked record and some fields of the record differ, update that record just like row_ins_sec_index_entry_by_modify() would do. btr_cur_update_alloc_zip(): Make the function public. mysql_row_templ_t: Add clust_rec_field_no. row_sel_store_mysql_rec(), row_sel_push_cache_row_for_mysql(): Add the flag rec_clust, for returning data at clust_rec_field_no instead of rec_field_no. Resurrect the debug assertion that the record not be marked for deletion. (Bug #55626) [UNIV_DEBUG \|\| UNIV_IBUF_DEBUG] ibuf_debug, buf_page_get_gen(), buf_flush_page_try(): Implement innodb_change_buffering_debug=1 for evicting pages from the buffer pool, so that change buffering will be attempted more frequently.	15 years ago
Sunny Bains	b37256b1ab	Fix bug# 55543 - InnoDB Plugin: Signal 6: Assertion failure in file fil/fil0fil.c line 4306 The bug is due to a double delete of a BLOB, once via: rollback -> btr_cur_pessimistic_delete() and the second time via purge. The bug is in row_upd_clust_rec_by_insert(). There we relinquish ownership of the non-updated BLOB columns in btr_cur_mark_extern_inherited_fields() before building the row entry that will be inserted and whose contents will be logged in the UNDO log. However, we don't set the BLOB column later to INHERITED so that a possible rollback will not free the original row's non-updated BLOB entries. This is because the condition that checks for that is in : if (node->upd_ext) {}. node->upd_ext is non-NULL only if a BLOB column was updated and that column is part of some key ordering (see row_upd_replace()). This results in the non-update BLOB columns being deleted during a rollback and subsequently by purge again. rb://413	16 years ago
Marko Mäkelä	62084feb55	Bug#54358: READ UNCOMMITTED access failure of off-page DYNAMIC or COMPRESSED columns When the server crashes after a record stub has been inserted and before all its off-page columns have been written, the record will contain incomplete off-page columns after crash recovery. Such records may only be accessed at the READ UNCOMMITTED isolation level or when rolling back a recovered transaction in recv_recovery_rollback_active(). Skip these records at the READ UNCOMMITTED isolation level. TODO: Add assertions for checking the above assumptions hold when an incomplete BLOB is encountered. btr_rec_copy_externally_stored_field(): Return NULL if the field is incomplete. row_prebuilt_t::templ_contains_blob: Clarify what "BLOB" means in this context. Hint: MySQL BLOBs are not the same as InnoDB BLOBs. row_sel_store_mysql_rec(): Return FALSE if not all columns could be retrieved. Previously this function always returned TRUE. Assert that the record is not delete-marked. row_sel_push_cache_row_for_mysql(): Return FALSE if not all columns could be retrieved. row_search_for_mysql(): Skip records containing incomplete off-page columns. Assert that the transaction isolation level is READ UNCOMMITTED. rb://380 approved by Jimmy Yang	16 years ago
Sergey Vojtovich	44a0ff6c76	Applying InnoDB snapshot Detailed revision comments: r6749 \| vasil \| 2010-02-20 18:45:41 +0200 (Sat, 20 Feb 2010) \| 5 lines Non-functional change: update copyright year to 2010 of the files that have been modified after 2010-01-01 according to svn. for f in $(svn log -v -r{2010-01-01}:HEAD \|grep "^ M " \|cut -b 16- \|sort -u) ; do sed -i "" -E 's/(Copyright $c$ [0-9]{4},) [0-9]{4}, (.*Innobase Oy.+All Rights Reserved)/\1 2010, \2/' $f ; done	16 years ago
Sergey Vojtovich	30ddb931ef	Applying InnoDB snapshot Detailed revision comments: r6559 \| marko \| 2010-02-04 13:21:18 +0200 (Thu, 04 Feb 2010) \| 14 lines branches/zip: Pass the file name and line number of the caller of the b-tree cursor functions to the buffer pool requests, in order to make the latch diagnostics more accurate. buf_page_optimistic_get_func(): Renamed to buf_page_optimistic_get(). btr_page_get_father_node_ptr(), btr_insert_on_non_leaf_level(), btr_pcur_open(), btr_pcur_open_with_no_init(), btr_pcur_open_on_user_rec(), btr_pcur_open_at_rnd_pos(), btr_pcur_restore_position(), btr_cur_open_at_index_side(), btr_cur_open_at_rnd_pos(): Rename the function to _func and add the parameters file, line. Define wrapper macros with __FILE__, __LINE__. btr_cur_search_to_nth_level(): Add the parameters file, line.	16 years ago
Jonathan Perkin	00eec234ee	Apply build fix	17 years ago
Alexander Nozdrin	b05eeab593	Fix 5.1 build error.	17 years ago
Satya B	3945d5e554	Adding innodb_plugin-1.0.4 as storage/innodb_plugin.	17 years ago

15 Commits (e1df69f75ac478b0456bc768b61dfd412fa510e9)