mariadb

Tree: 1a360822bd

Author	SHA1	Message	Date
marko	1a360822bd	branches/zip: Prevent infinite B-tree page splits by ensuring that there will always be enough space for two node pointer records in an empty B-tree page. This was reported as Mantis issue #73. page_zip_rec_needs_ext(): Add the parameter n_fields, for accurate estimation of the compressed size of the data dictionary information. Given that this function is only invoked for records on leaf pages, require that there be enough space for one record in the compressed page. We check elsewhere that there will be enough room for two node pointer records on higher-level pages. btr_cur_optimistic_insert(): Ensure that there will be enough room for two node pointer records on an empty non-leaf page. The rule for leaf-page records will be enforced by the callers of page_zip_rec_needs_ext(). btr_cur_pessimistic_insert(): Remove the insufficient check that the leaf page record should be compressible by itself. Instead, now we require that two node pointer records fit on a non-leaf page, and one record will fit in uncompressed form on the leaf page. page_zip_write_header(), page_zip_write_rec(): Re-enable the debug assertions that were violated by the insufficient check in btr_cur_pessimistic_insert(). innodb_bug36172.test: Use a larger compressed page size.	17 years ago
vasil	733e615e6d	branches/zip: Limit the number of the pages that are sampled so it is never greater than the total number of pages in the index. The parameter that specifies the number of pages to test is global for all tables. By limiting it this way we allow the user to set it "high" to suit "large" tables and to avoid unnecessary work for "small" tables (e.g. doing 100 dives in a table that has 5 pages, obviously testing some pages more than once). Suggested by: Ken Approved by: Marko	17 years ago
marko	5d47801078	branches/zip: btr_cur_insert_if_possible(): Correct the function comment.	17 years ago
vasil	4821672479	branches/zip: The cardinality of every index (the number of different key values) is calculated when the table is opened, at SHOW TABLE STATUS, ANALYZE TABLE and on other circumstances (like when the table has changed too much). Note that if the mysql client is running with the auto-rehash setting turned on (default) this causes all tables to be opened when it starts. Previously InnoDB sampled 8 random pages from the index to get an estimate of the cardinality. Now the number of sampled pages can be changed via the global parameter innodb_stats_sample_pages which can be tuned at runtime. The default value for this parameter is 8. If the value of this parameter is changed, there may be serious problems: - small values (say, 1) can cause an error in table stats; - values much larger than 8 (say, 100), can cause a big slowdown in table opening time, SHOW TABLE status, etc. - query plans may be different from the old ones. Approved by: Heikki	17 years ago
marko	046c1d91b8	branches/zip: Fix terminology that was introduced in r2569: replace rbmode and "rollback mode" with rb_ctx and "rollback context". Suggested by Sunny.	17 years ago
marko	8a73d35603	branches/zip: In the rollback of incomplete transactions after crash recovery, tolerate clustered index records whose externally stored columns have not been written. This should remove the assertion failures that were reported as Mantis issue#58, issue#62, issue#64. trx_is_recv(): New function: TRUE if this transaction is rolling back an incomplete transaction in crash recovery. enum trx_rbmode: Rollback modes: no rollback, normal rollback, crash recovery. btr_cur_pessimistic_delete(), btr_free_externally_stored_field(), btr_rec_free_externally_stored_fields(): Replace the ibool parameter with enum trx_rbmode. btr_free_externally_stored_field(): If field_ref is zero, return but assert ut_a(rbmode == RB_RECOVERY). Unless InnoDB has crashed while inserting a clustered index record, field_ref should not be zero. btr_rec_free_updated_extern_fields(): Add the parameter enum trx_rbmode. btr_cur_pessimistic_update(): Pass the rbmode parameter to btr_rec_free_updated_extern_fields(). row_undo_ins(), row_undo_mod_upd_del_sec(): If row_build_index_entry() fails, assert trx_is_recv() and skip this secondary index. row_undo_mod_upd_del_sec(): Empty the heap at the end of each loop iteration in order to conserve memory and to reduce the number of low-level memory allocations.	17 years ago
vasil	59790220f8	branches/zip: Fix typo in comment.	17 years ago
marko	e3b2541287	branches/zip: Rename ib_ulonglong to ib_uint64_t and ib_longlong to ib_int64_t.	18 years ago
marko	5dfe4d50bc	branches/zip: Fix a bug in the retrieval of old versions of records containing externally stored columns. innodb-zip.test: Correct the test case. Without the fixes, the test would fail, because the BLOB would be prepended with a 768-byte prefix of the data. row_upd_index_replace_new_col_vals_index_pos(), row_upd_index_replace_new_col_vals(): Use only one "heap" parameter that must be non-NULL. When fetching externally stored columns, use upd_field_t::orig_len. upd_get_field_by_field_no(): New accessor function, for retrieving an field from an update vector by field_no. row_upd_index_replace_new_col_val(): New function, for replacing the value from an update vector. This used to be duplicated code in row_upd_index_replace_new_col_vals_index_pos() and row_upd_index_replace_new_col_vals().	18 years ago
marko	7c119c9580	branches/zip: Improve the LRU algorithm with a separate unzip_LRU list of blocks that contains uncompressed and compressed frames. This patch was designed by Heikki and Inaam, implemented by Inaam, and refined and reviewed by Marko and Sunny. buf_buddy_n_frames, buf_buddy_min_n_frames, buf_buddy_max_n_frames: Remove. buf_page_belongs_to_unzip_LRU(): New predicate: bpage->zip.data && buf_page_get_state(bpage) == BUF_BLOCK_FILE_PAGE. buf_pool_t, buf_block_t: Add the linked list unzip_LRU. A block in the regular LRU list is in unzip_LRU iff buf_page_belongs_to_unzip_LRU() holds. buf_LRU_free_block(): Add a third return value to refine the case "cannot free the block". buf_LRU_search_and_free_block(): Update the documentation to reflect the implementation. buf_LRU_stat_t, buf_LRU_stat_cur, buf_LRU_stat_sum, buf_LRU_stat_arr[]: Statistics for the unzip_LRU algorithm. buf_LRU_stat_update(): New function: Update the statistics. Called once per second by srv_error_monitor_thread(). buf_LRU_validate(): Validate the unzip_LRU list as well. buf_LRU_evict_from_unzip_LRU(): New predicate: Use the unzip_LRU before falling back to the regular LRU? buf_LRU_free_from_unzip_LRU_list(), buf_LRU_free_from_common_LRU_list(): Subfunctions of buf_LRU_search_and_free_block(). buf_LRU_search_and_free_block(): Reimplement. Try to evict an uncompressed page from the unzip_LRU list before falling back to evicting an entire block from the common LRU list. buf_unzip_LRU_remove_block_if_needed(): New function. buf_unzip_LRU_add_block(): New function: Add a block to the unzip_LRU list.	18 years ago
calvin	66312acc63	branches/zip: Fix the compile errors in btr/btr0cur.c for Windows build. It is part of Mantis issue#18. Reviewed by Marko.	18 years ago
marko	36a5b2cd37	branches/zip: Merge 2263:2295 from branches/5.1.	18 years ago
marko	1d1dc31a06	branches/zip: Introduce UNIV_INTERN, a linkage specifier for InnoDB-global symbols. Use it for all definitions of non-static variables and functions. lexyy.c, make_flex.sh: Declare yylex as UNIV_INTERN, not static. It is referenced from pars0grm.c. Actually, according to nm .libs/ha_innodb.so\|grep -w '[ABCE-TVXYZ]' the following symbols are still global: * The vtable for class ha_innodb * pars0grm.c: The function yyparse() and the variables yychar, yylval, yynerrs The required changes to the Bison-generated file pars0grm.c will be addressed in a separate commit, which will add a script similar to make_flex.sh. The class ha_innodb is renamed from class ha_innobase by a #define. Thus, there will be no clash with the builtin InnoDB. However, there will be some overhead for invoking virtual methods of class ha_innodb. Ideas for making the vtable hidden are welcome. -fvisibility=hidden is not available in GCC 3.	18 years ago
marko	c9087d7d90	branches/zip: Apply some fixes suggested by Heikki. btr_cur_pessimistic_update(): Note why the externally stored columns of a record on a latched page cannot have been purged. trx_undo_get_undo_rec(): Clarify that the stack of versions is locked all the way down to the purge view. trx_undo_prev_version_build(): Set *old_vers = NULL also when the record could have been purged already. Add some clarifying comments.	18 years ago
marko	0a63ed9c4e	branches/zip: Introduce the accessor function dict_index_is_ibuf() for determining if an index is the insert buffer B-tree.	18 years ago
marko	8958f06278	branches/zip: Introduce the page type code FIL_PAGE_TYPE_ZBLOB2 for continuation pages containing compressed BLOBs. The first compressed BLOB page will be of type FIL_PAGE_TYPE_ZBLOB.	18 years ago
marko	b52a30e09d	branches/zip: When storing a longer prefix of an externally stored column to the undo log, also store the original length of the column, so that the changes will be correctly undone in transaction rollback or when fetching previous versions of the row. innodb-zip.test: New file, for tests of the compression. upd_field_t: Add orig_len, the original length of new_val. btr_push_update_extern_fields(): Restore the original prefix of the column. Add the parameter heap where memory will be allocated if necessary. trx_undo_rec_get_col_val(): Add the output parameter orig_len. trx_undo_page_report_modify_ext(): New function: Write an externally stored column to the undo log. This is only called from trx_undo_page_report_modify(), and this is the only caller of trx_undo_page_fetch_ext(). trx_undo_update_rec_get_update(): Read the original length of the column prefix to upd_field->orig_len.	18 years ago
marko	e693d5170d	branches/zip: Add comments about the lock and latch protection of externally stored columns (BLOBs). btr_copy_blob_prefix(), btr_copy_zblob_prefix(), btr_copy_externally_stored_field_prefix_low(), btr_copy_externally_stored_field_prefix(), btr_copy_externally_stored_field(), btr_rec_copy_externally_stored_field(): Note that the page containing the clustered index record that points to the BLOB must be latched. btr_copy_zblob_prefix(): Note that there is no latch on the page, and thus all accesses to a given page via this function must be covered by the same set of locks or latches. btr_copy_zblob_prefix(): Note that the block acquired by buf_page_get_zip() is protected by an exclusive table lock or or by a latch on the clustered index record.	18 years ago
marko	44340b6e8e	branches/zip: Add comments that clarify why the remaining calls to row_build(), row_upd_index_replace_new_col_vals_index_pos(), and row_upd_index_replace_new_col_vals() are safe. btr_cur_optimistic_update(), btr_cur_pessimistic_update(): Note that the B-tree page of the clustered index record is latched in mtr. trx_undo_prev_version_build(): Add const qualifiers to index_rec and rec. Note that the page of index_rec is latched in index_mtr. row_vers_impl_x_locked_off_kernel(), row_vers_old_has_index_entry(): Note that the stack of versions is locked by mtr and thus it is safe to call row_build().	18 years ago
marko	1cbf67cdbf	branches/zip: btr_copy_zblob_prefix(): Print d_stream->msg on decompression failure.	18 years ago
marko	cdb5b46445	branches/zip: Implement wrappers for all operations on the buffer pool mutex. buf_pool->mutex: Rename to buf_pool_mutex, so that the wrappers will have to be used when changes are merged from other source trees. buf_pool->zip_mutex: Rename to buf_pool_zip_mutex. buf_pool_mutex_own(), buf_pool_mutex_enter(), buf_pool_mutex_exit(): Wrappers for buf_pool_mutex.	18 years ago
marko	02819f3e55	branches/zip: btr_cur_mark_extern_inherited_fields(): Add a call to rec_offs_any_extern() as an optimization.	18 years ago
vasil	1ae59d9e65	branches/zip: Change the output format of transaction ids from 2 32bit numbers separated by space to a single hex number. Suggested by: Heikki Approved by: Heikki	18 years ago
marko	8c852371e5	branches/zip: Add some clarifying comments. btr_copy_blob_prefix(), btr_copy_externally_stored_field_prefix_low(): Document the return value as "number of bytes written", not "bytes written". trx_undo_page_fetch_ext(): Explain the assertion ut_a(ext_len). row_build_index_entry(): Explain the assertion ut_a(!ext).	18 years ago
marko	33e082be24	branches/zip: buf_buddy_alloc(): Assign *lru = TRUE whenever the buffer pool mutex is temporarily released. buf_LRU_free_block(), buf_buddy_alloc_clean(): Add an output parameter that will be assigned TRUE when the buffer pool mutex is released. This bug was spotted by and fix provided by Sunny.	18 years ago
marko	c440688147	branches/zip: btr_cur_pessimistic_update(), btr_cur_pessimistic_delete(): Use rec_offs_any_extern() as a condition for freeing externally stored columns. This is only a performance optimization.	18 years ago
marko	71cf1edf8d	branches/zip: btr_store_big_rec_extern_fields(): Note that the page number of the record containing the field reference may change.	18 years ago
marko	962ed52317	branches/zip: Clean up after r2129: univ.i: Do not define UNIV_DEBUG, UNIV_ZIP_DEBUG. btr_cur_del_unmark_for_ibuf(): Use the same comment in both btr0cur.c and btr0cur.h. Wrap long lines.	18 years ago
sunny	eca3cb2378	branches/zip: Fix a bug where the zipped page and the uncompressed page contents end up with conflicting versions of a record's state. The zipped page record was not being marked as "(un)deleted" because we were not passing the zipped page contents to the (un)delete function, which first (un)delete marks the uncompressed version and then based on whether page_zip is NULL or not (un)delete marks the record in the compressed page.	18 years ago
marko	06299dddd4	branches/zip: In purge, avoid dereferencing unset BLOB pointers of freshly inserted, uncommitted clustered index records when determining if a secondary index record that contains a column prefix of an externally stored column is referencing the clustered index record. field_ref_zero[]: A BLOB pointer full of zero, for use in comparisons. btr_copy_externally_stored_field_prefix(): Assert that the BLOB pointer is set. row_ext_lookup_ith(), row_ext_lookup(), row_ext_lookup_low(): Document that field_ref_zero is returned when the BLOB cannot be fetched. row_ext_lookup_low(): Return field_ref_zero and *len = 0 when the BLOB pointer is unset. row_build_index_entry(): Return NULL when a needed BLOB pointer cannot be dereferenced (row_ext_lookup returns field_ref_zero). Check the return value for NULL in callers. row_vers_impl_x_locked_off_kernel(): Avoid comparisons when row_build_index_entry() returns NULL. row_vers_old_has_index_entry(): Ignore records for which row_build_index_entry() returns NULL. The entry should never be NULL in rollback, but it may be NULL in purge. row_merge_buf_add(): Assert that row_ext_lookup() does not return field_ref_zero. The table will be locked during index creation.	18 years ago
marko	110781a39e	branches/zip: btr_cur_pessimistic_insert(): When calling btr_cur_optimistic_insert(), pass big_rec to it, so that the field references of externally stored columns (BLOB pointers) will not be left uninitialized after a successful optimistic insert. This bug was spotted by Sunny.	18 years ago
marko	36874d65f0	branches/zip: Fix a bug in the updates of index records that contain a column prefix of an externally stored column. row_upd_ext_fetch(): New function. row_upd_index_replace_new_col_vals(), row_upd_index_replace_new_col_vals_index_pos(): Fetch prefixes of externally stored columns when they are needed for column prefix indexes. For memory allocation, add the parameter ext_heap. Avoid repeating the inner loop after finding a matching upd_field->field_no.	18 years ago
marko	404d94b38c	branches/zip: btr_store_big_rec_extern_fields(), btr_free_externally_stored_field(): Add some page type assertions that were suggested by Sunny.	18 years ago
marko	8822b4f94f	branches/zip: btr_cur_pessimistic_insert(): For clarity, use return(DB_...) instead of assigning err = DB_... when possible.	18 years ago
marko	680586464e	branches/zip: btr_cur_pessimistic_update(): After btr_cur_pessimistic_insert(), update page_zip, because rec may be on a different page.	18 years ago
marko	d03bdd38bc	branches/zip: btr_free_externally_stored_field(): Clear rec_zip_size when rec == NULL.	18 years ago
marko	d1ef07b308	branches/zip: Check that page_zip->data is defined, not page_zip itself.	18 years ago
marko	589393da83	branches/zip: Improve Valgrind instrumentation. btr_cur_optimistic_insert(): On compressed tablespaces, check that both the compressed and the uncompressed page are completely initialized in the beginning of the function. page_zip_compress(): After successful compression, check that the compressed page is completely initialized.	18 years ago
marko	43becedf54	branches/zip: Cast void* pointers to byte* when doing pointer arithmetics.	18 years ago
marko	e7267909d7	branches/zip: Initialize dfield_t::ext as soon as possible. This should fix the bugs introduced in r1591. row_rec_to_index_entry_low(): Clear "n_ext". Do not allow it to be NULL. Add const qualifier to dict_index_t*. row_rec_to_index_entry(): Add the parameters "offsets" and "n_ext". btr_cur_optimistic_update(): Add an assertion that there are no externally stored columns. Remove the unreachable call to btr_cur_unmark_extern_fields() and the preceding unnecessary call to rec_get_offsets(). btr_push_update_extern_fields(): Remove the parameters index, offsets. Only report the additional externally stored columns of the update vector. row_build(), trx_undo_rec_get_partial_row(): Flag externally stored columns also with dfield_set_ext(). rec_copy_prefix_to_dtuple(): Assert that there are no externally stored columns in the prefix. row_build_row_ref(): Note and assert that the index is a secondary index, and assert that there are no externally stored columns. row_build_row_ref_fast(): Assert that there are no externally stored columns. rec_offs_get_n_alloc(): Expose the function. row_build_row_ref_in_tuple(): Assert that there are no externally stored columns in a record of a secondary index. row_build_row_ref_from_row(): Assert that there are no externally stored columns. row_upd_check_references_constraints(): Add the parameter offsets, to avoid a redundant call to rec_get_offsets(). row_upd_del_mark_clust_rec(): Add the parameter offsets. Remove duplicated code. row_ins_index_entry_set_vals(): Copy the external storage flag. sel_pop_prefetched_row(): Assert that there are no externally stored columns. row_scan_and_check_index(): Copy offsets to a temporary heap across the invocation of row_rec_to_index_entry().	18 years ago
marko	c6f5ea288a	branches/zip: btr_push_update_extern_fields(): Add parameter "index" and use it for flagging externally stored columns in the data tuple. The data tuple contains the same columns as the clustered index record, but in a different order. This error was introduced in r1591. TODO: the assertion ut_ad(!dfield_is_ext()) may fail in btr_cur_pessimistic_update().	18 years ago
marko	46b7cec164	branches/zip: btr_cur_mark_dtuple_inherited_extern(): Fix the loop condition that was broken in r1591. Spotted by Heikki and Vasil.	18 years ago
marko	2561797bb9	branches/zip: On compressed pages, always update the insert buffer bitmap. Do not assume anything about the contents of the bitmap. ibuf_update_free_bits_low(): Use this function only for uncompressed pages. Remove the parameter zip_size. This function avoids latching the bitmap page and updating the bitmap when the bits do not change. ibuf_update_free_bits_zip(): New function based on ibuf_update_free_bits_low(), for use on compressed pages. Remove the parameter max_insert_size that was used for computing the before image of the free bits. Always update the bitmap. ibuf_index_page_calc_free_zip(): New function, factored out from ibuf_index_page_calc_free(). ibuf_update_free_bits_if_full(): Document that this function must only be invoked on uncompressed pages. Add a debug assertion about this.	18 years ago
marko	2ddbc912e7	branches/zip: Silence most GCC 4.2.1 warnings about const pointers. For some reason, GCC 4.2.1 ignores casts (for removing constness) in calls to inline functions. page_align(), ut_align_down(): Make the parameter const void*, but still return a non-const pointer. This is ugly, but these functions cannot be replaced with a const-preserving macro in a portable way, given that the pointer argument is not always pointing to bytes. buf_block_get_page_zip(): Implement as a const-preserving macro. buf_frame_get_page_zip(), buf_block_align(): Add const qualifiers. lock_rec_get_prev(): Silence GCC 4.2.1 warnings. mlog_write_initial_log_record(), mlog_write_initial_log_record_fast(), mtr_memo_contains(): Add const qualifier to the pointer. page_header_get_ptr(): Rewrite as page_header_get_offs(), and implement as a macro that calls this function.	18 years ago
marko	3877f6910f	branches/zip: Define the macro rec_offs_init() and use it for initializing offsets_[] arrays, as suggested by Vasil. rec_offs_set_n_alloc(): Declare as a public function. Assert that n_alloc > REC_OFFS_HEADER_SIZE. rec_offs_get_n_alloc(): Assert that n_alloc > REC_OFFS_HEADER_SIZE.	18 years ago
marko	c09dd214be	branches/zip: Add const qualifiers to the upd_t* parameters of some functions.	18 years ago
marko	8634d343f7	branches/zip: Rename mem_heap_calloc() and mem_calloc() to mem_heap_zalloc() and mem_zalloc(), because calloc() in the C runtime library takes two size parameters, not one. mem_heap_zalloc(): Add debug assertions. Document that the return value is never NULL.	18 years ago
marko	f68d3656dc	branches/zip: Use mem_calloc and mem_heap_calloc when possible. row_ext_create(): After a memset() inside UNIV_DEBUG, flag the memory uninitialized with UNIV_MEM_ALLOC().	18 years ago
marko	e507ea0dd4	branches/zip: btr_cur_mark_dtuple_inherited_extern(): Initialize len. This was broken in r1591.	18 years ago
marko	632ec54002	branches/zip: Make merge sort handle externally stored columns. Some things still fail in innodb-index.test, and there seems to be a race condition (data dictionary lock wait) when running with --valgrind. dfield_t: Add an "external storage" flag, dfield->ext. dfield_is_null(), dfield_is_ext(), dfield_set_ext(), dfield_set_null(): New functions. dfield_copy(), dfield_copy_data(): Add const qualifiers, fix in/out comments. data_write_sql_null(): Use memset(). big_rec_field_t: Replace byte* data with const void* data. ut_ulint_sort(): Remove. upd_field_t: Remove extern_storage. upd_node_t: Replace ext_vec, n_ext_vec with n_ext. row_merge_copy_blobs(): New function. row_ins_index_entry(): Add the parameter "ibool foreign" for suppressing foreign key checks during fast index creation or when inserting into secondary indexes. btr_page_insert_fits(): Add const qualifiers. btr_cur_add_ext(), upd_ext_vec_contains(): Remove. dfield_print_also_hex(), dfield_print(): Replace if...else if with switch. Observe dfield_is_ext().	19 years ago

1 2 3 4

174 Commits (1a360822bd4a0015eea510d7c7835b70485f6028)