mariadb

Tree: d609478ae1

Author	SHA1	Message	Date
marko	e3b2541287	branches/zip: Rename ib_ulonglong to ib_uint64_t and ib_longlong to ib_int64_t.	18 years ago
marko	a9ae7311ec	branches/zip: Implement the configuration parameter and settable global variable innodb_file_format. Implement file format version stamping of *.ibd files and SYS_TABLES.TYPE. This change breaks introduces an incompatible change for for compressed tables. We can do this, as we have not released yet. innodb-zip.test: Add tests for stricter KEY_BLOCK_SIZE and ROW_FORMAT checks. DICT_TF_COMPRESSED_MASK, DICT_TF_COMPRESSED_SHIFT: Replace with DICT_TF_ZSSIZE_MASK, DICT_TF_ZSSIZE_SHIFT. DICT_TF_FORMAT_MASK, DICT_TF_FORMAT_SHIFT, DICT_TF_FORMAT_51, DICT_TF_FORMAT_ZIP: File format version, stored in table->flags, in the .ibd file header, and in SYS_TABLES.TYPE. dict_create_sys_tables_tuple(): Write the table flags to SYS_TABLES.TYPE if the format is at least DICT_TF_FORMAT_ZIP. For old formats (DICT_TF_FORMAT_51), write DICT_TABLE_ORDINARY as the table type. DB_TABLE_ZIP_NO_IBD: Remove the error code. The error handling is done in ha_innodb.cc; as a failsafe measure, dict_build_table_def_step() will silently clear the compression and format flags instead of returning this error. dict_mem_table_create(): Assert that no extra bits are set in the flags. dict_sys_tables_get_zip_size(): Rename to dict_sys_tables_get_flags(). Check all flag bits, and return ULINT_UNDEFINED if the combination is unsupported. dict_boot(): Document the SYS_TABLES columns N_COLS and TYPE. dict_table_get_format(), dict_table_set_format(), dict_table_flags_to_zip_size(): New accessors to table->flags. dtuple_convert_big_rec(): Introduce the auxiliary variables local_len, local_prefix_len. Store a 768-byte prefix locally if the file format is less than DICT_TF_FORMAT_ZIP. dtuple_convert_back_big_rec(): Restore the columns. srv_file_format: New variable: innodb_file_format. fil_create_new_single_table_tablespace(): Replace the parameter zip_size with table->flags. fil_open_single_table_tablespace(): Replace the parameter zip_size_in_k with table->flags. Check the flags. fil_space_struct, fil_space_create(), fil_op_write_log(): Replace zip_size with flags. fil_node_open_file(): Note a TODO item for InnoDB Hot Backup. Check that the tablespace flags match. fil_space_get_zip_size(): Rename to fil_space_get_flags(). Add a wrapper for fil_space_get_zip_size(). fsp_header_get_flags(): New function. fsp_header_init_fields(): Replace zip_size with flags. FSP_SPACE_FLAGS: New name for the tablespace flags. This field used to be called FSP_PAGE_ZIP_SIZE, or FSP_LOWEST_NO_WRITE. It has always been written as 0 in MySQL/InnoDB versions 4.1 to 5.1. MLOG_ZIP_FILE_CREATE: Rename to MLOG_FILE_CREATE2. Add a 32-bit parameter for the tablespace flags. ha_innobase::create(): Check the table attributes ROW_FORMAT and KEY_BLOCK_SIZE. Issue errors if they are inappropriate, or warnings if the inherited attributes (in ALTER TABLE) will be ignored. PAGE_ZIP_MIN_SIZE_SHIFT: New constant: the 2-logarithm of PAGE_ZIP_MIN_SIZE.	18 years ago
marko	58fd174a95	branches/zip: ibuf_update_free_bits_zip(): When after==0, call buf_page_make_young(), like ibuf_update_free_bits_if_full() does. This difference was pointed out by Sunny.	18 years ago
marko	1d1dc31a06	branches/zip: Introduce UNIV_INTERN, a linkage specifier for InnoDB-global symbols. Use it for all definitions of non-static variables and functions. lexyy.c, make_flex.sh: Declare yylex as UNIV_INTERN, not static. It is referenced from pars0grm.c. Actually, according to nm .libs/ha_innodb.so\|grep -w '[ABCE-TVXYZ]' the following symbols are still global: * The vtable for class ha_innodb * pars0grm.c: The function yyparse() and the variables yychar, yylval, yynerrs The required changes to the Bison-generated file pars0grm.c will be addressed in a separate commit, which will add a script similar to make_flex.sh. The class ha_innodb is renamed from class ha_innobase by a #define. Thus, there will be no clash with the builtin InnoDB. However, there will be some overhead for invoking virtual methods of class ha_innodb. Ideas for making the vtable hidden are welcome. -fvisibility=hidden is not available in GCC 3.	18 years ago
marko	adbb92becd	branches/zip: Implement a limit for the size of undo log records. innodb-index.test: Add a test with a large number of externally stored columns. Check that there may not be prefix indexes on too many columns. dict_index_too_big_for_undo(): New function: Check if the undo log may overflow. dict_index_add_to_cache(): Return DB_SUCCESS or DB_TOO_BIG_RECORD. Postpone the creation and linking of some data structures, so that when dict_index_too_big_for_undo() holds, it will be easier to clean up. Check the return status in all callers.	18 years ago
sunny	eca3cb2378	branches/zip: Fix a bug where the zipped page and the uncompressed page contents end up with conflicting versions of a record's state. The zipped page record was not being marked as "(un)deleted" because we were not passing the zipped page contents to the (un)delete function, which first (un)delete marks the uncompressed version and then based on whether page_zip is NULL or not (un)delete marks the record in the compressed page.	18 years ago
marko	4e37e2f826	branches/zip: Minor cleanup. buf_page_get_release_on_io(): Removed this unused function. ibuf_build_entry_from_ibuf_rec(): Justify why it is not necessary to add system columns to the dummy table pointed to by the dummy secondary index. page_zip_rec_set_deleted(): Add a page_zip_validate() assertion.	18 years ago
marko	996edd8a91	branches/zip: Do not access the fields of dfield_t directly. dfield_dup(): New function for duplicating the data pointed to by dfield_t. dfield_set_len(), dfield_set_data(): Add Valgrind instrumentation.	18 years ago
marko	cd12974298	branches/zip: Minor cleanup of B-tree cursor operations. btr_pcur_get_rel_pos(): Add a const qualifier. btr_pcur_get_btr_cur(), btr_pcur_get_page_cur(): btr_cur_get_page_cur(): Define as const-preserving macros. btr_pcur_is_on_user_rec(), btr_pcur_is_after_last_on_page(), btr_pcur_is_before_first_on_page(): Remove the unused parameter mtr. Add a const qualifier. btr_pcur_move_to_next_on_page(), btr_pcur_move_to_prev_on_page(): Remove the unused parameter mtr. page_cur_search(): Add const qualifiers. page_cur_get_page(), page_cur_is_before_first(), page_cur_is_after_last(): Add debug assertions.	18 years ago
marko	2561797bb9	branches/zip: On compressed pages, always update the insert buffer bitmap. Do not assume anything about the contents of the bitmap. ibuf_update_free_bits_low(): Use this function only for uncompressed pages. Remove the parameter zip_size. This function avoids latching the bitmap page and updating the bitmap when the bits do not change. ibuf_update_free_bits_zip(): New function based on ibuf_update_free_bits_low(), for use on compressed pages. Remove the parameter max_insert_size that was used for computing the before image of the free bits. Always update the bitmap. ibuf_index_page_calc_free_zip(): New function, factored out from ibuf_index_page_calc_free(). ibuf_update_free_bits_if_full(): Document that this function must only be invoked on uncompressed pages. Add a debug assertion about this.	18 years ago
marko	46468a5a4e	branches/zip: ibuf_insert_low(): Add the parameter entry_size in order to avoid repeated invocation of the costly function rec_get_converted_size().	18 years ago
marko	8835bdf92b	branches/zip: Improve diagnostic output in ibuf_insert_to_index_page() and add some debug assertions.	18 years ago
marko	31d63cd08f	branches/zip: Merge 1664:1783 from trunk.	18 years ago
marko	632ec54002	branches/zip: Make merge sort handle externally stored columns. Some things still fail in innodb-index.test, and there seems to be a race condition (data dictionary lock wait) when running with --valgrind. dfield_t: Add an "external storage" flag, dfield->ext. dfield_is_null(), dfield_is_ext(), dfield_set_ext(), dfield_set_null(): New functions. dfield_copy(), dfield_copy_data(): Add const qualifiers, fix in/out comments. data_write_sql_null(): Use memset(). big_rec_field_t: Replace byte* data with const void* data. ut_ulint_sort(): Remove. upd_field_t: Remove extern_storage. upd_node_t: Replace ext_vec, n_ext_vec with n_ext. row_merge_copy_blobs(): New function. row_ins_index_entry(): Add the parameter "ibool foreign" for suppressing foreign key checks during fast index creation or when inserting into secondary indexes. btr_page_insert_fits(): Add const qualifiers. btr_cur_add_ext(), upd_ext_vec_contains(): Remove. dfield_print_also_hex(), dfield_print(): Replace if...else if with switch. Observe dfield_is_ext().	19 years ago
marko	ab2f114ad9	branches/zip: Merge 1533:1553 from trunk.	19 years ago
marko	8a294fa351	branches/zip: Merge 1493:1533 from trunk. Make use of UT_BITS_IN_BYTES().	19 years ago
marko	382cb9fdf1	branches/zip: Document and obey the rules for modifying the free bits in the insert buffer bitmap. ibuf_set_free_bits_func(): Never disable redo logging. ibuf_update_free_bits_zip(): Remove. btr_page_reorganize_low(), page_zip_reorganize(): Do not update the insert buffer bitmap. Instead, document that callers will have to take care of it, and adapt the callers. btr_compress(): On error, reset the insert buffer free bits. btr_cur_insert_if_possible(): Do not modify the insert buffer bitmap. btr_compress(), btr_cur_optimistic_insert(): On compressed pages, reset the insert buffer bitmap. Document why. btr_cur_update_alloc_zip(): Document why it is necessary and sufficient to reset the insert buffer free bits. btr_cur_update_in_place(), btr_cur_optimistic_update(), btr_cur_pessimistic_update(): Update the free bits in the same mini-transaction. Document that the mini-transaction must be committed before latching any further pages. Verify that this is the case in all execution paths. row_ins_sec_index_entry_by_modify(), row_ins_clust_index_entry_by_modify(), row_undo_mod_clust_low(): Because these functions call btr_cur_update_in_place(), btr_cur_optimistic_update(), or btr_cur_pessimistic_update(), document that the mini-transaction must be committed before latching any further pages. Verify that this is the case in all execution paths.	19 years ago
marko	88df83bb3b	branches/zip: Merge revisions 1402:1493 from trunk. Remove mysql.patch, because the configuration parameter interface has changed.	19 years ago
marko	fa4f5fcfe5	branches/zip: Fix two bugs. mtr_commit(): Do not acquire the log mutex if no log records are to be written. ibuf_set_free_bits_func(): Do not clear mtr.modifications, because that would prevent the dirty insert buffer bitmap block from being moved to the flush list. Instead, assume that mtr_commit() will not acquire log_sys->mutex because redo logging is disabled. buf_page_init_for_read(): I/O-fix the block before buf_buddy_alloc(), because the newly allocated block could otherwise be reused by the compressed page.	19 years ago
marko	4df65a761e	branches/zip: ibuf_set_free_bits_func(): Disable redo logging when crash recovery is in progress. This avoids a hang when btr_parse_page_reorganize(), called from an I/O handler thread, attempts to acquire log_sys->mutex while it is being held by the main thread (the one that runs innobase_init()). This change was committed accidentally. It may be unsafe to clear mtr.modifications, because buf_page_release() at mtr_commit() may forget to put modified pages to the flush list. Cleanup: Remove the "type" parameter from many ibuf functions. Let the caller check that !dict_index_is_clust(). This should avoid function calls and register spilling. ibuf_set_free_bits_func(), ibuf_set_free_bits(): Remove the parameter "type". ibuf_reset_free_bits_with_type(): Rename to ibuf_reset_free_bits(). Remove the parameter "type". ibuf_update_free_bits_if_full(), ibuf_update_free_bits_zip(), ibuf_update_free_bits_low(), ibuf_update_free_bits_for_two_pages_low(): Remove the parameter "index".	19 years ago
marko	d5028fac23	branches/zip: ibuf_set_free_bits(): Rename to ibuf_set_free_bits_func(), and remove the max_val parameter unless #ifdef UNIV_IBUF_DEBUG	19 years ago
marko	d32cd9647b	branches/zip: Note that page_cur_tuple_insert() can fail without attempting to reorganize a compressed page. This mistake was made in r1308.	19 years ago
marko	eddd572abd	branches/zip: Split UNIV_IBUF_COUNT_DEBUG from UNIV_IBUF_DEBUG.	19 years ago
marko	0f792cb278	branches/zip: Many places: Avoid re-reorganizing compressed pages after failing insert. Reorganization will have been attempted in page_cur_tuple_insert() or page_cur_rec_insert(). page_zip_reorganize(): Recompute the insert buffer free bits for leaf pages of secondary indexes. ibuf_data_enough_free_for_insert(): Simplify.	19 years ago
marko	efd79c7a86	branches/zip: Enable the insert buffer on compressed tablespaces. page_zip_max_ins_size(): New function. btr_cur_optimistic_insert(), btr_cur_optimistic_delete(), btr_page_split_and_insert(), btr_compress(): Do not update the ibuf free bits for non-leaf pages or pages belonging to a clustered index. The insert buffer only covers operations on leaf pages of secondary indexes. For pages covered by the insert buffer, limit the max_ins_size to page_zip_max_ins_size(). buf_page_get_gen(): Merge the insert buffer after decompressing the page. buf_page_io_complete(): Relax the assertion about ibuf_count. For compressed-only pages, the insert buffer merge takes place in buf_page_get_gen(). ibuf_index_page_calc_free_bits(), ibuf_index_page_calc_free_from_bits(), ibuf_index_page_calc_free(), ibuf_update_free_bits_if_full(), ibuf_update_free_bits_low(), ibuf_update_free_bits_for_two_pages_low(), ibuf_set_free_bits_low(): Add the parameter zip_size. Limit the maximum insert size to page_zip_max_ins_size().	19 years ago
marko	463aaf28ba	branches/zip: Avoid memory fragmentation when adding column definitions to tables. dict_mem_table_add_col(): Add the parameter "heap" for temporary memory allocation. Allow it and "name" to be NULL. These parameters are NULL when creating dummy indexes. dict_add_col_name(): Remove calls to ut_malloc() and ut_free(). dict_table_get_col_name(): Allow table->col_names to be NULL. dict_table_add_system_columns(), dict_table_add_to_cache(): Add the parameter "heap".	19 years ago
marko	5fc003b65d	branches/zip: Make mutex_own() work with UNIV_DEBUG, without UNIV_SYNC_DEBUG.	19 years ago
marko	8f65d7a661	branches/zip: Merge revisions 1165:1206 from trunk.	19 years ago
marko	f116afa724	branches/zip: Remove the fil_space_get_zip_size() call from buf_page_get_gen(). This saves one mutex operation per block request. buf_page_get_gen(), various macros and functions: Add parameter zip_size. btr_node_ptr_get_child(): Add parameter index. fil_space_get_latch(): Add optional output parameter zip_size. fil_space_get_zip_size(): Return 0 for space id==0, because the system tablespace is never compressed. fsp_header_init(): Remove the parameter zip_size. ibuf_free_excess_pages(): Remove the parameter zip_size. trx_rseg_t, trx_undo_t: Add field zip_size. xdes_lst_get_next(): Remove, unused.	19 years ago
marko	53853ee573	branches/zip: Replace all references to block->space and block->offset, which should have been replaced in r1057.	19 years ago
marko	e803e13754	branches/zip: Make ibuf_validate_low() static and enclose in #ifdef UNIV_DEBUG.	19 years ago
marko	5e45ac48e9	branches/zip: ibuf_merge_or_delete_for_page(): Replace all calls buf_block_get_frame(block) with block->frame in order to avoid debug assertions about block->buf_fix_count == 0.	19 years ago
marko	6ee593b116	branches/zip: Add const qualifiers to read-only pointers. rec_get_nth_field_offs_old(): Split from rec_get_nth_field_old(). rec_get_nth_field_old(): Reimplement as a constness-preserving macro. dict_drop_index_tree(), dict_truncate_index_tree(): Note that rec is an in/out parameter, as the B-tree root page number will be updated.	19 years ago
marko	1834af7672	branches/zip: Minor cleanup. page_set_max_trx_id(): Replace page_t* parameter with buf_block_t*, to avoid a buf_block_align() call. Replace some occurrences of page_get_page_no() with buf_block_get_page_no(). page_cur_delete_rec(): Replace buf_block_align() with page_cur_get_block().	19 years ago
marko	b6e26f24d7	branches/zip: Remove some more buf_block_align() calls. btr_cur_t: Move page_block to page_cur_t::block. page_cur_get_block(), page_cur_get_page_zip(): New functions. page_cur_position(): Add parameter block. Remove many page_zip parameters, now that there is page_cur_get_page_zip(). Replace some page, page_zip parameters with block. Add some const qualifiers to function parameters and remove casts. PAGE_HEAP_NO_INFIMUM, PAGE_HEAP_NO_SUPREMUM, PAGE_HEAP_NO_USER_LOW: New constants. Replace some cursor code in low-level diagnostic functions with direct management of rec, because buf_block_t::buf_fix_count may be 0 when the functions are called, and debug assertions would fail.	19 years ago
marko	2c7e55dd52	branches/zip: buf_merge_or_delete_for_page(): Correct some debug assertions. We cannot call buf_block_get_space() or buf_block_get_page_no(), because the assertions there can fail.	19 years ago
marko	d6c684798e	branches/zip: Add const qualifiers to function parameters, mostly dtuple_t.	19 years ago
marko	cdb096b609	branches/zip: Remove many fil_space_get_zip_size() calls. ibuf_page(), ibuf_page_low(), ibuf_free_excess_pages(), ibuf_insert(), buf_read_page(), buf_read_ahead_linear(), buf_read_recv_pages(): Add parameter zip_size.	19 years ago
marko	24d0bbf86c	branches/zip: Remove some more buf_block_align() calls. btr_search_build_page_hash_index(), fsp_parse_init_file_page(), ibuf_parse_bitmap_init(): Replace page_t* with buf_block_t. buf_read_ibuf_merge_pages(): Add const qualifiers. Retrieve zip_size. ibuf_merge_or_delete_for_page(): Replace page_t with buf_block_t*. Add parameter zip_size.	19 years ago
marko	fbe4fe6022	branches/zip: Remove the casts introduced in r920 and r925. Replace the accessors returning pointers with macros that preserve const qualifiers. In UNIV_DEBUG builds, retain the accessors and cast away constness there. dfield_get_type(), dfield_get_data(), dtuple_get_nth_field(), dict_table_get_nth_col(), dict_table_get_sys_col(): Implement as macro unless #ifdef UNIV_DEBUG. rec_get_nth_field(): Replace with rec_get_nth_field_offs() that does not do pointer arithmetics. Implement rec_get_nth_field() as a macro.	19 years ago
marko	d583b1803c	branches/zip: Eliminate many buf_block_align() calls. Replace page_t* or page_zip_des_t* parameters in some functions with buf_block_t. buf_frame_get_page_zip(): Disable unless #ifdef UNIV_DEBUG \|\| UNIV_ZIP_DEBUG. btr_cur_t: Add buf_block_t page_block. btr_pcur_get_block(), btr_cur_get_block(), btr_cur_get_page_zip(): New functions. btr_cur_position(): Add the parameter block.	19 years ago
marko	8f07794024	branches/zip: Introduce const qualifiers to many read-only parameters and modify some functions to return const pointers. Add const qualifiers to local variable declarations or casts to remove the const qualifier in those places where write access is needed.	19 years ago
marko	968de5a54e	branches/zip: Eliminate some more buf_block_align() calls. ibuf_update_free_bits_low(), ibuf_update_free_bits_for_two_pages_low(), ibuf_set_free_bits_low(): Replace page_t with buf_block_t.	19 years ago
marko	f4051140c8	branches/zip: Remove more references to buf_block_align(). ibuf_reset_free_bits(): Remove, as there already is a similar function ibuf_reset_free_bits_with_type(). ibuf_reset_free_bits_with_type(), ibuf_set_free_bits(), ibuf_update_free_bits_if_full(), btr_leaf_page_release(), buf_page_make_young(): Replace page_t with buf_block_t. btr_compress(): Replace btr_page_get() with btr_block_get().	19 years ago
marko	5669dc3d2a	branches/zip: Reduce the number of buf_block_align() calls. btr_block_get(): New function to return buf_block_t. btr_page_alloc(), buf_page_get_release_on_io(), buf_page_get_gen(), buf_page_create(), fseg_create(), fseg_create_general(): Return buf_block_t. buf_page_get_known_nowait(): Expect buf_block_t instead of buf_frame_t. buf_frame_get_newest_modification(): Replace with buf_block_get_newest_modification(). buf_page_dbg_add_level(): Replace with buf_block_dbg_add_level(). buf_block_get_zip_size(): New function. buf_block_get_page_zip(): Reintroduce. recv_recover_page(): Replace page, space, page_no with block. ibuf_bitmap_page_init(): Replace page, zip_size with block. ibuf_parse_bitmap_init(): Remove the parameter zip_size. btr_search_drop_page_hash_index(): Replace page with block.	19 years ago
marko	736a8ffb01	branches/zip: Replace buf_frame_get_page_no() and buf_frame_get_space_id() with page_get_page_no() and page_get_space_id(). We want to avoid buf_block_align() calls, and the page_no and space_id are now stamped on the pages early on.	19 years ago
marko	334b002c5c	branches/zip: Add function mtr_memo_contains_page() #ifdef UNIV_DEBUG. Replace ut_ad(mtr_memo_contains(mtr, buf_block_align(ptr), ...)) with ut_ad(mtr_memo_contains_page(mtr, ptr, ...)) in order to reduce the number of buf_block_align() calls.	19 years ago
marko	469a27ac2a	branches/zip: Replace buf_frame_align() with page_align(), page_offset(), ut_align_down(), or ut_align_offset().	19 years ago
marko	d374183221	branches/zip: buf_block_get_page_zip(): Replace with buf_frame_get_page_zip().	19 years ago
marko	174bd7b7ca	branches/zip: dtuple_convert_big_rec(): Do not store anything locally of externally stored columns, and fix bugs introduced in r873. (Bug #22496) btr_page_get_sure_split_rec(), btr_page_insert_fits(), rec_get_converted_size(), rec_convert_dtuple_to_rec(), rec_convert_dtuple_to_rec_old(), rec_convert_dtuple_to_rec_new(): Add parameters ext and n_ext. Flag external fields during the conversion. rec_set_field_extern_bits(), rec_set_field_extern_bits_new(), rec_offs_set_nth_extern(), rec_set_nth_field_extern_bit_old(): Remove. The bits are set by rec_convert_dtuple_to_rec(). page_cur_insert_rec_low(): Remove the parameters ext and n_ext. btr_cur_add_ext(): New utility function for updating and sorting ext[]. Low-level functions now expect the array to be in ascending order for performance reasons. Used in btr_cur_optimistic_insert(), btr_cur_pessimistic_insert(), and btr_cur_pessimistic_update(). btr_cur_optimistic_insert(): Remove some defensive code, because we cannot compute the added parameters of rec_get_converted_size(). btr_push_update_extern_fields(): Sort the array. Require the array to be twice the maximum usage, so that ut_ulint_sort() can be used. dtuple_convert_big_rec(): Allocate new space for the BLOB pointer, to avoid overwriting prefix indexes to the same column. Adapt dtuple_convert_back_big_rec(). row_build_index_entry(): Fetch the columns also for prefix indexes of the clustered index. page_zip_apply_log(), page_zip_decompress_clust(): Allow externally stored fields to lack a locally stored part.	19 years ago

1 2

76 Commits (d609478ae192d5daae9dfa4df61a655a3c3ea362)