mariadb

Tree: 3802ff3f8c

Author	SHA1	Message	Date
marko	3802ff3f8c	branches/zip: page0zip.c: Add more page_zip_fail() diagnostics to some decompression functions. page_zip_apply_log_ext(), page_zip_apply_log(): Call page_zip_fail() with appropriate diagnostics before returning NULL. page_zip_decompress_node_ptrs(), page_zip_decompress_sec(), page_zip_decompress_clust(): When detecting that the zlib stream followed by the modification log overlaps the trailer, do not let an assertion fail, but invoke page_zip_fail() and return FALSE. Corrupt data should never lead into assertion failures in decompression functions.	18 years ago
marko	48eb4c4b67	branches/zip: page0zip.c: Define and use the auxiliary macros ASSERT_ZERO() and ASSERT_ZERO_BLOB() for asserting that certain blocks of memory are filled with zero.	18 years ago
marko	4e37e2f826	branches/zip: Minor cleanup. buf_page_get_release_on_io(): Removed this unused function. ibuf_build_entry_from_ibuf_rec(): Justify why it is not necessary to add system columns to the dummy table pointed to by the dummy secondary index. page_zip_rec_set_deleted(): Add a page_zip_validate() assertion.	18 years ago
marko	7be8ffc2c6	branches/zip: Introduce the function page_zip_fail_func() and the wrapper macro page_zip_fail() for displaying error messages. When the error output is enabled (at compile-time), a breakpoint may be set in page_zip_fail_func to easily debug all decompression errors in the context where they occur.	18 years ago
marko	58bea7acaf	branches/zip: Undo r2079 and fix the cause of uninitialized data bytes in page_zip_decompress(). page_zip_decompress_clust(), page_zip_decompress_clust_ext(): Zero-fill the columns DB_TRX_ID and DB_ROLL_PTR on the uncompressed page.	18 years ago
marko	97a03baf41	branches/zip: page_zip_decompress(): Note that the uncompressed page may contain uninitialized bytes when the space of a deleted record is reused by a shorter record.	18 years ago
marko	96c6fe18b1	branches/zip: Improve Valgrind instrumentation. page_zip_get_trailer_len(), page_zip_write_header(): Correct the UNIV_MEM_ASSERT_RW() assertions. page_zip_validate(): Read the validity bits of page, page_zip, and page_zip->data.	18 years ago
marko	cf7d407119	branches/zip: Improve Valgrind instrumentation. page_zip_decompress(): Assert that the uncompressed page is completely defined. page_zip_validate(): Assert that the compressed and uncompressed pages are completely defined. Fetch the "valid" bits, so that they can be examined when run under valgrind --db-attach=yes.	18 years ago
marko	98546caa6f	branches/zip: page_zip_apply_log(): Write the status bits of the record before invoking rec_get_offsets().	18 years ago
marko	4b23d55164	branches/zip: page_zip_compress(): Do not call deflate(&c_stream, Z_FULL_FLUSH) unless more than 6 bytes are available for the output. This should remove the possibility that deflate() returns Z_OK without consuming all input.	18 years ago
marko	63968a86e6	branches/zip: page_zip_compress(): Flag the compressed stream completely initialized, although Valgrind believes that some bits in the 7th or 8th bytes from the end are uninitialized. (They might be, but the decompressor should not care about those bits after encountering the end-of-stream marker in the compressed bit stream.)	18 years ago
marko	d1ef07b308	branches/zip: Check that page_zip->data is defined, not page_zip itself.	18 years ago
marko	589393da83	branches/zip: Improve Valgrind instrumentation. btr_cur_optimistic_insert(): On compressed tablespaces, check that both the compressed and the uncompressed page are completely initialized in the beginning of the function. page_zip_compress(): After successful compression, check that the compressed page is completely initialized.	18 years ago
marko	15d01d3e31	branches/zip: Add some more Valgrind instrumentation. page_zip_write_rec(), page_zip_write_blob_ptr(), page_zip_write_node_ptr(), page_zip_write_trx_id_and_roll_ptr(), page_zip_clear_rec(), page_zip_rec_set_deleted(), page_zip_rec_set_owned(), page_zip_dir_insert(), page_zip_dir_delete(), page_zip_dir_add_slot(), page_zip_reorganize(), page_zip_copy(), page_zip_get_trailer_len(), page_zip_write_header(): Assert that the complete contents of the compressed page is defined.	18 years ago
marko	f8e3c447a3	branches/zip: Improve Valgrind instrumentation. page_zip_compress(): Assert that the contents of the uncompressed page is entirely initialized. page_zip_decompress(): Assert that the contents of the compressed page is entirely initialized. Assert that the uncompressed page is entirely writeable. Flag the uncompressed page uninitialized in the beginning.	18 years ago
marko	6eb43b3d44	branches/zip: Correct the Valgrind instrumentation of r2041. Use UNIV_MEM_ASSERT_RW() instead of UNIV_MEM_VALID(). The latter flags memory defined; the former checks that it is defined.	18 years ago
marko	b013f0d321	branches/zip: page0zip.c: Add some Valgrind instrumentation.	18 years ago
marko	2ddbc912e7	branches/zip: Silence most GCC 4.2.1 warnings about const pointers. For some reason, GCC 4.2.1 ignores casts (for removing constness) in calls to inline functions. page_align(), ut_align_down(): Make the parameter const void*, but still return a non-const pointer. This is ugly, but these functions cannot be replaced with a const-preserving macro in a portable way, given that the pointer argument is not always pointing to bytes. buf_block_get_page_zip(): Implement as a const-preserving macro. buf_frame_get_page_zip(), buf_block_align(): Add const qualifiers. lock_rec_get_prev(): Silence GCC 4.2.1 warnings. mlog_write_initial_log_record(), mlog_write_initial_log_record_fast(), mtr_memo_contains(): Add const qualifier to the pointer. page_header_get_ptr(): Rewrite as page_header_get_offs(), and implement as a macro that calls this function.	18 years ago
marko	8634d343f7	branches/zip: Rename mem_heap_calloc() and mem_calloc() to mem_heap_zalloc() and mem_zalloc(), because calloc() in the C runtime library takes two size parameters, not one. mem_heap_zalloc(): Add debug assertions. Document that the return value is never NULL.	18 years ago
marko	f24fa9e6b0	branches/zip: Define mem_heap_calloc() and mem_calloc(). Use them when allocating zero-filled memory.	19 years ago
marko	42e1de99df	branches/zip: Revert r1523. Passing extra parameters to SORT_FUN and CMP_FUN of UT_SORT_FUNCTION_BODY is best done by defining SORT_FUN and CMP_FUN as macros when needed. The solution of r1523 allows for only one extra parameter.	19 years ago
marko	1b83baaddf	branches/zip: UT_SORT_FUNCTION_BODY(): Add the parameter CTX, which will be needed in row0merge.c for merge sorting the small blocks in main memory. Pass CTX also to SORT_FUN. Adjust all users.	19 years ago
marko	382cb9fdf1	branches/zip: Document and obey the rules for modifying the free bits in the insert buffer bitmap. ibuf_set_free_bits_func(): Never disable redo logging. ibuf_update_free_bits_zip(): Remove. btr_page_reorganize_low(), page_zip_reorganize(): Do not update the insert buffer bitmap. Instead, document that callers will have to take care of it, and adapt the callers. btr_compress(): On error, reset the insert buffer free bits. btr_cur_insert_if_possible(): Do not modify the insert buffer bitmap. btr_compress(), btr_cur_optimistic_insert(): On compressed pages, reset the insert buffer bitmap. Document why. btr_cur_update_alloc_zip(): Document why it is necessary and sufficient to reset the insert buffer free bits. btr_cur_update_in_place(), btr_cur_optimistic_update(), btr_cur_pessimistic_update(): Update the free bits in the same mini-transaction. Document that the mini-transaction must be committed before latching any further pages. Verify that this is the case in all execution paths. row_ins_sec_index_entry_by_modify(), row_ins_clust_index_entry_by_modify(), row_undo_mod_clust_low(): Because these functions call btr_cur_update_in_place(), btr_cur_optimistic_update(), or btr_cur_pessimistic_update(), document that the mini-transaction must be committed before latching any further pages. Verify that this is the case in all execution paths.	19 years ago
marko	4df65a761e	branches/zip: ibuf_set_free_bits_func(): Disable redo logging when crash recovery is in progress. This avoids a hang when btr_parse_page_reorganize(), called from an I/O handler thread, attempts to acquire log_sys->mutex while it is being held by the main thread (the one that runs innobase_init()). This change was committed accidentally. It may be unsafe to clear mtr.modifications, because buf_page_release() at mtr_commit() may forget to put modified pages to the flush list. Cleanup: Remove the "type" parameter from many ibuf functions. Let the caller check that !dict_index_is_clust(). This should avoid function calls and register spilling. ibuf_set_free_bits_func(), ibuf_set_free_bits(): Remove the parameter "type". ibuf_reset_free_bits_with_type(): Rename to ibuf_reset_free_bits(). Remove the parameter "type". ibuf_update_free_bits_if_full(), ibuf_update_free_bits_zip(), ibuf_update_free_bits_low(), ibuf_update_free_bits_for_two_pages_low(): Remove the parameter "index".	19 years ago
marko	35db9631fd	branches/zip: Add statistics on page compression and decompression counts. ha_innodb.cc: Add the columns COMPRESSED, COMPRESSED_OK, DECOMPRESSED to INFORMATION_SCHEMA.INNODB_BUDDY. page_zip_compress_count[], page_zip_compress_ok[]: New statistic counters, incremented in page_zip_compress(). page_zip_decompress_count[]: New statistic counter, incremented in page_zip_decompress().	19 years ago
marko	5f44e6b308	branches/zip: ibuf_update_free_bits_if_full(): Remove the zip_size parameter. Add ibuf_update_free_bits_zip() for compressed pages.	19 years ago
marko	0f792cb278	branches/zip: Many places: Avoid re-reorganizing compressed pages after failing insert. Reorganization will have been attempted in page_cur_tuple_insert() or page_cur_rec_insert(). page_zip_reorganize(): Recompute the insert buffer free bits for leaf pages of secondary indexes. ibuf_data_enough_free_for_insert(): Simplify.	19 years ago
marko	ea9a89da09	branches/zip: page_zip_get_trailer_len(), page_zip_available(): Replace the parameter "dict_index_t* index" with "ibool is_clust". dict_index_is_clust(): Add __attribute__((pure)).	19 years ago
marko	52f7bb36ca	branches/zip: page_zip_write_rec(): Treat records containing externally stored columns as a special case. page_zip_write_rec_ext(): New function for writing records containing externally stored columns.	19 years ago
marko	8fb1f51344	branches/zip: page_zip_decompress_clust(): Optimize the case when the record does not contain externally stored columns. page_zip_decompress_clust_ext(): New function for decompressing records containing externally stored columns.	19 years ago
marko	ec06cfdb00	branches/zip: Introduce separate functions for handling records containing externally stored columns. page_zip_compress_clust_ext(), page_zip_apply_log_ext(): New functions. page_zip_compress_clust(), page_zip_apply_log(): Check rec_offs_any_extern() and avoid invoking the costly loop in most cases. Similar optimizations can be made in page_zip_decompress_clust() and page_zip_write_rec().	19 years ago
marko	8ef3075630	branches/zip: Speed up rec_offs_any_extern() and rec_offs_n_extern() by adding the REC_OFFS_EXTERNAL flag to rec_offs_base(offsets)[0]. This reduces the processor usage of page_zip_write_rec() by about 40% in one test case. The code could be sped up further by testing rec_offs_any_extern() outside of loops that check rec_offs_nth_extern(). The vast majority of records does not contain any externally stored columns.	19 years ago
marko	fad9e98767	branches/zip: Split the function page_cur_insert_rec_low(). page_cur_insert_rec_zip_reorg(): New function: Recompress or reorganize a compressed page. page_cur_insert_rec_zip(): New function: insert a record to a compressed page. page_cur_insert_rec_low(): Only handle inserts to uncompressed pages.	19 years ago
marko	463aaf28ba	branches/zip: Avoid memory fragmentation when adding column definitions to tables. dict_mem_table_add_col(): Add the parameter "heap" for temporary memory allocation. Allow it and "name" to be NULL. These parameters are NULL when creating dummy indexes. dict_add_col_name(): Remove calls to ut_malloc() and ut_free(). dict_table_get_col_name(): Allow table->col_names to be NULL. dict_table_add_system_columns(), dict_table_add_to_cache(): Add the parameter "heap".	19 years ago
marko	6e2c59d421	branches/zip: Make zlib use InnoDB memory heaps when processing BLOBs. page_zip_set_alloc(): New function, to configure zlib to use a memory heap. btr_store_big_rec_extern_fields(): Reduce memLevel, so that a heap of 256 kilobytes will suffice. Use page_zip_set_alloc(). btr_copy_externally_stored_field_prefix_low(): Use page_zip_set_alloc(). page_zip_compress(), page_zip_decompress(): Use page_zip_set_alloc().	19 years ago
marko	e77a82a054	branches/zip: buf_flush_write_block_low(): Avoid recomputing the compressed page checksum of compressed-only blocks. Pass the compressed page frame to fil_io() when needed. page_zip_calc_checksum(): Skip also FIL_PAGE_LSN and FIL_PAGE_FILE_FLUSH_LSN. buf_flush_init_for_writing(): Expect page to be non-NULL.	19 years ago
marko	1f7bc8e232	branches/zip: Remove a race condition. buf_page_init_for_read(): Protect the buf_zip_decompress() call with an X-lock on the block. page_zip_validate(): Move the page_is_comp() assertion after the page header comparison.	19 years ago
marko	69afb619cc	branches/zip: buf_LRU_free_block(): Correct the logic for releasing dirty uncompressed blocks. Move the page_zip_validate() call to a more appropriate place, buf_LRU_block_remove_hashed_page(). buf_LRU_block_remove_hashed_page(): Allow bpage->oldest_modification != 0 for uncompressed pages if there is a compressed page that will be preserved. Release the mutexes before calling buf_print() and buf_LRU_print(). page_zip_validate(): Remove the assertion containing the buf_frame_get_page_zip() call. We might already be holding buf_pool->mutex.	19 years ago
marko	9d9714b8f3	branches/zip: page_zip_fields_decode(): Do not dereference a null pointer.	19 years ago
marko	e5c3aa1224	branches/zip: Make zlib use InnoDB memory heaps. page_zip_malloc(): Replace ut_malloc(...) with mem_heap_alloc(opaque, ...). page_zip_free(): Dummy function (cannot deallocate from InnoDB memory heaps, and zlib seems to defer deallocation to the very end anyway). page_zip_compress(): Create a big enough heap for the needs of zlib. Replace deflateInit() with deflateInit2(). page_zip_decompress(): Create a big enough heap for the needs of zlib. Replace inflateInit() with inflateInit2().	19 years ago
marko	ba1fb95598	branches/zip: Try to adhere to ISO 9899:1989 where possible. (The recently introduced ib_longlong and %ll length specifier are not C89.) row_ext_t: Change char* buf to byte* buf to avoid signedness warnings. Change the type of all bit fields to unsigned. rec_get_next_ptr(): Add (byte) casts to avoid arithmetics on void. page_zip_decompress_clust(): Do not mix declarations and code.	19 years ago
marko	7b32620f7d	branches/zip: page_zip_des_t: Enclose m_start in #ifdef UNIV_DEBUG. Add m_nonempty for facilitating the test in page_zip_alloc(). This reduces the combined size of the bit-fields to 32 bits. Thus, sizeof(page_zip_des_t) == 2 machine words on 32-bit and wider systems.	19 years ago
marko	0d7111c9cd	branches/zip: page_zip_des_t: Rename "size" to "ssize" and reduce the storage size from 16 to 3 bits. page_zip_get_size(), page_zip_set_size(): New functions. Replace direct references to page_zip_des_t:size with calls to buf_block_get_zip_size(), page_zip_get_size(), and page_zip_set_size().	19 years ago
marko	12b589c357	page_zip_validate(): Replace %lu with %d, because the bit-fields introduced in r1057 are of the int type.	19 years ago
marko	8332c45ce4	branches/zip: Introduce buf_page_t, a common structure for compressed-only and uncompressed buffer pool pages. buf_block_t: Replace page_zip, space, and offset with buf_page_t page. Replace some integers with bit-fields. enum buf_block_state: Rename to buf_page_state. Add BUF_BLOCK_ZIP_PAGE. page_zip_des_t: Add the field "state". Make the integer fields bit-fields. page_zip_copy(): Document which fields are copied.	19 years ago
marko	438cb8a6c1	branches/zip: Remove all buf_block_align() calls from lock0lock.c. Replace all page_t* parameters with buf_block_t, and replace many rec_t parameters with heap_no. This eliminates also many rec_get_heap_no() calls, which became more expensive with the introduction of ROW_FORMAT=COMPACT in MySQL/InnoDB 5.0.3. page_rec_get_heap_no(), lock_get_min_heap_no(): New functions.	19 years ago
marko	22015f3949	branches/zip: Minor cleanup. buf_calc_page_new_checksum(), buf_calc_page_old_checksum(), buf_page_is_corrupted(), buf_page_print(): Add const qualifier to byte* parameter. page_parse_create_zip(): Remove this declaration of non-existing function. page_rec_check(), page_rec_print(), row_vers_impl_x_locked_off_kernel(): Add const qualifier to rec_t* parameter. row_vers_old_has_index_entry(): Add const qualifier to dtuple_t* parameter. row_sel_sec_rec_is_for_blob(): Add const qualifiers. Replace some occurrences of mtr_memo_contains_page() with mtr_memo_contains().	19 years ago
marko	1834af7672	branches/zip: Minor cleanup. page_set_max_trx_id(): Replace page_t* parameter with buf_block_t*, to avoid a buf_block_align() call. Replace some occurrences of page_get_page_no() with buf_block_get_page_no(). page_cur_delete_rec(): Replace buf_block_align() with page_cur_get_block().	19 years ago
marko	b6e26f24d7	branches/zip: Remove some more buf_block_align() calls. btr_cur_t: Move page_block to page_cur_t::block. page_cur_get_block(), page_cur_get_page_zip(): New functions. page_cur_position(): Add parameter block. Remove many page_zip parameters, now that there is page_cur_get_page_zip(). Replace some page, page_zip parameters with block. Add some const qualifiers to function parameters and remove casts. PAGE_HEAP_NO_INFIMUM, PAGE_HEAP_NO_SUPREMUM, PAGE_HEAP_NO_USER_LOW: New constants. Replace some cursor code in low-level diagnostic functions with direct management of rec, because buf_block_t::buf_fix_count may be 0 when the functions are called, and debug assertions would fail.	19 years ago
marko	8919884f45	branches/zip: Remove some more casts. dict_index_get_nth_col_pos(), dict_index_get_sys_col_pos(), dict_index_get_n_fields(), fil_page_get_type(): Add const qualifier to parameter. dict_index_get_nth_field(): Implement as a macro unless #ifdef UNIV_DEBUG.	19 years ago

1 2 3 4

159 Commits (3802ff3f8cf0c4337da8930faabf2729cdddd1ea)