mariadb

Tree: e906213f1e

Author	SHA1	Message	Date
marko	e906213f1e	branches/zip: buf_LRU_free_block(): Clarify the function comment.	17 years ago
marko	c5f2c51d3b	branches/zip: Merge revisions 2722:2744 from branches/5.1: ------------------------------------------------------------------------ r2742 \| inaam \| 2008-10-08 22:02:15 +0300 (Wed, 08 Oct 2008) \| 11 lines branches/5.1: Fix Bug#39939 DROP TABLE/DISCARD TABLESPACE takes long time in buf_LRU_invalidate_tablespace() Improve implementation of buf_LRU_invalidate_tablespace by attempting hash index drop in batches instead of doing it one by one. Reviewed by: Heikki, Sunny, Marko Approved by: Heikki ------------------------------------------------------------------------ r2744 \| marko \| 2008-10-09 10:53:09 +0300 (Thu, 09 Oct 2008) \| 5 lines branches/5.1: ha_innobase::delete_all_rows(): In response to a user asking <http://forums.innodb.com/read.php?4,215,215> why DELETE FROM t is not mapped to TRUNCATE TABLE t as it is in MyISAM, note in a comment that DELETE is transactional while TRUNCATE is not. ------------------------------------------------------------------------	17 years ago
marko	1c8162be09	branches/zip: Apply some white-space changes from branches/innodb+.	17 years ago
marko	aab31a41c8	branches/zip: buf_LRU_get_free_block(): Add missing spaces to and correct the grammar of the warning message.	17 years ago
marko	3d3bd9877e	branches/zip: Introduce UNIV_LRU_DEBUG for debugging the LRU buffer pool cache, especially buf_pool->LRU_old and bpage->old. buf_LRU_old_adjust_len(), buf_LRU_remove_block(): Check that blocks in buf_pool->LRU_old have the "old" flag set and the blocks preceding buf_pool->LRU_old have the "old" flag clear. buf_LRU_add_block_low(), buf_relocate(): Check that buf_pool->LRU_old is the first block in the LRU list whose "old" flag is set. buf_LRU_free_block(): When replacing a control block in the LRU list with a control block for a compressed page, assert that the "old" flags in the neighboring LRU list entries grow monotonically. buf_page_set_old(): Assert that the "old" flags in the neighboring LRU list entries grow monotonically.	17 years ago
marko	81ec8249ac	branches/zip: buf_page_set_old(): Add the assertion ut_ad(bpage->in_LRU_list). Adjust the callers where necessary: buf_LRU_add_block_to_end_low() and buf_LRU_add_block_low().	17 years ago
marko	c976021ef4	branches/zip: buf_LRU_free_block(): When buf_page_is_old(b), we incremented buf_pool->LRU_old_len. However, we forgot to check if buf_pool->LRU_old happens to point to b's successor in the LRU list. If it does, we must assign buf_pool->LRU_old = b. The following invariants hold: In the LRU list, the "old" flag should grow monotonically, i.e., it is 0 for the first few items and 1 from thereafter. If buf_pool->LRU_old != NULL, it must point to the first item with old=1 in the LRU list, and there must be buf_pool->LRU_old_len old items in the list. This should fix Mantis issue#50 and issue#68.	18 years ago
marko	94a7cee3c2	branches/zip: Improve the diagnostics for tracking down issue#63 and issue#65. buf_flush_init_for_writing(), buf_LRU_block_remove_hashed_page(): Dump the page frame featuring the incorrect FIL_PAGE_TYPE along with the page_zip->data that might contain an earlier version of the page.	18 years ago
marko	07315cdba3	branches/zip: Always display a newline after ut_print_buf(stderr, ...).	18 years ago
marko	76dea7ad1d	branches/zip: Dump the compressed page when fil_page_get_type() returns an incorrect value. This is to track down Mantis issue#63 and issue#65. buf_LRU_block_remove_hashed_page(), buf_flush_init_for_writing(): dump the compressed page before ut_error.	18 years ago
marko	7c119c9580	branches/zip: Improve the LRU algorithm with a separate unzip_LRU list of blocks that contains uncompressed and compressed frames. This patch was designed by Heikki and Inaam, implemented by Inaam, and refined and reviewed by Marko and Sunny. buf_buddy_n_frames, buf_buddy_min_n_frames, buf_buddy_max_n_frames: Remove. buf_page_belongs_to_unzip_LRU(): New predicate: bpage->zip.data && buf_page_get_state(bpage) == BUF_BLOCK_FILE_PAGE. buf_pool_t, buf_block_t: Add the linked list unzip_LRU. A block in the regular LRU list is in unzip_LRU iff buf_page_belongs_to_unzip_LRU() holds. buf_LRU_free_block(): Add a third return value to refine the case "cannot free the block". buf_LRU_search_and_free_block(): Update the documentation to reflect the implementation. buf_LRU_stat_t, buf_LRU_stat_cur, buf_LRU_stat_sum, buf_LRU_stat_arr[]: Statistics for the unzip_LRU algorithm. buf_LRU_stat_update(): New function: Update the statistics. Called once per second by srv_error_monitor_thread(). buf_LRU_validate(): Validate the unzip_LRU list as well. buf_LRU_evict_from_unzip_LRU(): New predicate: Use the unzip_LRU before falling back to the regular LRU? buf_LRU_free_from_unzip_LRU_list(), buf_LRU_free_from_common_LRU_list(): Subfunctions of buf_LRU_search_and_free_block(). buf_LRU_search_and_free_block(): Reimplement. Try to evict an uncompressed page from the unzip_LRU list before falling back to evicting an entire block from the common LRU list. buf_unzip_LRU_remove_block_if_needed(): New function. buf_unzip_LRU_add_block(): New function: Add a block to the unzip_LRU list.	18 years ago
marko	da54bec75b	branches/zip: Minor fixes. buf_LRU_old_adjust_len(): Replace a constant ut_ad() with a preprocessor check. buf_LRU_free_block(): Remove the check for the unlikely case buf_pool->LRU_old == prev_b in order to simplify the function. The check was implemented as part of r2306.	18 years ago
marko	060a95bea6	branches/zip: buf_LRU_free_block(): When freeing the uncompressed page corresponding to a compressed page, do not flag the block as recently used, but maintain the position of the control block on the LRU list.	18 years ago
marko	17127565dd	branches/zip: buf0lru.c: Minor cleanup. Use ut_d() in assignments to bpage->in_LRU_list instead of #ifdef UNIV_DEBUG. buf_LRU_remove_block(): Move an assertion to a more appropriate place.	18 years ago
marko	1d1dc31a06	branches/zip: Introduce UNIV_INTERN, a linkage specifier for InnoDB-global symbols. Use it for all definitions of non-static variables and functions. lexyy.c, make_flex.sh: Declare yylex as UNIV_INTERN, not static. It is referenced from pars0grm.c. Actually, according to nm .libs/ha_innodb.so\|grep -w '[ABCE-TVXYZ]' the following symbols are still global: * The vtable for class ha_innodb * pars0grm.c: The function yyparse() and the variables yychar, yylval, yynerrs The required changes to the Bison-generated file pars0grm.c will be addressed in a separate commit, which will add a script similar to make_flex.sh. The class ha_innodb is renamed from class ha_innobase by a #define. Thus, there will be no clash with the builtin InnoDB. However, there will be some overhead for invoking virtual methods of class ha_innodb. Ideas for making the vtable hidden are welcome. -fvisibility=hidden is not available in GCC 3.	18 years ago
marko	8958f06278	branches/zip: Introduce the page type code FIL_PAGE_TYPE_ZBLOB2 for continuation pages containing compressed BLOBs. The first compressed BLOB page will be of type FIL_PAGE_TYPE_ZBLOB.	18 years ago
marko	0d02593440	branches/zip: buf_LRU_search_and_free_block(): Do not forbid the release of the buffer pool mutex. Apparently, it is temporarily released also in older versions of MySQL/InnoDB for the duration of the btr_search_drop_page_hash_index() call [in buf_LRU_free_block()].	18 years ago
marko	7ac6a34347	branches/zip: Add instrumentation for prohibiting the release of the buffer pool mutex. The instrumentation can be activated by defining UNIV_DEBUG or UNIV_BUF_DEBUG at compilation time. buf_pool_mutex_exit_forbidden: New variable. When this is nonzero, an assertion will fail in buf_pool_mutex_exit(). buf_pool_mutex_exit_forbid(): Macro for declaring that the buffer pool mutex must not be released. Calls may be nested. buf_pool_mutex_exit_allow(): Macro for declaring that the buffer pool mutex may be released. Calls may be nested. buf_LRU_search_and_free_block(): Prohibit buf_pool_mutex_exit() in the scope of the function. buf_LRU_free_block(): Prohibit buf_pool_mutex_exit() in buf_buddy_alloc() and buf_buddy_free(). buf_LRU_block_remove_hashed_page(): Prohibit buf_pool_mutex_exit() in buf_buddy_free().	18 years ago
marko	cdb5b46445	branches/zip: Implement wrappers for all operations on the buffer pool mutex. buf_pool->mutex: Rename to buf_pool_mutex, so that the wrappers will have to be used when changes are merged from other source trees. buf_pool->zip_mutex: Rename to buf_pool_zip_mutex. buf_pool_mutex_own(), buf_pool_mutex_enter(), buf_pool_mutex_exit(): Wrappers for buf_pool_mutex.	18 years ago
marko	33e082be24	branches/zip: buf_buddy_alloc(): Assign *lru = TRUE whenever the buffer pool mutex is temporarily released. buf_LRU_free_block(), buf_buddy_alloc_clean(): Add an output parameter that will be assigned TRUE when the buffer pool mutex is released. This bug was spotted by and fix provided by Sunny.	18 years ago
marko	fe6e780d82	branches/zip: buf_LRU_block_remove_hashed_page(): Add some Valgrind instrumentation.	18 years ago
marko	902893c719	branches/zip: Improve Valgrind instrumentation. buf_LRU_free_block(): Check that the block descriptor contains valid data. buf_buddy_relocate(): Check that the source block contains valid data. buf_page_get_gen(): Do not dereference bpage after calling buf_relocate(). This avoids a bogus Valgrind warning; the memory itself was valid. buf_page_hash_get(): Check that the returned block descriptor contains valid data.	18 years ago
marko	ddc116e14a	branches/zip: Improve Valgrind instrumentation. UNIV_MEM_ASSERT_RW(): New macro, to check that the contents of a memory area is defined. UNIV_MEM_ASSERT_W(): New macro, to check that a memory area is writable. UNIV_MEM_ASSERT_AND_FREE(): New macro, to check that the memory is writable before declaring it free (unwritable). This replaces UNIV_MEM_FREE() in many places. mem_init_buf(): Declare the memory undefined. mem_erase_buf(): Declare the memory freed.	18 years ago
marko	b4715b5d5e	branches/zip: buf_LRU_block_free_non_file_page(): Flag block->frame allocated before overwriting some fields and flagging the frame freed. Some of the buffer frame may have been flagged freed by the user (such as mem_heap).	19 years ago
marko	df5a9376ea	branches/zip: buf_LRU_search_and_free_block(): Do not try to preserve compressed pages, because doing so would require a call to buf_buddy_alloc(), which in turn can invoke buf_LRU_get_free_block() and buf_LRU_search_and_free_block(), potentially infinite recursion.	19 years ago
marko	9c663a0c46	branches/zip: buf_LRU_search_and_free_block(): Avoid freeing compressed-only pages when they do not occupy too much of the buffer pool. buf0buddy.c, buf0buddy.h: Export buf_buddy_n_frames and buf_buddy_min_n_frames.	19 years ago
marko	6f6aefb553	branches/zip: buf_LRU_get_free_block(): Add an UNIV_UNLIKELY hint around zip_size. zip_size will be 0 most of the time.	19 years ago
marko	75cf6705ac	branches/zip: buf_page_init_for_read(): Defer the costly calls to buf_buddy_alloc() until after the block has been found not to exist in the buffer pool. buf_buddy_alloc(), buf_buddy_alloc_low(): Make lru an output parameter. Improve documentation. buf_page_init_for_read(): Simplify the code. Check buf_page_hash_get() again if buf_buddy_alloc() released buf_pool->mutex.	19 years ago
marko	1f0b739f4f	branches/zip: Do not decompress blocks in buf_page_init_for_read(), but in buf_page_get_gen(). Adjust comments. This prevents unnecessary decompression in read-ahead when the compressed block exists in the buffer pool. If the block does not exist in the buffer pool, read-ahead will still allocate an uncompressed page and decompress the block. Move buf_block_init_low() and buf_zip_decompress() earlier in the file, because some compilers are unable to inline functions that are defined after the invocation. buf_page_get_gen(): Decompress the block if needed. buf_page_init_for_read(): Do not decompress.	19 years ago
marko	ae0ecb11aa	branches/zip: buf_page_t: Add the in_page_hash and in_zip_hash for extra validation in UNIV_DEBUG builds.	19 years ago
marko	f56b6cf4cb	branches/zip: Fix bugs. buf_LRU_free_block(): Do not allocate the control block when a compressed-only page is being freed. buf_validate(): Note that buf_pool->zip_clean may contain blocks with io_fix == BUF_IO_READ.	19 years ago
marko	5fc003b65d	branches/zip: Make mutex_own() work with UNIV_DEBUG, without UNIV_SYNC_DEBUG.	19 years ago
marko	ca662b43e6	branches/zip: buf_LRU_free_block(): Allocate a control block for the compressed page also when freeing an unmodified uncompressed page.	19 years ago
marko	f116afa724	branches/zip: Remove the fil_space_get_zip_size() call from buf_page_get_gen(). This saves one mutex operation per block request. buf_page_get_gen(), various macros and functions: Add parameter zip_size. btr_node_ptr_get_child(): Add parameter index. fil_space_get_latch(): Add optional output parameter zip_size. fil_space_get_zip_size(): Return 0 for space id==0, because the system tablespace is never compressed. fsp_header_init(): Remove the parameter zip_size. ibuf_free_excess_pages(): Remove the parameter zip_size. trx_rseg_t, trx_undo_t: Add field zip_size. xdes_lst_get_next(): Remove, unused.	19 years ago
marko	a85fd29415	branches/zip: Allow dirty compressed-only blocks to exist in the buffer pool and to be flushed to disk. buf_LRU_free_block(): Enable the freeing of uncompressed pages of compressed tablespaces. trx_doublewrite->buf_block_arr[]: Change the type from buf_block_t* to buf_page_t. buf_flush_ready_for_flush(): Add debug assertion. buf_flush_buffered_writes(), buf_flush_try_page(): Support blocks of type BUF_BLOCK_ZIP_DIRTY. buf_flush_post_to_doublewrite_buf(), buf_flush_write_block_low(): Change the type of the parameter from buf_block_t to buf_page_t*. buf_flush_init_for_writing(): Allow page to be NULL if page_zip_ is non-NULL.	19 years ago
marko	94e49a71b2	branches/zip: Note that b (the newly created control block for the compressed page) is not protected by block_mutex. Before releasing buf_pool->mutex, we can simply assign b->io_fix. After reacquiring buf_pool->mutex, we also have to acquire buf_pool->zip_mutex.	19 years ago
marko	9fe82e80e3	branches/zip: buf_LRU_free_block(): When preserving the compressed page of a block, do not release buf_pool->mutex between the time the old control block is removed from buf_pool->page_hash and the new control block is added to it. Prevent operations on the compressed-only block while calling btr_search_drop_page_hash_index() or page_zip_calc_checksum(). buf_LRU_invalidate_tablespace(): Revert the change done in r1223. buf_pool->zip_mutex will have been released by buf_LRU_block_remove_hashed_page() when it returns BUF_BLOCK_ZIP_FREE.	19 years ago
marko	c3599a3ab1	branches/zip: buf_flush_remove(): buf_page_get_mutex(bpage) must have been acquired by the caller. buf_LRU_invalidate_tablespace(): Do not leak buf_pool->zip_mutex when rescanning the LRU list.	19 years ago
marko	91546e7d10	branches/zip: buf_LRU_free_block(): When freeing the uncompressed page of a dirty block, relocate the control blocks on buf_pool->flush_list instead of calling buf_flush_insert_into_flush_list().	19 years ago
marko	82f4ef9596	branches/zip: buf_LRU_block_remove_hashed_page(): When preserving the compressed page, copy the contents from the uncompressed page if necessary.	19 years ago
marko	ac24d86c49	branches/zip: buf_LRU_free_block(): Flag the buffer frame fully initialized when calling btr_search_drop_page_hash_index() and uninitialized after that.	19 years ago
marko	2b82ae8cb4	branches/zip: Improve Valgrind instrumentation of allocated memory. UNIV_MEM_FREE(): Declare a memory area free. UNIV_MEM_ALLOC(): Declare a memory area allocated (but uninitialized). UNIV_MEM_DESC(): Associate a memory area with a control block. UNIV_MEM_UNDESC(): Unassociate a control block. trx_sys_create_doublewrite_buf(): Clear the buffer with memset(). buf_page_init(): Add a bogus UNIV_MEM_VALID(block->frame) to silence valid warnings about InnoDB data pages containing uninitialized data. buf_LRU_get_free_only(): Add UNIV_MEM_ALLOC(block->frame). buf_LRU_get_free_block(): Add UNIV_MEM_DESC(block->page.zip.data, block). buf_LRU_free_block(): Add UNIV_MEM_DESC(b->zip.data, b) when allocating a compressed-only control block for a compressed page. buf_LRU_block_free_non_file_page(): Replace UNIV_MEM_INVALID() with UNIV_MEM_FREE(). buf_LRU_block_remove_hashed_page(): Add UNIV_MEM_UNDESC(bpage) when deallocating a compressed-only control block. Add UNIV_MEM_INVALID(block->frame). (The frame should be flagged free by buf_LRU_block_free_non_file_page() moments later.) buf0buddy.c: Disable some extra checks in Valgrind-instrumented builds. Add UNIV_MEM_VALID(), UNIV_MEM_INVALID(), UNIV_MEM_FREE(), UNIV_MEM_ALLOC() as necessary.	19 years ago
marko	a0c093d93d	branches/zip: buf_LRU_invalidate_tablespace(): After freeing a compressed-only page, restart the scan of the LRU list, because prev_bpage may have been relocated. buf_LRU_block_remove_hashed_page(): Note that other compressed pages or compressed-only blocks may be relocated when freeing a compressed page or a compressed-only block descriptor.	19 years ago
marko	69afb619cc	branches/zip: buf_LRU_free_block(): Correct the logic for releasing dirty uncompressed blocks. Move the page_zip_validate() call to a more appropriate place, buf_LRU_block_remove_hashed_page(). buf_LRU_block_remove_hashed_page(): Allow bpage->oldest_modification != 0 for uncompressed pages if there is a compressed page that will be preserved. Release the mutexes before calling buf_print() and buf_LRU_print(). page_zip_validate(): Remove the assertion containing the buf_frame_get_page_zip() call. We might already be holding buf_pool->mutex.	19 years ago
marko	f54da50c1e	branches/zip: buf_buddy_alloc_clean(): Restart the scan after a successful buf_LRU_free_block(). buf_LRU_free_block(): Note that buf_pool->mutex may be temporarily released.	19 years ago
marko	d3917bb5c3	branches/zip: buf_page_t: Introduce the field in_flush_list #ifdef UNIV_DEBUG. buf_flush_batch(): Remove the local variable "found".	19 years ago
marko	5a094438a5	branches/zip: Maintain a valid checksum for compressed-only pages kept in the buffer pool. buf_LRU_free_block(): When preserving the compressed page, compute the checksum. Also assert page_zip_validate(). buf_zip_decompress(): Add parameter "check" for enabling checksum tests. Before decompressing a compressed-only page from the buffer pool, verify the compressed page checksum.	19 years ago
marko	186549b911	branches/zip: buf_LRU_free_block(): Check if the block has been reloaded to the buffer pool while we were not holding buf_pool->mutex. If it was, do not attempt to preserve the compressed page.	19 years ago
marko	91c87cd343	branches/zip: Allow dirty blocks to be relocated. Allow a transition to BUF_BLOCK_ZIP_DIRTY (discarding the uncompressed page corresponding to a modified compressed page that has not been flushed to disk). buf_page_can_relocate(): New function, modelled after buf_flush_ready_for_replace(). buf_LRU_free_block(): Allow the transition to BUF_BLOCK_ZIP_DIRTY. buf_flush_insert_into_flush_list(): Make the prototype public. buf_buddy_relocate(): Remove an unnecessary switch statement. Use buf_page_can_relocate() instead of buf_flush_ready_for_replace(). buf_page_peek(): Made UNIV_INLINE. Document UNIV_ZIP_DEBUG.	19 years ago
marko	ede8e7bcd0	branches/zip: Fix two bugs in the relocation of block descriptors. Improve diagnostics. buf_buddy_relocate(): Do not assume all small blocks to contain only one buf_page_t object. Require an exact size match, as in the case of compressed pages. buf_relocate(): Relocate buf_pool->LRU_old if needed. Validate the LRU list in debug builds. buf_LRU_block_remove_hashed_page(): Do not overwrite FIL_PAGE_OFFSET or FIL_PAGE_ARCH_LOG_NO_OR_SPACE_ID. The memory will be invalidated in debug builds of buf_buddy_free(). buf0buddy.c: Add assertions for not holding buf_pool->zip_mutex. The calling thread should not be holding any block->mutex, but it might be too time-consuming to assert against that, even #ifdef UNIV_SYNC_DEBUG.	19 years ago

1 2

94 Commits (e906213f1ece65b3bc825a265795027dc32b1ea1)