You can not select more than 25 topics Topics must start with a letter or number, can include dashes ('-') and can be up to 35 characters long.

4409 lines
123 KiB

WL#4738 streamline/simplify @@variable creation process Bug#16565 mysqld --help --verbose does not order variablesBug#20413 sql_slave_skip_counter is not shown in show variables Bug#20415 Output of mysqld --help --verbose is incomplete Bug#25430 variable not found in SELECT @@global.ft_max_word_len; Bug#32902 plugin variables don't know their names Bug#34599 MySQLD Option and Variable Reference need to be consistent in formatting! Bug#34829 No default value for variable and setting default does not raise error Bug#34834 ? Is accepted as a valid sql mode Bug#34878 Few variables have default value according to documentation but error occurs Bug#34883 ft_boolean_syntax cant be assigned from user variable to global var. Bug#37187 `INFORMATION_SCHEMA`.`GLOBAL_VARIABLES`: inconsistent status Bug#40988 log_output_basic.test succeeded though syntactically false. Bug#41010 enum-style command-line options are not honoured (maria.maria-recover fails) Bug#42103 Setting key_buffer_size to a negative value may lead to very large allocations Bug#44691 Some plugins configured as MYSQL_PLUGIN_MANDATORY in can be disabled Bug#44797 plugins w/o command-line options have no disabling option in --help Bug#46314 string system variables don't support expressions Bug#46470 sys_vars.max_binlog_cache_size_basic_32 is broken Bug#46586 When using the plugin interface the type "set" for options caused a crash. Bug#47212 Crash in DBUG_PRINT in mysqltest.cc when trying to print octal number Bug#48758 mysqltest crashes on sys_vars.collation_server_basic in gcov builds Bug#49417 some complaints about mysqld --help --verbose output Bug#49540 DEFAULT value of binlog_format isn't the default value Bug#49640 ambiguous option '--skip-skip-myisam' (double skip prefix) Bug#49644 init_connect and \0 Bug#49645 init_slave and multi-byte characters Bug#49646 mysql --show-warnings crashes when server dies
16 years ago
Backport of revno 2630.28.10, 2630.28.31, 2630.28.26, 2630.33.1, 2630.39.1, 2630.28.29, 2630.34.3, 2630.34.2, 2630.34.1, 2630.29.29, 2630.29.28, 2630.31.1, 2630.28.13, 2630.28.10, 2617.23.14 and some other minor revisions. This patch implements: WL#4264 "Backup: Stabilize Service Interface" -- all the server prerequisites except si_objects.{h,cc} themselves (they can be just copied over, when needed). WL#4435: Support OUT-parameters in prepared statements. (and all issues in the initial patches for these two tasks, that were discovered in pushbuild and during testing). Bug#39519: mysql_stmt_close() should flush all data associated with the statement. After execution of a prepared statement, send OUT parameters of the invoked stored procedure, if any, to the client. When using the binary protocol, send the parameters in an additional result set over the wire. When using the text protocol, assign out parameters to the user variables from the CALL(@var1, @var2, ...) specification. The following refactoring has been made: - Protocol::send_fields() was renamed to Protocol::send_result_set_metadata(); - A new Protocol::send_result_set_row() was introduced to incapsulate common functionality for sending row data. - Signature of Protocol::prepare_for_send() was changed: this operation does not need a list of items, the number of items is fully sufficient. The following backward incompatible changes have been made: - CLIENT_MULTI_RESULTS is now enabled by default in the client; - CLIENT_PS_MULTI_RESUTLS is now enabled by default in the client.
16 years ago
Backport of revno 2630.28.10, 2630.28.31, 2630.28.26, 2630.33.1, 2630.39.1, 2630.28.29, 2630.34.3, 2630.34.2, 2630.34.1, 2630.29.29, 2630.29.28, 2630.31.1, 2630.28.13, 2630.28.10, 2617.23.14 and some other minor revisions. This patch implements: WL#4264 "Backup: Stabilize Service Interface" -- all the server prerequisites except si_objects.{h,cc} themselves (they can be just copied over, when needed). WL#4435: Support OUT-parameters in prepared statements. (and all issues in the initial patches for these two tasks, that were discovered in pushbuild and during testing). Bug#39519: mysql_stmt_close() should flush all data associated with the statement. After execution of a prepared statement, send OUT parameters of the invoked stored procedure, if any, to the client. When using the binary protocol, send the parameters in an additional result set over the wire. When using the text protocol, assign out parameters to the user variables from the CALL(@var1, @var2, ...) specification. The following refactoring has been made: - Protocol::send_fields() was renamed to Protocol::send_result_set_metadata(); - A new Protocol::send_result_set_row() was introduced to incapsulate common functionality for sending row data. - Signature of Protocol::prepare_for_send() was changed: this operation does not need a list of items, the number of items is fully sufficient. The following backward incompatible changes have been made: - CLIENT_MULTI_RESULTS is now enabled by default in the client; - CLIENT_PS_MULTI_RESUTLS is now enabled by default in the client.
16 years ago
A fix for Bug#26750 "valgrind leak in sp_head" (and post-review fixes). The legend: on a replication slave, in case a trigger creation was filtered out because of application of replicate-do-table/ replicate-ignore-table rule, the parsed definition of a trigger was not cleaned up properly. LEX::sphead member was left around and leaked memory. Until the actual implementation of support of replicate-ignore-table rules for triggers by the patch for Bug 24478 it was never the case that "case SQLCOM_CREATE_TRIGGER" was not executed once a trigger was parsed, so the deletion of lex->sphead there worked and the memory did not leak. The fix: The real cause of the bug is that there is no 1 or 2 places where we can clean up the main LEX after parse. And the reason we can not have just one or two places where we clean up the LEX is asymmetric behaviour of MYSQLparse in case of success or error. One of the root causes of this behaviour is the code in Item::Item() constructor. There, a newly created item adds itself to THD::free_list - a single-linked list of Items used in a statement. Yuck. This code is unaware that we may have more than one statement active at a time, and always assumes that the free_list of the current statement is located in THD::free_list. One day we need to be able to explicitly allocate an item in a given Query_arena. Thus, when parsing a definition of a stored procedure, like CREATE PROCEDURE p1() BEGIN SELECT a FROM t1; SELECT b FROM t1; END; we actually need to reset THD::mem_root, THD::free_list and THD::lex to parse the nested procedure statement (SELECT *). The actual reset and restore is implemented in semantic actions attached to sp_proc_stmt grammar rule. The problem is that in case of a parsing error inside a nested statement Bison generated parser would abort immediately, without executing the restore part of the semantic action. This would leave THD in an in-the-middle-of-parsing state. This is why we couldn't have had a single place where we clean up the LEX after MYSQLparse - in case of an error we needed to do a clean up immediately, in case of success a clean up could have been delayed. This left the door open for a memory leak. One of the following possibilities were considered when working on a fix: - patch the replication logic to do the clean up. Rejected as breaks module borders, replication code should not need to know the gory details of clean up procedure after CREATE TRIGGER. - wrap MYSQLparse with a function that would do a clean up. Rejected as ideally we should fix the problem when it happens, not adjust for it outside of the problematic code. - make sure MYSQLparse cleans up after itself by invoking the clean up functionality in the appropriate places before return. Implemented in this patch. - use %destructor rule for sp_proc_stmt to restore THD - cleaner than the prevoius approach, but rejected because needs a careful analysis of the side effects, and this patch is for 5.0, and long term we need to use the next alternative anyway - make sure that sp_proc_stmt doesn't juggle with THD - this is a large work that will affect many modules. Cleanup: move main_lex and main_mem_root from Statement to its only two descendants Prepared_statement and THD. This ensures that when a Statement instance was created for purposes of statement backup, we do not involve LEX constructor/destructor, which is fairly expensive. In order to track that the transformation produces equivalent functionality please check the respective constructors and destructors of Statement, Prepared_statement and THD - these members were used only there. This cleanup is unrelated to the patch.
19 years ago
A fix for Bug#26750 "valgrind leak in sp_head" (and post-review fixes). The legend: on a replication slave, in case a trigger creation was filtered out because of application of replicate-do-table/ replicate-ignore-table rule, the parsed definition of a trigger was not cleaned up properly. LEX::sphead member was left around and leaked memory. Until the actual implementation of support of replicate-ignore-table rules for triggers by the patch for Bug 24478 it was never the case that "case SQLCOM_CREATE_TRIGGER" was not executed once a trigger was parsed, so the deletion of lex->sphead there worked and the memory did not leak. The fix: The real cause of the bug is that there is no 1 or 2 places where we can clean up the main LEX after parse. And the reason we can not have just one or two places where we clean up the LEX is asymmetric behaviour of MYSQLparse in case of success or error. One of the root causes of this behaviour is the code in Item::Item() constructor. There, a newly created item adds itself to THD::free_list - a single-linked list of Items used in a statement. Yuck. This code is unaware that we may have more than one statement active at a time, and always assumes that the free_list of the current statement is located in THD::free_list. One day we need to be able to explicitly allocate an item in a given Query_arena. Thus, when parsing a definition of a stored procedure, like CREATE PROCEDURE p1() BEGIN SELECT a FROM t1; SELECT b FROM t1; END; we actually need to reset THD::mem_root, THD::free_list and THD::lex to parse the nested procedure statement (SELECT *). The actual reset and restore is implemented in semantic actions attached to sp_proc_stmt grammar rule. The problem is that in case of a parsing error inside a nested statement Bison generated parser would abort immediately, without executing the restore part of the semantic action. This would leave THD in an in-the-middle-of-parsing state. This is why we couldn't have had a single place where we clean up the LEX after MYSQLparse - in case of an error we needed to do a clean up immediately, in case of success a clean up could have been delayed. This left the door open for a memory leak. One of the following possibilities were considered when working on a fix: - patch the replication logic to do the clean up. Rejected as breaks module borders, replication code should not need to know the gory details of clean up procedure after CREATE TRIGGER. - wrap MYSQLparse with a function that would do a clean up. Rejected as ideally we should fix the problem when it happens, not adjust for it outside of the problematic code. - make sure MYSQLparse cleans up after itself by invoking the clean up functionality in the appropriate places before return. Implemented in this patch. - use %destructor rule for sp_proc_stmt to restore THD - cleaner than the prevoius approach, but rejected because needs a careful analysis of the side effects, and this patch is for 5.0, and long term we need to use the next alternative anyway - make sure that sp_proc_stmt doesn't juggle with THD - this is a large work that will affect many modules. Cleanup: move main_lex and main_mem_root from Statement to its only two descendants Prepared_statement and THD. This ensures that when a Statement instance was created for purposes of statement backup, we do not involve LEX constructor/destructor, which is fairly expensive. In order to track that the transformation produces equivalent functionality please check the respective constructors and destructors of Statement, Prepared_statement and THD - these members were used only there. This cleanup is unrelated to the patch.
19 years ago
Backport of revno 2630.28.10, 2630.28.31, 2630.28.26, 2630.33.1, 2630.39.1, 2630.28.29, 2630.34.3, 2630.34.2, 2630.34.1, 2630.29.29, 2630.29.28, 2630.31.1, 2630.28.13, 2630.28.10, 2617.23.14 and some other minor revisions. This patch implements: WL#4264 "Backup: Stabilize Service Interface" -- all the server prerequisites except si_objects.{h,cc} themselves (they can be just copied over, when needed). WL#4435: Support OUT-parameters in prepared statements. (and all issues in the initial patches for these two tasks, that were discovered in pushbuild and during testing). Bug#39519: mysql_stmt_close() should flush all data associated with the statement. After execution of a prepared statement, send OUT parameters of the invoked stored procedure, if any, to the client. When using the binary protocol, send the parameters in an additional result set over the wire. When using the text protocol, assign out parameters to the user variables from the CALL(@var1, @var2, ...) specification. The following refactoring has been made: - Protocol::send_fields() was renamed to Protocol::send_result_set_metadata(); - A new Protocol::send_result_set_row() was introduced to incapsulate common functionality for sending row data. - Signature of Protocol::prepare_for_send() was changed: this operation does not need a list of items, the number of items is fully sufficient. The following backward incompatible changes have been made: - CLIENT_MULTI_RESULTS is now enabled by default in the client; - CLIENT_PS_MULTI_RESUTLS is now enabled by default in the client.
16 years ago
A fix for Bug#26750 "valgrind leak in sp_head" (and post-review fixes). The legend: on a replication slave, in case a trigger creation was filtered out because of application of replicate-do-table/ replicate-ignore-table rule, the parsed definition of a trigger was not cleaned up properly. LEX::sphead member was left around and leaked memory. Until the actual implementation of support of replicate-ignore-table rules for triggers by the patch for Bug 24478 it was never the case that "case SQLCOM_CREATE_TRIGGER" was not executed once a trigger was parsed, so the deletion of lex->sphead there worked and the memory did not leak. The fix: The real cause of the bug is that there is no 1 or 2 places where we can clean up the main LEX after parse. And the reason we can not have just one or two places where we clean up the LEX is asymmetric behaviour of MYSQLparse in case of success or error. One of the root causes of this behaviour is the code in Item::Item() constructor. There, a newly created item adds itself to THD::free_list - a single-linked list of Items used in a statement. Yuck. This code is unaware that we may have more than one statement active at a time, and always assumes that the free_list of the current statement is located in THD::free_list. One day we need to be able to explicitly allocate an item in a given Query_arena. Thus, when parsing a definition of a stored procedure, like CREATE PROCEDURE p1() BEGIN SELECT a FROM t1; SELECT b FROM t1; END; we actually need to reset THD::mem_root, THD::free_list and THD::lex to parse the nested procedure statement (SELECT *). The actual reset and restore is implemented in semantic actions attached to sp_proc_stmt grammar rule. The problem is that in case of a parsing error inside a nested statement Bison generated parser would abort immediately, without executing the restore part of the semantic action. This would leave THD in an in-the-middle-of-parsing state. This is why we couldn't have had a single place where we clean up the LEX after MYSQLparse - in case of an error we needed to do a clean up immediately, in case of success a clean up could have been delayed. This left the door open for a memory leak. One of the following possibilities were considered when working on a fix: - patch the replication logic to do the clean up. Rejected as breaks module borders, replication code should not need to know the gory details of clean up procedure after CREATE TRIGGER. - wrap MYSQLparse with a function that would do a clean up. Rejected as ideally we should fix the problem when it happens, not adjust for it outside of the problematic code. - make sure MYSQLparse cleans up after itself by invoking the clean up functionality in the appropriate places before return. Implemented in this patch. - use %destructor rule for sp_proc_stmt to restore THD - cleaner than the prevoius approach, but rejected because needs a careful analysis of the side effects, and this patch is for 5.0, and long term we need to use the next alternative anyway - make sure that sp_proc_stmt doesn't juggle with THD - this is a large work that will affect many modules. Cleanup: move main_lex and main_mem_root from Statement to its only two descendants Prepared_statement and THD. This ensures that when a Statement instance was created for purposes of statement backup, we do not involve LEX constructor/destructor, which is fairly expensive. In order to track that the transformation produces equivalent functionality please check the respective constructors and destructors of Statement, Prepared_statement and THD - these members were used only there. This cleanup is unrelated to the patch.
19 years ago
Backport of revno 2630.28.10, 2630.28.31, 2630.28.26, 2630.33.1, 2630.39.1, 2630.28.29, 2630.34.3, 2630.34.2, 2630.34.1, 2630.29.29, 2630.29.28, 2630.31.1, 2630.28.13, 2630.28.10, 2617.23.14 and some other minor revisions. This patch implements: WL#4264 "Backup: Stabilize Service Interface" -- all the server prerequisites except si_objects.{h,cc} themselves (they can be just copied over, when needed). WL#4435: Support OUT-parameters in prepared statements. (and all issues in the initial patches for these two tasks, that were discovered in pushbuild and during testing). Bug#39519: mysql_stmt_close() should flush all data associated with the statement. After execution of a prepared statement, send OUT parameters of the invoked stored procedure, if any, to the client. When using the binary protocol, send the parameters in an additional result set over the wire. When using the text protocol, assign out parameters to the user variables from the CALL(@var1, @var2, ...) specification. The following refactoring has been made: - Protocol::send_fields() was renamed to Protocol::send_result_set_metadata(); - A new Protocol::send_result_set_row() was introduced to incapsulate common functionality for sending row data. - Signature of Protocol::prepare_for_send() was changed: this operation does not need a list of items, the number of items is fully sufficient. The following backward incompatible changes have been made: - CLIENT_MULTI_RESULTS is now enabled by default in the client; - CLIENT_PS_MULTI_RESUTLS is now enabled by default in the client.
16 years ago
WL#3817: Simplify string / memory area types and make things more consistent (first part) The following type conversions was done: - Changed byte to uchar - Changed gptr to uchar* - Change my_string to char * - Change my_size_t to size_t - Change size_s to size_t Removed declaration of byte, gptr, my_string, my_size_t and size_s. Following function parameter changes was done: - All string functions in mysys/strings was changed to use size_t instead of uint for string lengths. - All read()/write() functions changed to use size_t (including vio). - All protocoll functions changed to use size_t instead of uint - Functions that used a pointer to a string length was changed to use size_t* - Changed malloc(), free() and related functions from using gptr to use void * as this requires fewer casts in the code and is more in line with how the standard functions work. - Added extra length argument to dirname_part() to return the length of the created string. - Changed (at least) following functions to take uchar* as argument: - db_dump() - my_net_write() - net_write_command() - net_store_data() - DBUG_DUMP() - decimal2bin() & bin2decimal() - Changed my_compress() and my_uncompress() to use size_t. Changed one argument to my_uncompress() from a pointer to a value as we only return one value (makes function easier to use). - Changed type of 'pack_data' argument to packfrm() to avoid casts. - Changed in readfrm() and writefrom(), ha_discover and handler::discover() the type for argument 'frmdata' to uchar** to avoid casts. - Changed most Field functions to use uchar* instead of char* (reduced a lot of casts). - Changed field->val_xxx(xxx, new_ptr) to take const pointers. Other changes: - Removed a lot of not needed casts - Added a few new cast required by other changes - Added some cast to my_multi_malloc() arguments for safety (as string lengths needs to be uint, not size_t). - Fixed all calls to hash-get-key functions to use size_t*. (Needed to be done explicitely as this conflict was often hided by casting the function to hash_get_key). - Changed some buffers to memory regions to uchar* to avoid casts. - Changed some string lengths from uint to size_t. - Changed field->ptr to be uchar* instead of char*. This allowed us to get rid of a lot of casts. - Some changes from true -> TRUE, false -> FALSE, unsigned char -> uchar - Include zlib.h in some files as we needed declaration of crc32() - Changed MY_FILE_ERROR to be (size_t) -1. - Changed many variables to hold the result of my_read() / my_write() to be size_t. This was needed to properly detect errors (which are returned as (size_t) -1). - Removed some very old VMS code - Changed packfrm()/unpackfrm() to not be depending on uint size (portability fix) - Removed windows specific code to restore cursor position as this causes slowdown on windows and we should not mix read() and pread() calls anyway as this is not thread safe. Updated function comment to reflect this. Changed function that depended on original behavior of my_pwrite() to itself restore the cursor position (one such case). - Added some missing checking of return value of malloc(). - Changed definition of MOD_PAD_CHAR_TO_FULL_LENGTH to avoid 'long' overflow. - Changed type of table_def::m_size from my_size_t to ulong to reflect that m_size is the number of elements in the array, not a string/memory length. - Moved THD::max_row_length() to table.cc (as it's not depending on THD). Inlined max_row_length_blob() into this function. - More function comments - Fixed some compiler warnings when compiled without partitions. - Removed setting of LEX_STRING() arguments in declaration (portability fix). - Some trivial indentation/variable name changes. - Some trivial code simplifications: - Replaced some calls to alloc_root + memcpy to use strmake_root()/strdup_root(). - Changed some calls from memdup() to strmake() (Safety fix) - Simpler loops in client-simple.c
19 years ago
Backport of revno 2630.28.10, 2630.28.31, 2630.28.26, 2630.33.1, 2630.39.1, 2630.28.29, 2630.34.3, 2630.34.2, 2630.34.1, 2630.29.29, 2630.29.28, 2630.31.1, 2630.28.13, 2630.28.10, 2617.23.14 and some other minor revisions. This patch implements: WL#4264 "Backup: Stabilize Service Interface" -- all the server prerequisites except si_objects.{h,cc} themselves (they can be just copied over, when needed). WL#4435: Support OUT-parameters in prepared statements. (and all issues in the initial patches for these two tasks, that were discovered in pushbuild and during testing). Bug#39519: mysql_stmt_close() should flush all data associated with the statement. After execution of a prepared statement, send OUT parameters of the invoked stored procedure, if any, to the client. When using the binary protocol, send the parameters in an additional result set over the wire. When using the text protocol, assign out parameters to the user variables from the CALL(@var1, @var2, ...) specification. The following refactoring has been made: - Protocol::send_fields() was renamed to Protocol::send_result_set_metadata(); - A new Protocol::send_result_set_row() was introduced to incapsulate common functionality for sending row data. - Signature of Protocol::prepare_for_send() was changed: this operation does not need a list of items, the number of items is fully sufficient. The following backward incompatible changes have been made: - CLIENT_MULTI_RESULTS is now enabled by default in the client; - CLIENT_PS_MULTI_RESUTLS is now enabled by default in the client.
16 years ago
22 years ago
23 years ago
23 years ago
23 years ago
23 years ago
23 years ago
23 years ago
23 years ago
23 years ago
23 years ago
23 years ago
23 years ago
23 years ago
23 years ago
23 years ago
23 years ago
23 years ago
23 years ago
23 years ago
23 years ago
This changeset is largely a handler cleanup changeset (WL#3281), but includes fixes and cleanups that was found necessary while testing the handler changes Changes that requires code changes in other code of other storage engines. (Note that all changes are very straightforward and one should find all issues by compiling a --debug build and fixing all compiler errors and all asserts in field.cc while running the test suite), - New optional handler function introduced: reset() This is called after every DML statement to make it easy for a handler to statement specific cleanups. (The only case it's not called is if force the file to be closed) - handler::extra(HA_EXTRA_RESET) is removed. Code that was there before should be moved to handler::reset() - table->read_set contains a bitmap over all columns that are needed in the query. read_row() and similar functions only needs to read these columns - table->write_set contains a bitmap over all columns that will be updated in the query. write_row() and update_row() only needs to update these columns. The above bitmaps should now be up to date in all context (including ALTER TABLE, filesort()). The handler is informed of any changes to the bitmap after fix_fields() by calling the virtual function handler::column_bitmaps_signal(). If the handler does caching of these bitmaps (instead of using table->read_set, table->write_set), it should redo the caching in this code. as the signal() may be sent several times, it's probably best to set a variable in the signal and redo the caching on read_row() / write_row() if the variable was set. - Removed the read_set and write_set bitmap objects from the handler class - Removed all column bit handling functions from the handler class. (Now one instead uses the normal bitmap functions in my_bitmap.c instead of handler dedicated bitmap functions) - field->query_id is removed. One should instead instead check table->read_set and table->write_set if a field is used in the query. - handler::extra(HA_EXTRA_RETRIVE_ALL_COLS) and handler::extra(HA_EXTRA_RETRIEVE_PRIMARY_KEY) are removed. One should now instead use table->read_set to check for which columns to retrieve. - If a handler needs to call Field->val() or Field->store() on columns that are not used in the query, one should install a temporary all-columns-used map while doing so. For this, we provide the following functions: my_bitmap_map *old_map= dbug_tmp_use_all_columns(table, table->read_set); field->val(); dbug_tmp_restore_column_map(table->read_set, old_map); and similar for the write map: my_bitmap_map *old_map= dbug_tmp_use_all_columns(table, table->write_set); field->val(); dbug_tmp_restore_column_map(table->write_set, old_map); If this is not done, you will sooner or later hit a DBUG_ASSERT in the field store() / val() functions. (For not DBUG binaries, the dbug_tmp_restore_column_map() and dbug_tmp_restore_column_map() are inline dummy functions and should be optimized away be the compiler). - If one needs to temporary set the column map for all binaries (and not just to avoid the DBUG_ASSERT() in the Field::store() / Field::val() methods) one should use the functions tmp_use_all_columns() and tmp_restore_column_map() instead of the above dbug_ variants. - All 'status' fields in the handler base class (like records, data_file_length etc) are now stored in a 'stats' struct. This makes it easier to know what status variables are provided by the base handler. This requires some trivial variable names in the extra() function. - New virtual function handler::records(). This is called to optimize COUNT(*) if (handler::table_flags() & HA_HAS_RECORDS()) is true. (stats.records is not supposed to be an exact value. It's only has to be 'reasonable enough' for the optimizer to be able to choose a good optimization path). - Non virtual handler::init() function added for caching of virtual constants from engine. - Removed has_transactions() virtual method. Now one should instead return HA_NO_TRANSACTIONS in table_flags() if the table handler DOES NOT support transactions. - The 'xxxx_create_handler()' function now has a MEM_ROOT_root argument that is to be used with 'new handler_name()' to allocate the handler in the right area. The xxxx_create_handler() function is also responsible for any initialization of the object before returning. For example, one should change: static handler *myisam_create_handler(TABLE_SHARE *table) { return new ha_myisam(table); } -> static handler *myisam_create_handler(TABLE_SHARE *table, MEM_ROOT *mem_root) { return new (mem_root) ha_myisam(table); } - New optional virtual function: use_hidden_primary_key(). This is called in case of an update/delete when (table_flags() and HA_PRIMARY_KEY_REQUIRED_FOR_DELETE) is defined but we don't have a primary key. This allows the handler to take precisions in remembering any hidden primary key to able to update/delete any found row. The default handler marks all columns to be read. - handler::table_flags() now returns a ulonglong (to allow for more flags). - New/changed table_flags() - HA_HAS_RECORDS Set if ::records() is supported - HA_NO_TRANSACTIONS Set if engine doesn't support transactions - HA_PRIMARY_KEY_REQUIRED_FOR_DELETE Set if we should mark all primary key columns for read when reading rows as part of a DELETE statement. If there is no primary key, all columns are marked for read. - HA_PARTIAL_COLUMN_READ Set if engine will not read all columns in some cases (based on table->read_set) - HA_PRIMARY_KEY_ALLOW_RANDOM_ACCESS Renamed to HA_PRIMARY_KEY_REQUIRED_FOR_POSITION. - HA_DUPP_POS Renamed to HA_DUPLICATE_POS - HA_REQUIRES_KEY_COLUMNS_FOR_DELETE Set this if we should mark ALL key columns for read when when reading rows as part of a DELETE statement. In case of an update we will mark all keys for read for which key part changed value. - HA_STATS_RECORDS_IS_EXACT Set this if stats.records is exact. (This saves us some extra records() calls when optimizing COUNT(*)) - Removed table_flags() - HA_NOT_EXACT_COUNT Now one should instead use HA_HAS_RECORDS if handler::records() gives an exact count() and HA_STATS_RECORDS_IS_EXACT if stats.records is exact. - HA_READ_RND_SAME Removed (no one supported this one) - Removed not needed functions ha_retrieve_all_cols() and ha_retrieve_all_pk() - Renamed handler::dupp_pos to handler::dup_pos - Removed not used variable handler::sortkey Upper level handler changes: - ha_reset() now does some overall checks and calls ::reset() - ha_table_flags() added. This is a cached version of table_flags(). The cache is updated on engine creation time and updated on open. MySQL level changes (not obvious from the above): - DBUG_ASSERT() added to check that column usage matches what is set in the column usage bit maps. (This found a LOT of bugs in current column marking code). - In 5.1 before, all used columns was marked in read_set and only updated columns was marked in write_set. Now we only mark columns for which we need a value in read_set. - Column bitmaps are created in open_binary_frm() and open_table_from_share(). (Before this was in table.cc) - handler::table_flags() calls are replaced with handler::ha_table_flags() - For calling field->val() you must have the corresponding bit set in table->read_set. For calling field->store() you must have the corresponding bit set in table->write_set. (There are asserts in all store()/val() functions to catch wrong usage) - thd->set_query_id is renamed to thd->mark_used_columns and instead of setting this to an integer value, this has now the values: MARK_COLUMNS_NONE, MARK_COLUMNS_READ, MARK_COLUMNS_WRITE Changed also all variables named 'set_query_id' to mark_used_columns. - In filesort() we now inform the handler of exactly which columns are needed doing the sort and choosing the rows. - The TABLE_SHARE object has a 'all_set' column bitmap one can use when one needs a column bitmap with all columns set. (This is used for table->use_all_columns() and other places) - The TABLE object has 3 column bitmaps: - def_read_set Default bitmap for columns to be read - def_write_set Default bitmap for columns to be written - tmp_set Can be used as a temporary bitmap when needed. The table object has also two pointer to bitmaps read_set and write_set that the handler should use to find out which columns are used in which way. - count() optimization now calls handler::records() instead of using handler->stats.records (if (table_flags() & HA_HAS_RECORDS) is true). - Added extra argument to Item::walk() to indicate if we should also traverse sub queries. - Added TABLE parameter to cp_buffer_from_ref() - Don't close tables created with CREATE ... SELECT but keep them in the table cache. (Faster usage of newly created tables). New interfaces: - table->clear_column_bitmaps() to initialize the bitmaps for tables at start of new statements. - table->column_bitmaps_set() to set up new column bitmaps and signal the handler about this. - table->column_bitmaps_set_no_signal() for some few cases where we need to setup new column bitmaps but don't signal the handler (as the handler has already been signaled about these before). Used for the momement only in opt_range.cc when doing ROR scans. - table->use_all_columns() to install a bitmap where all columns are marked as use in the read and the write set. - table->default_column_bitmaps() to install the normal read and write column bitmaps, but not signaling the handler about this. This is mainly used when creating TABLE instances. - table->mark_columns_needed_for_delete(), table->mark_columns_needed_for_delete() and table->mark_columns_needed_for_insert() to allow us to put additional columns in column usage maps if handler so requires. (The handler indicates what it neads in handler->table_flags()) - table->prepare_for_position() to allow us to tell handler that it needs to read primary key parts to be able to store them in future table->position() calls. (This replaces the table->file->ha_retrieve_all_pk function) - table->mark_auto_increment_column() to tell handler are going to update columns part of any auto_increment key. - table->mark_columns_used_by_index() to mark all columns that is part of an index. It will also send extra(HA_EXTRA_KEYREAD) to handler to allow it to quickly know that it only needs to read colums that are part of the key. (The handler can also use the column map for detecting this, but simpler/faster handler can just monitor the extra() call). - table->mark_columns_used_by_index_no_reset() to in addition to other columns, also mark all columns that is used by the given key. - table->restore_column_maps_after_mark_index() to restore to default column maps after a call to table->mark_columns_used_by_index(). - New item function register_field_in_read_map(), for marking used columns in table->read_map. Used by filesort() to mark all used columns - Maintain in TABLE->merge_keys set of all keys that are used in query. (Simplices some optimization loops) - Maintain Field->part_of_key_not_clustered which is like Field->part_of_key but the field in the clustered key is not assumed to be part of all index. (used in opt_range.cc for faster loops) - dbug_tmp_use_all_columns(), dbug_tmp_restore_column_map() tmp_use_all_columns() and tmp_restore_column_map() functions to temporally mark all columns as usable. The 'dbug_' version is primarily intended inside a handler when it wants to just call Field:store() & Field::val() functions, but don't need the column maps set for any other usage. (ie:: bitmap_is_set() is never called) - We can't use compare_records() to skip updates for handlers that returns a partial column set and the read_set doesn't cover all columns in the write set. The reason for this is that if we have a column marked only for write we can't in the MySQL level know if the value changed or not. The reason this worked before was that MySQL marked all to be written columns as also to be read. The new 'optimal' bitmaps exposed this 'hidden bug'. - open_table_from_share() does not anymore setup temporary MEM_ROOT object as a thread specific variable for the handler. Instead we send the to-be-used MEMROOT to get_new_handler(). (Simpler, faster code) Bugs fixed: - Column marking was not done correctly in a lot of cases. (ALTER TABLE, when using triggers, auto_increment fields etc) (Could potentially result in wrong values inserted in table handlers relying on that the old column maps or field->set_query_id was correct) Especially when it comes to triggers, there may be cases where the old code would cause lost/wrong values for NDB and/or InnoDB tables. - Split thd->options flag OPTION_STATUS_NO_TRANS_UPDATE to two flags: OPTION_STATUS_NO_TRANS_UPDATE and OPTION_KEEP_LOG. This allowed me to remove some wrong warnings about: "Some non-transactional changed tables couldn't be rolled back" - Fixed handling of INSERT .. SELECT and CREATE ... SELECT that wrongly reset (thd->options & OPTION_STATUS_NO_TRANS_UPDATE) which caused us to loose some warnings about "Some non-transactional changed tables couldn't be rolled back") - Fixed use of uninitialized memory in ha_ndbcluster.cc::delete_table() which could cause delete_table to report random failures. - Fixed core dumps for some tests when running with --debug - Added missing FN_LIBCHAR in mysql_rm_tmp_tables() (This has probably caused us to not properly remove temporary files after crash) - slow_logs was not properly initialized, which could maybe cause extra/lost entries in slow log. - If we get an duplicate row on insert, change column map to read and write all columns while retrying the operation. This is required by the definition of REPLACE and also ensures that fields that are only part of UPDATE are properly handled. This fixed a bug in NDB and REPLACE where REPLACE wrongly copied some column values from the replaced row. - For table handler that doesn't support NULL in keys, we would give an error when creating a primary key with NULL fields, even after the fields has been automaticly converted to NOT NULL. - Creating a primary key on a SPATIAL key, would fail if field was not declared as NOT NULL. Cleanups: - Removed not used condition argument to setup_tables - Removed not needed item function reset_query_id_processor(). - Field->add_index is removed. Now this is instead maintained in (field->flags & FIELD_IN_ADD_INDEX) - Field->fieldnr is removed (use field->field_index instead) - New argument to filesort() to indicate that it should return a set of row pointers (not used columns). This allowed me to remove some references to sql_command in filesort and should also enable us to return column results in some cases where we couldn't before. - Changed column bitmap handling in opt_range.cc to be aligned with TABLE bitmap, which allowed me to use bitmap functions instead of looping over all fields to create some needed bitmaps. (Faster and smaller code) - Broke up found too long lines - Moved some variable declaration at start of function for better code readability. - Removed some not used arguments from functions. (setup_fields(), mysql_prepare_insert_check_table()) - setup_fields() now takes an enum instead of an int for marking columns usage. - For internal temporary tables, use handler::write_row(), handler::delete_row() and handler::update_row() instead of handler::ha_xxxx() for faster execution. - Changed some constants to enum's and define's. - Using separate column read and write sets allows for easier checking of timestamp field was set by statement. - Remove calls to free_io_cache() as this is now done automaticly in ha_reset() - Don't build table->normalized_path as this is now identical to table->path (after bar's fixes to convert filenames) - Fixed some missed DBUG_PRINT(.."%lx") to use "0x%lx" to make it easier to do comparision with the 'convert-dbug-for-diff' tool. Things left to do in 5.1: - We wrongly log failed CREATE TABLE ... SELECT in some cases when using row based logging (as shown by testcase binlog_row_mix_innodb_myisam.result) Mats has promised to look into this. - Test that my fix for CREATE TABLE ... SELECT is indeed correct. (I added several test cases for this, but in this case it's better that someone else also tests this throughly). Lars has promosed to do this.
20 years ago
23 years ago
23 years ago
23 years ago
22 years ago
22 years ago
22 years ago
Implement new type-of-operation-aware metadata locks. Add a wait-for graph based deadlock detector to the MDL subsystem. Fixes bug #46272 "MySQL 5.4.4, new MDL: unnecessary deadlock" and bug #37346 "innodb does not detect deadlock between update and alter table". The first bug manifested itself as an unwarranted abort of a transaction with ER_LOCK_DEADLOCK error by a concurrent ALTER statement, when this transaction tried to repeat use of a table, which it has already used in a similar fashion before ALTER started. The second bug showed up as a deadlock between table-level locks and InnoDB row locks, which was "detected" only after innodb_lock_wait_timeout timeout. A transaction would start using the table and modify a few rows. Then ALTER TABLE would come in, and start copying rows into a temporary table. Eventually it would stumble on the modified records and get blocked on a row lock. The first transaction would try to do more updates, and get blocked on thr_lock.c lock. This situation of circular wait would only get resolved by a timeout. Both these bugs stemmed from inadequate solutions to the problem of deadlocks occurring between different locking subsystems. In the first case we tried to avoid deadlocks between metadata locking and table-level locking subsystems, when upgrading shared metadata lock to exclusive one. Transactions holding the shared lock on the table and waiting for some table-level lock used to be aborted too aggressively. We also allowed ALTER TABLE to start in presence of transactions that modify the subject table. ALTER TABLE acquires TL_WRITE_ALLOW_READ lock at start, and that block all writes against the table (naturally, we don't want any writes to be lost when switching the old and the new table). TL_WRITE_ALLOW_READ lock, in turn, would block the started transaction on thr_lock.c lock, should they do more updates. This, again, lead to the need to abort such transactions. The second bug occurred simply because we didn't have any mechanism to detect deadlocks between the table-level locks in thr_lock.c and row-level locks in InnoDB, other than innodb_lock_wait_timeout. This patch solves both these problems by moving lock conflicts which are causing these deadlocks into the metadata locking subsystem, thus making it possible to avoid or detect such deadlocks inside MDL. To do this we introduce new type-of-operation-aware metadata locks, which allow MDL subsystem to know not only the fact that transaction has used or is going to use some object but also what kind of operation it has carried out or going to carry out on the object. This, along with the addition of a special kind of upgradable metadata lock, allows ALTER TABLE to wait until all transactions which has updated the table to go away. This solves the second issue. Another special type of upgradable metadata lock is acquired by LOCK TABLE WRITE. This second lock type allows to solve the first issue, since abortion of table-level locks in event of DDL under LOCK TABLES becomes also unnecessary. Below follows the list of incompatible changes introduced by this patch: - From now on, ALTER TABLE and CREATE/DROP TRIGGER SQL (i.e. those statements that acquire TL_WRITE_ALLOW_READ lock) wait for all transactions which has *updated* the table to complete. - From now on, LOCK TABLES ... WRITE, REPAIR/OPTIMIZE TABLE (i.e. all statements which acquire TL_WRITE table-level lock) wait for all transaction which *updated or read* from the table to complete. As a consequence, innodb_table_locks=0 option no longer applies to LOCK TABLES ... WRITE. - DROP DATABASE, DROP TABLE, RENAME TABLE no longer abort statements or transactions which use tables being dropped or renamed, and instead wait for these transactions to complete. - Since LOCK TABLES WRITE now takes a special metadata lock, not compatible with with reads or writes against the subject table and transaction-wide, thr_lock.c deadlock avoidance algorithm that used to ensure absence of deadlocks between LOCK TABLES WRITE and other statements is no longer sufficient, even for MyISAM. The wait-for graph based deadlock detector of MDL subsystem may sometimes be necessary and is involved. This may lead to ER_LOCK_DEADLOCK error produced for multi-statement transactions even if these only use MyISAM: session 1: session 2: begin; update t1 ... lock table t2 write, t1 write; -- gets a lock on t2, blocks on t1 update t2 ... (ER_LOCK_DEADLOCK) - Finally, support of LOW_PRIORITY option for LOCK TABLES ... WRITE was abandoned. LOCK TABLE ... LOW_PRIORITY WRITE from now on has the same priority as the usual LOCK TABLE ... WRITE. SELECT HIGH PRIORITY no longer trumps LOCK TABLE ... WRITE in the wait queue. - We do not take upgradable metadata locks on implicitly locked tables. So if one has, say, a view v1 that uses table t1, and issues: LOCK TABLE v1 WRITE; FLUSH TABLE t1; -- (or just 'FLUSH TABLES'), an error is produced. In order to be able to perform DDL on a table under LOCK TABLES, the table must be locked explicitly in the LOCK TABLES list.
16 years ago
21 years ago
21 years ago
WL#3817: Simplify string / memory area types and make things more consistent (first part) The following type conversions was done: - Changed byte to uchar - Changed gptr to uchar* - Change my_string to char * - Change my_size_t to size_t - Change size_s to size_t Removed declaration of byte, gptr, my_string, my_size_t and size_s. Following function parameter changes was done: - All string functions in mysys/strings was changed to use size_t instead of uint for string lengths. - All read()/write() functions changed to use size_t (including vio). - All protocoll functions changed to use size_t instead of uint - Functions that used a pointer to a string length was changed to use size_t* - Changed malloc(), free() and related functions from using gptr to use void * as this requires fewer casts in the code and is more in line with how the standard functions work. - Added extra length argument to dirname_part() to return the length of the created string. - Changed (at least) following functions to take uchar* as argument: - db_dump() - my_net_write() - net_write_command() - net_store_data() - DBUG_DUMP() - decimal2bin() & bin2decimal() - Changed my_compress() and my_uncompress() to use size_t. Changed one argument to my_uncompress() from a pointer to a value as we only return one value (makes function easier to use). - Changed type of 'pack_data' argument to packfrm() to avoid casts. - Changed in readfrm() and writefrom(), ha_discover and handler::discover() the type for argument 'frmdata' to uchar** to avoid casts. - Changed most Field functions to use uchar* instead of char* (reduced a lot of casts). - Changed field->val_xxx(xxx, new_ptr) to take const pointers. Other changes: - Removed a lot of not needed casts - Added a few new cast required by other changes - Added some cast to my_multi_malloc() arguments for safety (as string lengths needs to be uint, not size_t). - Fixed all calls to hash-get-key functions to use size_t*. (Needed to be done explicitely as this conflict was often hided by casting the function to hash_get_key). - Changed some buffers to memory regions to uchar* to avoid casts. - Changed some string lengths from uint to size_t. - Changed field->ptr to be uchar* instead of char*. This allowed us to get rid of a lot of casts. - Some changes from true -> TRUE, false -> FALSE, unsigned char -> uchar - Include zlib.h in some files as we needed declaration of crc32() - Changed MY_FILE_ERROR to be (size_t) -1. - Changed many variables to hold the result of my_read() / my_write() to be size_t. This was needed to properly detect errors (which are returned as (size_t) -1). - Removed some very old VMS code - Changed packfrm()/unpackfrm() to not be depending on uint size (portability fix) - Removed windows specific code to restore cursor position as this causes slowdown on windows and we should not mix read() and pread() calls anyway as this is not thread safe. Updated function comment to reflect this. Changed function that depended on original behavior of my_pwrite() to itself restore the cursor position (one such case). - Added some missing checking of return value of malloc(). - Changed definition of MOD_PAD_CHAR_TO_FULL_LENGTH to avoid 'long' overflow. - Changed type of table_def::m_size from my_size_t to ulong to reflect that m_size is the number of elements in the array, not a string/memory length. - Moved THD::max_row_length() to table.cc (as it's not depending on THD). Inlined max_row_length_blob() into this function. - More function comments - Fixed some compiler warnings when compiled without partitions. - Removed setting of LEX_STRING() arguments in declaration (portability fix). - Some trivial indentation/variable name changes. - Some trivial code simplifications: - Replaced some calls to alloc_root + memcpy to use strmake_root()/strdup_root(). - Changed some calls from memdup() to strmake() (Safety fix) - Simpler loops in client-simple.c
19 years ago
This changeset is largely a handler cleanup changeset (WL#3281), but includes fixes and cleanups that was found necessary while testing the handler changes Changes that requires code changes in other code of other storage engines. (Note that all changes are very straightforward and one should find all issues by compiling a --debug build and fixing all compiler errors and all asserts in field.cc while running the test suite), - New optional handler function introduced: reset() This is called after every DML statement to make it easy for a handler to statement specific cleanups. (The only case it's not called is if force the file to be closed) - handler::extra(HA_EXTRA_RESET) is removed. Code that was there before should be moved to handler::reset() - table->read_set contains a bitmap over all columns that are needed in the query. read_row() and similar functions only needs to read these columns - table->write_set contains a bitmap over all columns that will be updated in the query. write_row() and update_row() only needs to update these columns. The above bitmaps should now be up to date in all context (including ALTER TABLE, filesort()). The handler is informed of any changes to the bitmap after fix_fields() by calling the virtual function handler::column_bitmaps_signal(). If the handler does caching of these bitmaps (instead of using table->read_set, table->write_set), it should redo the caching in this code. as the signal() may be sent several times, it's probably best to set a variable in the signal and redo the caching on read_row() / write_row() if the variable was set. - Removed the read_set and write_set bitmap objects from the handler class - Removed all column bit handling functions from the handler class. (Now one instead uses the normal bitmap functions in my_bitmap.c instead of handler dedicated bitmap functions) - field->query_id is removed. One should instead instead check table->read_set and table->write_set if a field is used in the query. - handler::extra(HA_EXTRA_RETRIVE_ALL_COLS) and handler::extra(HA_EXTRA_RETRIEVE_PRIMARY_KEY) are removed. One should now instead use table->read_set to check for which columns to retrieve. - If a handler needs to call Field->val() or Field->store() on columns that are not used in the query, one should install a temporary all-columns-used map while doing so. For this, we provide the following functions: my_bitmap_map *old_map= dbug_tmp_use_all_columns(table, table->read_set); field->val(); dbug_tmp_restore_column_map(table->read_set, old_map); and similar for the write map: my_bitmap_map *old_map= dbug_tmp_use_all_columns(table, table->write_set); field->val(); dbug_tmp_restore_column_map(table->write_set, old_map); If this is not done, you will sooner or later hit a DBUG_ASSERT in the field store() / val() functions. (For not DBUG binaries, the dbug_tmp_restore_column_map() and dbug_tmp_restore_column_map() are inline dummy functions and should be optimized away be the compiler). - If one needs to temporary set the column map for all binaries (and not just to avoid the DBUG_ASSERT() in the Field::store() / Field::val() methods) one should use the functions tmp_use_all_columns() and tmp_restore_column_map() instead of the above dbug_ variants. - All 'status' fields in the handler base class (like records, data_file_length etc) are now stored in a 'stats' struct. This makes it easier to know what status variables are provided by the base handler. This requires some trivial variable names in the extra() function. - New virtual function handler::records(). This is called to optimize COUNT(*) if (handler::table_flags() & HA_HAS_RECORDS()) is true. (stats.records is not supposed to be an exact value. It's only has to be 'reasonable enough' for the optimizer to be able to choose a good optimization path). - Non virtual handler::init() function added for caching of virtual constants from engine. - Removed has_transactions() virtual method. Now one should instead return HA_NO_TRANSACTIONS in table_flags() if the table handler DOES NOT support transactions. - The 'xxxx_create_handler()' function now has a MEM_ROOT_root argument that is to be used with 'new handler_name()' to allocate the handler in the right area. The xxxx_create_handler() function is also responsible for any initialization of the object before returning. For example, one should change: static handler *myisam_create_handler(TABLE_SHARE *table) { return new ha_myisam(table); } -> static handler *myisam_create_handler(TABLE_SHARE *table, MEM_ROOT *mem_root) { return new (mem_root) ha_myisam(table); } - New optional virtual function: use_hidden_primary_key(). This is called in case of an update/delete when (table_flags() and HA_PRIMARY_KEY_REQUIRED_FOR_DELETE) is defined but we don't have a primary key. This allows the handler to take precisions in remembering any hidden primary key to able to update/delete any found row. The default handler marks all columns to be read. - handler::table_flags() now returns a ulonglong (to allow for more flags). - New/changed table_flags() - HA_HAS_RECORDS Set if ::records() is supported - HA_NO_TRANSACTIONS Set if engine doesn't support transactions - HA_PRIMARY_KEY_REQUIRED_FOR_DELETE Set if we should mark all primary key columns for read when reading rows as part of a DELETE statement. If there is no primary key, all columns are marked for read. - HA_PARTIAL_COLUMN_READ Set if engine will not read all columns in some cases (based on table->read_set) - HA_PRIMARY_KEY_ALLOW_RANDOM_ACCESS Renamed to HA_PRIMARY_KEY_REQUIRED_FOR_POSITION. - HA_DUPP_POS Renamed to HA_DUPLICATE_POS - HA_REQUIRES_KEY_COLUMNS_FOR_DELETE Set this if we should mark ALL key columns for read when when reading rows as part of a DELETE statement. In case of an update we will mark all keys for read for which key part changed value. - HA_STATS_RECORDS_IS_EXACT Set this if stats.records is exact. (This saves us some extra records() calls when optimizing COUNT(*)) - Removed table_flags() - HA_NOT_EXACT_COUNT Now one should instead use HA_HAS_RECORDS if handler::records() gives an exact count() and HA_STATS_RECORDS_IS_EXACT if stats.records is exact. - HA_READ_RND_SAME Removed (no one supported this one) - Removed not needed functions ha_retrieve_all_cols() and ha_retrieve_all_pk() - Renamed handler::dupp_pos to handler::dup_pos - Removed not used variable handler::sortkey Upper level handler changes: - ha_reset() now does some overall checks and calls ::reset() - ha_table_flags() added. This is a cached version of table_flags(). The cache is updated on engine creation time and updated on open. MySQL level changes (not obvious from the above): - DBUG_ASSERT() added to check that column usage matches what is set in the column usage bit maps. (This found a LOT of bugs in current column marking code). - In 5.1 before, all used columns was marked in read_set and only updated columns was marked in write_set. Now we only mark columns for which we need a value in read_set. - Column bitmaps are created in open_binary_frm() and open_table_from_share(). (Before this was in table.cc) - handler::table_flags() calls are replaced with handler::ha_table_flags() - For calling field->val() you must have the corresponding bit set in table->read_set. For calling field->store() you must have the corresponding bit set in table->write_set. (There are asserts in all store()/val() functions to catch wrong usage) - thd->set_query_id is renamed to thd->mark_used_columns and instead of setting this to an integer value, this has now the values: MARK_COLUMNS_NONE, MARK_COLUMNS_READ, MARK_COLUMNS_WRITE Changed also all variables named 'set_query_id' to mark_used_columns. - In filesort() we now inform the handler of exactly which columns are needed doing the sort and choosing the rows. - The TABLE_SHARE object has a 'all_set' column bitmap one can use when one needs a column bitmap with all columns set. (This is used for table->use_all_columns() and other places) - The TABLE object has 3 column bitmaps: - def_read_set Default bitmap for columns to be read - def_write_set Default bitmap for columns to be written - tmp_set Can be used as a temporary bitmap when needed. The table object has also two pointer to bitmaps read_set and write_set that the handler should use to find out which columns are used in which way. - count() optimization now calls handler::records() instead of using handler->stats.records (if (table_flags() & HA_HAS_RECORDS) is true). - Added extra argument to Item::walk() to indicate if we should also traverse sub queries. - Added TABLE parameter to cp_buffer_from_ref() - Don't close tables created with CREATE ... SELECT but keep them in the table cache. (Faster usage of newly created tables). New interfaces: - table->clear_column_bitmaps() to initialize the bitmaps for tables at start of new statements. - table->column_bitmaps_set() to set up new column bitmaps and signal the handler about this. - table->column_bitmaps_set_no_signal() for some few cases where we need to setup new column bitmaps but don't signal the handler (as the handler has already been signaled about these before). Used for the momement only in opt_range.cc when doing ROR scans. - table->use_all_columns() to install a bitmap where all columns are marked as use in the read and the write set. - table->default_column_bitmaps() to install the normal read and write column bitmaps, but not signaling the handler about this. This is mainly used when creating TABLE instances. - table->mark_columns_needed_for_delete(), table->mark_columns_needed_for_delete() and table->mark_columns_needed_for_insert() to allow us to put additional columns in column usage maps if handler so requires. (The handler indicates what it neads in handler->table_flags()) - table->prepare_for_position() to allow us to tell handler that it needs to read primary key parts to be able to store them in future table->position() calls. (This replaces the table->file->ha_retrieve_all_pk function) - table->mark_auto_increment_column() to tell handler are going to update columns part of any auto_increment key. - table->mark_columns_used_by_index() to mark all columns that is part of an index. It will also send extra(HA_EXTRA_KEYREAD) to handler to allow it to quickly know that it only needs to read colums that are part of the key. (The handler can also use the column map for detecting this, but simpler/faster handler can just monitor the extra() call). - table->mark_columns_used_by_index_no_reset() to in addition to other columns, also mark all columns that is used by the given key. - table->restore_column_maps_after_mark_index() to restore to default column maps after a call to table->mark_columns_used_by_index(). - New item function register_field_in_read_map(), for marking used columns in table->read_map. Used by filesort() to mark all used columns - Maintain in TABLE->merge_keys set of all keys that are used in query. (Simplices some optimization loops) - Maintain Field->part_of_key_not_clustered which is like Field->part_of_key but the field in the clustered key is not assumed to be part of all index. (used in opt_range.cc for faster loops) - dbug_tmp_use_all_columns(), dbug_tmp_restore_column_map() tmp_use_all_columns() and tmp_restore_column_map() functions to temporally mark all columns as usable. The 'dbug_' version is primarily intended inside a handler when it wants to just call Field:store() & Field::val() functions, but don't need the column maps set for any other usage. (ie:: bitmap_is_set() is never called) - We can't use compare_records() to skip updates for handlers that returns a partial column set and the read_set doesn't cover all columns in the write set. The reason for this is that if we have a column marked only for write we can't in the MySQL level know if the value changed or not. The reason this worked before was that MySQL marked all to be written columns as also to be read. The new 'optimal' bitmaps exposed this 'hidden bug'. - open_table_from_share() does not anymore setup temporary MEM_ROOT object as a thread specific variable for the handler. Instead we send the to-be-used MEMROOT to get_new_handler(). (Simpler, faster code) Bugs fixed: - Column marking was not done correctly in a lot of cases. (ALTER TABLE, when using triggers, auto_increment fields etc) (Could potentially result in wrong values inserted in table handlers relying on that the old column maps or field->set_query_id was correct) Especially when it comes to triggers, there may be cases where the old code would cause lost/wrong values for NDB and/or InnoDB tables. - Split thd->options flag OPTION_STATUS_NO_TRANS_UPDATE to two flags: OPTION_STATUS_NO_TRANS_UPDATE and OPTION_KEEP_LOG. This allowed me to remove some wrong warnings about: "Some non-transactional changed tables couldn't be rolled back" - Fixed handling of INSERT .. SELECT and CREATE ... SELECT that wrongly reset (thd->options & OPTION_STATUS_NO_TRANS_UPDATE) which caused us to loose some warnings about "Some non-transactional changed tables couldn't be rolled back") - Fixed use of uninitialized memory in ha_ndbcluster.cc::delete_table() which could cause delete_table to report random failures. - Fixed core dumps for some tests when running with --debug - Added missing FN_LIBCHAR in mysql_rm_tmp_tables() (This has probably caused us to not properly remove temporary files after crash) - slow_logs was not properly initialized, which could maybe cause extra/lost entries in slow log. - If we get an duplicate row on insert, change column map to read and write all columns while retrying the operation. This is required by the definition of REPLACE and also ensures that fields that are only part of UPDATE are properly handled. This fixed a bug in NDB and REPLACE where REPLACE wrongly copied some column values from the replaced row. - For table handler that doesn't support NULL in keys, we would give an error when creating a primary key with NULL fields, even after the fields has been automaticly converted to NOT NULL. - Creating a primary key on a SPATIAL key, would fail if field was not declared as NOT NULL. Cleanups: - Removed not used condition argument to setup_tables - Removed not needed item function reset_query_id_processor(). - Field->add_index is removed. Now this is instead maintained in (field->flags & FIELD_IN_ADD_INDEX) - Field->fieldnr is removed (use field->field_index instead) - New argument to filesort() to indicate that it should return a set of row pointers (not used columns). This allowed me to remove some references to sql_command in filesort and should also enable us to return column results in some cases where we couldn't before. - Changed column bitmap handling in opt_range.cc to be aligned with TABLE bitmap, which allowed me to use bitmap functions instead of looping over all fields to create some needed bitmaps. (Faster and smaller code) - Broke up found too long lines - Moved some variable declaration at start of function for better code readability. - Removed some not used arguments from functions. (setup_fields(), mysql_prepare_insert_check_table()) - setup_fields() now takes an enum instead of an int for marking columns usage. - For internal temporary tables, use handler::write_row(), handler::delete_row() and handler::update_row() instead of handler::ha_xxxx() for faster execution. - Changed some constants to enum's and define's. - Using separate column read and write sets allows for easier checking of timestamp field was set by statement. - Remove calls to free_io_cache() as this is now done automaticly in ha_reset() - Don't build table->normalized_path as this is now identical to table->path (after bar's fixes to convert filenames) - Fixed some missed DBUG_PRINT(.."%lx") to use "0x%lx" to make it easier to do comparision with the 'convert-dbug-for-diff' tool. Things left to do in 5.1: - We wrongly log failed CREATE TABLE ... SELECT in some cases when using row based logging (as shown by testcase binlog_row_mix_innodb_myisam.result) Mats has promised to look into this. - Test that my fix for CREATE TABLE ... SELECT is indeed correct. (I added several test cases for this, but in this case it's better that someone else also tests this throughly). Lars has promosed to do this.
20 years ago
This changeset is largely a handler cleanup changeset (WL#3281), but includes fixes and cleanups that was found necessary while testing the handler changes Changes that requires code changes in other code of other storage engines. (Note that all changes are very straightforward and one should find all issues by compiling a --debug build and fixing all compiler errors and all asserts in field.cc while running the test suite), - New optional handler function introduced: reset() This is called after every DML statement to make it easy for a handler to statement specific cleanups. (The only case it's not called is if force the file to be closed) - handler::extra(HA_EXTRA_RESET) is removed. Code that was there before should be moved to handler::reset() - table->read_set contains a bitmap over all columns that are needed in the query. read_row() and similar functions only needs to read these columns - table->write_set contains a bitmap over all columns that will be updated in the query. write_row() and update_row() only needs to update these columns. The above bitmaps should now be up to date in all context (including ALTER TABLE, filesort()). The handler is informed of any changes to the bitmap after fix_fields() by calling the virtual function handler::column_bitmaps_signal(). If the handler does caching of these bitmaps (instead of using table->read_set, table->write_set), it should redo the caching in this code. as the signal() may be sent several times, it's probably best to set a variable in the signal and redo the caching on read_row() / write_row() if the variable was set. - Removed the read_set and write_set bitmap objects from the handler class - Removed all column bit handling functions from the handler class. (Now one instead uses the normal bitmap functions in my_bitmap.c instead of handler dedicated bitmap functions) - field->query_id is removed. One should instead instead check table->read_set and table->write_set if a field is used in the query. - handler::extra(HA_EXTRA_RETRIVE_ALL_COLS) and handler::extra(HA_EXTRA_RETRIEVE_PRIMARY_KEY) are removed. One should now instead use table->read_set to check for which columns to retrieve. - If a handler needs to call Field->val() or Field->store() on columns that are not used in the query, one should install a temporary all-columns-used map while doing so. For this, we provide the following functions: my_bitmap_map *old_map= dbug_tmp_use_all_columns(table, table->read_set); field->val(); dbug_tmp_restore_column_map(table->read_set, old_map); and similar for the write map: my_bitmap_map *old_map= dbug_tmp_use_all_columns(table, table->write_set); field->val(); dbug_tmp_restore_column_map(table->write_set, old_map); If this is not done, you will sooner or later hit a DBUG_ASSERT in the field store() / val() functions. (For not DBUG binaries, the dbug_tmp_restore_column_map() and dbug_tmp_restore_column_map() are inline dummy functions and should be optimized away be the compiler). - If one needs to temporary set the column map for all binaries (and not just to avoid the DBUG_ASSERT() in the Field::store() / Field::val() methods) one should use the functions tmp_use_all_columns() and tmp_restore_column_map() instead of the above dbug_ variants. - All 'status' fields in the handler base class (like records, data_file_length etc) are now stored in a 'stats' struct. This makes it easier to know what status variables are provided by the base handler. This requires some trivial variable names in the extra() function. - New virtual function handler::records(). This is called to optimize COUNT(*) if (handler::table_flags() & HA_HAS_RECORDS()) is true. (stats.records is not supposed to be an exact value. It's only has to be 'reasonable enough' for the optimizer to be able to choose a good optimization path). - Non virtual handler::init() function added for caching of virtual constants from engine. - Removed has_transactions() virtual method. Now one should instead return HA_NO_TRANSACTIONS in table_flags() if the table handler DOES NOT support transactions. - The 'xxxx_create_handler()' function now has a MEM_ROOT_root argument that is to be used with 'new handler_name()' to allocate the handler in the right area. The xxxx_create_handler() function is also responsible for any initialization of the object before returning. For example, one should change: static handler *myisam_create_handler(TABLE_SHARE *table) { return new ha_myisam(table); } -> static handler *myisam_create_handler(TABLE_SHARE *table, MEM_ROOT *mem_root) { return new (mem_root) ha_myisam(table); } - New optional virtual function: use_hidden_primary_key(). This is called in case of an update/delete when (table_flags() and HA_PRIMARY_KEY_REQUIRED_FOR_DELETE) is defined but we don't have a primary key. This allows the handler to take precisions in remembering any hidden primary key to able to update/delete any found row. The default handler marks all columns to be read. - handler::table_flags() now returns a ulonglong (to allow for more flags). - New/changed table_flags() - HA_HAS_RECORDS Set if ::records() is supported - HA_NO_TRANSACTIONS Set if engine doesn't support transactions - HA_PRIMARY_KEY_REQUIRED_FOR_DELETE Set if we should mark all primary key columns for read when reading rows as part of a DELETE statement. If there is no primary key, all columns are marked for read. - HA_PARTIAL_COLUMN_READ Set if engine will not read all columns in some cases (based on table->read_set) - HA_PRIMARY_KEY_ALLOW_RANDOM_ACCESS Renamed to HA_PRIMARY_KEY_REQUIRED_FOR_POSITION. - HA_DUPP_POS Renamed to HA_DUPLICATE_POS - HA_REQUIRES_KEY_COLUMNS_FOR_DELETE Set this if we should mark ALL key columns for read when when reading rows as part of a DELETE statement. In case of an update we will mark all keys for read for which key part changed value. - HA_STATS_RECORDS_IS_EXACT Set this if stats.records is exact. (This saves us some extra records() calls when optimizing COUNT(*)) - Removed table_flags() - HA_NOT_EXACT_COUNT Now one should instead use HA_HAS_RECORDS if handler::records() gives an exact count() and HA_STATS_RECORDS_IS_EXACT if stats.records is exact. - HA_READ_RND_SAME Removed (no one supported this one) - Removed not needed functions ha_retrieve_all_cols() and ha_retrieve_all_pk() - Renamed handler::dupp_pos to handler::dup_pos - Removed not used variable handler::sortkey Upper level handler changes: - ha_reset() now does some overall checks and calls ::reset() - ha_table_flags() added. This is a cached version of table_flags(). The cache is updated on engine creation time and updated on open. MySQL level changes (not obvious from the above): - DBUG_ASSERT() added to check that column usage matches what is set in the column usage bit maps. (This found a LOT of bugs in current column marking code). - In 5.1 before, all used columns was marked in read_set and only updated columns was marked in write_set. Now we only mark columns for which we need a value in read_set. - Column bitmaps are created in open_binary_frm() and open_table_from_share(). (Before this was in table.cc) - handler::table_flags() calls are replaced with handler::ha_table_flags() - For calling field->val() you must have the corresponding bit set in table->read_set. For calling field->store() you must have the corresponding bit set in table->write_set. (There are asserts in all store()/val() functions to catch wrong usage) - thd->set_query_id is renamed to thd->mark_used_columns and instead of setting this to an integer value, this has now the values: MARK_COLUMNS_NONE, MARK_COLUMNS_READ, MARK_COLUMNS_WRITE Changed also all variables named 'set_query_id' to mark_used_columns. - In filesort() we now inform the handler of exactly which columns are needed doing the sort and choosing the rows. - The TABLE_SHARE object has a 'all_set' column bitmap one can use when one needs a column bitmap with all columns set. (This is used for table->use_all_columns() and other places) - The TABLE object has 3 column bitmaps: - def_read_set Default bitmap for columns to be read - def_write_set Default bitmap for columns to be written - tmp_set Can be used as a temporary bitmap when needed. The table object has also two pointer to bitmaps read_set and write_set that the handler should use to find out which columns are used in which way. - count() optimization now calls handler::records() instead of using handler->stats.records (if (table_flags() & HA_HAS_RECORDS) is true). - Added extra argument to Item::walk() to indicate if we should also traverse sub queries. - Added TABLE parameter to cp_buffer_from_ref() - Don't close tables created with CREATE ... SELECT but keep them in the table cache. (Faster usage of newly created tables). New interfaces: - table->clear_column_bitmaps() to initialize the bitmaps for tables at start of new statements. - table->column_bitmaps_set() to set up new column bitmaps and signal the handler about this. - table->column_bitmaps_set_no_signal() for some few cases where we need to setup new column bitmaps but don't signal the handler (as the handler has already been signaled about these before). Used for the momement only in opt_range.cc when doing ROR scans. - table->use_all_columns() to install a bitmap where all columns are marked as use in the read and the write set. - table->default_column_bitmaps() to install the normal read and write column bitmaps, but not signaling the handler about this. This is mainly used when creating TABLE instances. - table->mark_columns_needed_for_delete(), table->mark_columns_needed_for_delete() and table->mark_columns_needed_for_insert() to allow us to put additional columns in column usage maps if handler so requires. (The handler indicates what it neads in handler->table_flags()) - table->prepare_for_position() to allow us to tell handler that it needs to read primary key parts to be able to store them in future table->position() calls. (This replaces the table->file->ha_retrieve_all_pk function) - table->mark_auto_increment_column() to tell handler are going to update columns part of any auto_increment key. - table->mark_columns_used_by_index() to mark all columns that is part of an index. It will also send extra(HA_EXTRA_KEYREAD) to handler to allow it to quickly know that it only needs to read colums that are part of the key. (The handler can also use the column map for detecting this, but simpler/faster handler can just monitor the extra() call). - table->mark_columns_used_by_index_no_reset() to in addition to other columns, also mark all columns that is used by the given key. - table->restore_column_maps_after_mark_index() to restore to default column maps after a call to table->mark_columns_used_by_index(). - New item function register_field_in_read_map(), for marking used columns in table->read_map. Used by filesort() to mark all used columns - Maintain in TABLE->merge_keys set of all keys that are used in query. (Simplices some optimization loops) - Maintain Field->part_of_key_not_clustered which is like Field->part_of_key but the field in the clustered key is not assumed to be part of all index. (used in opt_range.cc for faster loops) - dbug_tmp_use_all_columns(), dbug_tmp_restore_column_map() tmp_use_all_columns() and tmp_restore_column_map() functions to temporally mark all columns as usable. The 'dbug_' version is primarily intended inside a handler when it wants to just call Field:store() & Field::val() functions, but don't need the column maps set for any other usage. (ie:: bitmap_is_set() is never called) - We can't use compare_records() to skip updates for handlers that returns a partial column set and the read_set doesn't cover all columns in the write set. The reason for this is that if we have a column marked only for write we can't in the MySQL level know if the value changed or not. The reason this worked before was that MySQL marked all to be written columns as also to be read. The new 'optimal' bitmaps exposed this 'hidden bug'. - open_table_from_share() does not anymore setup temporary MEM_ROOT object as a thread specific variable for the handler. Instead we send the to-be-used MEMROOT to get_new_handler(). (Simpler, faster code) Bugs fixed: - Column marking was not done correctly in a lot of cases. (ALTER TABLE, when using triggers, auto_increment fields etc) (Could potentially result in wrong values inserted in table handlers relying on that the old column maps or field->set_query_id was correct) Especially when it comes to triggers, there may be cases where the old code would cause lost/wrong values for NDB and/or InnoDB tables. - Split thd->options flag OPTION_STATUS_NO_TRANS_UPDATE to two flags: OPTION_STATUS_NO_TRANS_UPDATE and OPTION_KEEP_LOG. This allowed me to remove some wrong warnings about: "Some non-transactional changed tables couldn't be rolled back" - Fixed handling of INSERT .. SELECT and CREATE ... SELECT that wrongly reset (thd->options & OPTION_STATUS_NO_TRANS_UPDATE) which caused us to loose some warnings about "Some non-transactional changed tables couldn't be rolled back") - Fixed use of uninitialized memory in ha_ndbcluster.cc::delete_table() which could cause delete_table to report random failures. - Fixed core dumps for some tests when running with --debug - Added missing FN_LIBCHAR in mysql_rm_tmp_tables() (This has probably caused us to not properly remove temporary files after crash) - slow_logs was not properly initialized, which could maybe cause extra/lost entries in slow log. - If we get an duplicate row on insert, change column map to read and write all columns while retrying the operation. This is required by the definition of REPLACE and also ensures that fields that are only part of UPDATE are properly handled. This fixed a bug in NDB and REPLACE where REPLACE wrongly copied some column values from the replaced row. - For table handler that doesn't support NULL in keys, we would give an error when creating a primary key with NULL fields, even after the fields has been automaticly converted to NOT NULL. - Creating a primary key on a SPATIAL key, would fail if field was not declared as NOT NULL. Cleanups: - Removed not used condition argument to setup_tables - Removed not needed item function reset_query_id_processor(). - Field->add_index is removed. Now this is instead maintained in (field->flags & FIELD_IN_ADD_INDEX) - Field->fieldnr is removed (use field->field_index instead) - New argument to filesort() to indicate that it should return a set of row pointers (not used columns). This allowed me to remove some references to sql_command in filesort and should also enable us to return column results in some cases where we couldn't before. - Changed column bitmap handling in opt_range.cc to be aligned with TABLE bitmap, which allowed me to use bitmap functions instead of looping over all fields to create some needed bitmaps. (Faster and smaller code) - Broke up found too long lines - Moved some variable declaration at start of function for better code readability. - Removed some not used arguments from functions. (setup_fields(), mysql_prepare_insert_check_table()) - setup_fields() now takes an enum instead of an int for marking columns usage. - For internal temporary tables, use handler::write_row(), handler::delete_row() and handler::update_row() instead of handler::ha_xxxx() for faster execution. - Changed some constants to enum's and define's. - Using separate column read and write sets allows for easier checking of timestamp field was set by statement. - Remove calls to free_io_cache() as this is now done automaticly in ha_reset() - Don't build table->normalized_path as this is now identical to table->path (after bar's fixes to convert filenames) - Fixed some missed DBUG_PRINT(.."%lx") to use "0x%lx" to make it easier to do comparision with the 'convert-dbug-for-diff' tool. Things left to do in 5.1: - We wrongly log failed CREATE TABLE ... SELECT in some cases when using row based logging (as shown by testcase binlog_row_mix_innodb_myisam.result) Mats has promised to look into this. - Test that my fix for CREATE TABLE ... SELECT is indeed correct. (I added several test cases for this, but in this case it's better that someone else also tests this throughly). Lars has promosed to do this.
20 years ago
21 years ago
Implement new type-of-operation-aware metadata locks. Add a wait-for graph based deadlock detector to the MDL subsystem. Fixes bug #46272 "MySQL 5.4.4, new MDL: unnecessary deadlock" and bug #37346 "innodb does not detect deadlock between update and alter table". The first bug manifested itself as an unwarranted abort of a transaction with ER_LOCK_DEADLOCK error by a concurrent ALTER statement, when this transaction tried to repeat use of a table, which it has already used in a similar fashion before ALTER started. The second bug showed up as a deadlock between table-level locks and InnoDB row locks, which was "detected" only after innodb_lock_wait_timeout timeout. A transaction would start using the table and modify a few rows. Then ALTER TABLE would come in, and start copying rows into a temporary table. Eventually it would stumble on the modified records and get blocked on a row lock. The first transaction would try to do more updates, and get blocked on thr_lock.c lock. This situation of circular wait would only get resolved by a timeout. Both these bugs stemmed from inadequate solutions to the problem of deadlocks occurring between different locking subsystems. In the first case we tried to avoid deadlocks between metadata locking and table-level locking subsystems, when upgrading shared metadata lock to exclusive one. Transactions holding the shared lock on the table and waiting for some table-level lock used to be aborted too aggressively. We also allowed ALTER TABLE to start in presence of transactions that modify the subject table. ALTER TABLE acquires TL_WRITE_ALLOW_READ lock at start, and that block all writes against the table (naturally, we don't want any writes to be lost when switching the old and the new table). TL_WRITE_ALLOW_READ lock, in turn, would block the started transaction on thr_lock.c lock, should they do more updates. This, again, lead to the need to abort such transactions. The second bug occurred simply because we didn't have any mechanism to detect deadlocks between the table-level locks in thr_lock.c and row-level locks in InnoDB, other than innodb_lock_wait_timeout. This patch solves both these problems by moving lock conflicts which are causing these deadlocks into the metadata locking subsystem, thus making it possible to avoid or detect such deadlocks inside MDL. To do this we introduce new type-of-operation-aware metadata locks, which allow MDL subsystem to know not only the fact that transaction has used or is going to use some object but also what kind of operation it has carried out or going to carry out on the object. This, along with the addition of a special kind of upgradable metadata lock, allows ALTER TABLE to wait until all transactions which has updated the table to go away. This solves the second issue. Another special type of upgradable metadata lock is acquired by LOCK TABLE WRITE. This second lock type allows to solve the first issue, since abortion of table-level locks in event of DDL under LOCK TABLES becomes also unnecessary. Below follows the list of incompatible changes introduced by this patch: - From now on, ALTER TABLE and CREATE/DROP TRIGGER SQL (i.e. those statements that acquire TL_WRITE_ALLOW_READ lock) wait for all transactions which has *updated* the table to complete. - From now on, LOCK TABLES ... WRITE, REPAIR/OPTIMIZE TABLE (i.e. all statements which acquire TL_WRITE table-level lock) wait for all transaction which *updated or read* from the table to complete. As a consequence, innodb_table_locks=0 option no longer applies to LOCK TABLES ... WRITE. - DROP DATABASE, DROP TABLE, RENAME TABLE no longer abort statements or transactions which use tables being dropped or renamed, and instead wait for these transactions to complete. - Since LOCK TABLES WRITE now takes a special metadata lock, not compatible with with reads or writes against the subject table and transaction-wide, thr_lock.c deadlock avoidance algorithm that used to ensure absence of deadlocks between LOCK TABLES WRITE and other statements is no longer sufficient, even for MyISAM. The wait-for graph based deadlock detector of MDL subsystem may sometimes be necessary and is involved. This may lead to ER_LOCK_DEADLOCK error produced for multi-statement transactions even if these only use MyISAM: session 1: session 2: begin; update t1 ... lock table t2 write, t1 write; -- gets a lock on t2, blocks on t1 update t2 ... (ER_LOCK_DEADLOCK) - Finally, support of LOW_PRIORITY option for LOCK TABLES ... WRITE was abandoned. LOCK TABLE ... LOW_PRIORITY WRITE from now on has the same priority as the usual LOCK TABLE ... WRITE. SELECT HIGH PRIORITY no longer trumps LOCK TABLE ... WRITE in the wait queue. - We do not take upgradable metadata locks on implicitly locked tables. So if one has, say, a view v1 that uses table t1, and issues: LOCK TABLE v1 WRITE; FLUSH TABLE t1; -- (or just 'FLUSH TABLES'), an error is produced. In order to be able to perform DDL on a table under LOCK TABLES, the table must be locked explicitly in the LOCK TABLES list.
16 years ago
21 years ago
21 years ago
This changeset is largely a handler cleanup changeset (WL#3281), but includes fixes and cleanups that was found necessary while testing the handler changes Changes that requires code changes in other code of other storage engines. (Note that all changes are very straightforward and one should find all issues by compiling a --debug build and fixing all compiler errors and all asserts in field.cc while running the test suite), - New optional handler function introduced: reset() This is called after every DML statement to make it easy for a handler to statement specific cleanups. (The only case it's not called is if force the file to be closed) - handler::extra(HA_EXTRA_RESET) is removed. Code that was there before should be moved to handler::reset() - table->read_set contains a bitmap over all columns that are needed in the query. read_row() and similar functions only needs to read these columns - table->write_set contains a bitmap over all columns that will be updated in the query. write_row() and update_row() only needs to update these columns. The above bitmaps should now be up to date in all context (including ALTER TABLE, filesort()). The handler is informed of any changes to the bitmap after fix_fields() by calling the virtual function handler::column_bitmaps_signal(). If the handler does caching of these bitmaps (instead of using table->read_set, table->write_set), it should redo the caching in this code. as the signal() may be sent several times, it's probably best to set a variable in the signal and redo the caching on read_row() / write_row() if the variable was set. - Removed the read_set and write_set bitmap objects from the handler class - Removed all column bit handling functions from the handler class. (Now one instead uses the normal bitmap functions in my_bitmap.c instead of handler dedicated bitmap functions) - field->query_id is removed. One should instead instead check table->read_set and table->write_set if a field is used in the query. - handler::extra(HA_EXTRA_RETRIVE_ALL_COLS) and handler::extra(HA_EXTRA_RETRIEVE_PRIMARY_KEY) are removed. One should now instead use table->read_set to check for which columns to retrieve. - If a handler needs to call Field->val() or Field->store() on columns that are not used in the query, one should install a temporary all-columns-used map while doing so. For this, we provide the following functions: my_bitmap_map *old_map= dbug_tmp_use_all_columns(table, table->read_set); field->val(); dbug_tmp_restore_column_map(table->read_set, old_map); and similar for the write map: my_bitmap_map *old_map= dbug_tmp_use_all_columns(table, table->write_set); field->val(); dbug_tmp_restore_column_map(table->write_set, old_map); If this is not done, you will sooner or later hit a DBUG_ASSERT in the field store() / val() functions. (For not DBUG binaries, the dbug_tmp_restore_column_map() and dbug_tmp_restore_column_map() are inline dummy functions and should be optimized away be the compiler). - If one needs to temporary set the column map for all binaries (and not just to avoid the DBUG_ASSERT() in the Field::store() / Field::val() methods) one should use the functions tmp_use_all_columns() and tmp_restore_column_map() instead of the above dbug_ variants. - All 'status' fields in the handler base class (like records, data_file_length etc) are now stored in a 'stats' struct. This makes it easier to know what status variables are provided by the base handler. This requires some trivial variable names in the extra() function. - New virtual function handler::records(). This is called to optimize COUNT(*) if (handler::table_flags() & HA_HAS_RECORDS()) is true. (stats.records is not supposed to be an exact value. It's only has to be 'reasonable enough' for the optimizer to be able to choose a good optimization path). - Non virtual handler::init() function added for caching of virtual constants from engine. - Removed has_transactions() virtual method. Now one should instead return HA_NO_TRANSACTIONS in table_flags() if the table handler DOES NOT support transactions. - The 'xxxx_create_handler()' function now has a MEM_ROOT_root argument that is to be used with 'new handler_name()' to allocate the handler in the right area. The xxxx_create_handler() function is also responsible for any initialization of the object before returning. For example, one should change: static handler *myisam_create_handler(TABLE_SHARE *table) { return new ha_myisam(table); } -> static handler *myisam_create_handler(TABLE_SHARE *table, MEM_ROOT *mem_root) { return new (mem_root) ha_myisam(table); } - New optional virtual function: use_hidden_primary_key(). This is called in case of an update/delete when (table_flags() and HA_PRIMARY_KEY_REQUIRED_FOR_DELETE) is defined but we don't have a primary key. This allows the handler to take precisions in remembering any hidden primary key to able to update/delete any found row. The default handler marks all columns to be read. - handler::table_flags() now returns a ulonglong (to allow for more flags). - New/changed table_flags() - HA_HAS_RECORDS Set if ::records() is supported - HA_NO_TRANSACTIONS Set if engine doesn't support transactions - HA_PRIMARY_KEY_REQUIRED_FOR_DELETE Set if we should mark all primary key columns for read when reading rows as part of a DELETE statement. If there is no primary key, all columns are marked for read. - HA_PARTIAL_COLUMN_READ Set if engine will not read all columns in some cases (based on table->read_set) - HA_PRIMARY_KEY_ALLOW_RANDOM_ACCESS Renamed to HA_PRIMARY_KEY_REQUIRED_FOR_POSITION. - HA_DUPP_POS Renamed to HA_DUPLICATE_POS - HA_REQUIRES_KEY_COLUMNS_FOR_DELETE Set this if we should mark ALL key columns for read when when reading rows as part of a DELETE statement. In case of an update we will mark all keys for read for which key part changed value. - HA_STATS_RECORDS_IS_EXACT Set this if stats.records is exact. (This saves us some extra records() calls when optimizing COUNT(*)) - Removed table_flags() - HA_NOT_EXACT_COUNT Now one should instead use HA_HAS_RECORDS if handler::records() gives an exact count() and HA_STATS_RECORDS_IS_EXACT if stats.records is exact. - HA_READ_RND_SAME Removed (no one supported this one) - Removed not needed functions ha_retrieve_all_cols() and ha_retrieve_all_pk() - Renamed handler::dupp_pos to handler::dup_pos - Removed not used variable handler::sortkey Upper level handler changes: - ha_reset() now does some overall checks and calls ::reset() - ha_table_flags() added. This is a cached version of table_flags(). The cache is updated on engine creation time and updated on open. MySQL level changes (not obvious from the above): - DBUG_ASSERT() added to check that column usage matches what is set in the column usage bit maps. (This found a LOT of bugs in current column marking code). - In 5.1 before, all used columns was marked in read_set and only updated columns was marked in write_set. Now we only mark columns for which we need a value in read_set. - Column bitmaps are created in open_binary_frm() and open_table_from_share(). (Before this was in table.cc) - handler::table_flags() calls are replaced with handler::ha_table_flags() - For calling field->val() you must have the corresponding bit set in table->read_set. For calling field->store() you must have the corresponding bit set in table->write_set. (There are asserts in all store()/val() functions to catch wrong usage) - thd->set_query_id is renamed to thd->mark_used_columns and instead of setting this to an integer value, this has now the values: MARK_COLUMNS_NONE, MARK_COLUMNS_READ, MARK_COLUMNS_WRITE Changed also all variables named 'set_query_id' to mark_used_columns. - In filesort() we now inform the handler of exactly which columns are needed doing the sort and choosing the rows. - The TABLE_SHARE object has a 'all_set' column bitmap one can use when one needs a column bitmap with all columns set. (This is used for table->use_all_columns() and other places) - The TABLE object has 3 column bitmaps: - def_read_set Default bitmap for columns to be read - def_write_set Default bitmap for columns to be written - tmp_set Can be used as a temporary bitmap when needed. The table object has also two pointer to bitmaps read_set and write_set that the handler should use to find out which columns are used in which way. - count() optimization now calls handler::records() instead of using handler->stats.records (if (table_flags() & HA_HAS_RECORDS) is true). - Added extra argument to Item::walk() to indicate if we should also traverse sub queries. - Added TABLE parameter to cp_buffer_from_ref() - Don't close tables created with CREATE ... SELECT but keep them in the table cache. (Faster usage of newly created tables). New interfaces: - table->clear_column_bitmaps() to initialize the bitmaps for tables at start of new statements. - table->column_bitmaps_set() to set up new column bitmaps and signal the handler about this. - table->column_bitmaps_set_no_signal() for some few cases where we need to setup new column bitmaps but don't signal the handler (as the handler has already been signaled about these before). Used for the momement only in opt_range.cc when doing ROR scans. - table->use_all_columns() to install a bitmap where all columns are marked as use in the read and the write set. - table->default_column_bitmaps() to install the normal read and write column bitmaps, but not signaling the handler about this. This is mainly used when creating TABLE instances. - table->mark_columns_needed_for_delete(), table->mark_columns_needed_for_delete() and table->mark_columns_needed_for_insert() to allow us to put additional columns in column usage maps if handler so requires. (The handler indicates what it neads in handler->table_flags()) - table->prepare_for_position() to allow us to tell handler that it needs to read primary key parts to be able to store them in future table->position() calls. (This replaces the table->file->ha_retrieve_all_pk function) - table->mark_auto_increment_column() to tell handler are going to update columns part of any auto_increment key. - table->mark_columns_used_by_index() to mark all columns that is part of an index. It will also send extra(HA_EXTRA_KEYREAD) to handler to allow it to quickly know that it only needs to read colums that are part of the key. (The handler can also use the column map for detecting this, but simpler/faster handler can just monitor the extra() call). - table->mark_columns_used_by_index_no_reset() to in addition to other columns, also mark all columns that is used by the given key. - table->restore_column_maps_after_mark_index() to restore to default column maps after a call to table->mark_columns_used_by_index(). - New item function register_field_in_read_map(), for marking used columns in table->read_map. Used by filesort() to mark all used columns - Maintain in TABLE->merge_keys set of all keys that are used in query. (Simplices some optimization loops) - Maintain Field->part_of_key_not_clustered which is like Field->part_of_key but the field in the clustered key is not assumed to be part of all index. (used in opt_range.cc for faster loops) - dbug_tmp_use_all_columns(), dbug_tmp_restore_column_map() tmp_use_all_columns() and tmp_restore_column_map() functions to temporally mark all columns as usable. The 'dbug_' version is primarily intended inside a handler when it wants to just call Field:store() & Field::val() functions, but don't need the column maps set for any other usage. (ie:: bitmap_is_set() is never called) - We can't use compare_records() to skip updates for handlers that returns a partial column set and the read_set doesn't cover all columns in the write set. The reason for this is that if we have a column marked only for write we can't in the MySQL level know if the value changed or not. The reason this worked before was that MySQL marked all to be written columns as also to be read. The new 'optimal' bitmaps exposed this 'hidden bug'. - open_table_from_share() does not anymore setup temporary MEM_ROOT object as a thread specific variable for the handler. Instead we send the to-be-used MEMROOT to get_new_handler(). (Simpler, faster code) Bugs fixed: - Column marking was not done correctly in a lot of cases. (ALTER TABLE, when using triggers, auto_increment fields etc) (Could potentially result in wrong values inserted in table handlers relying on that the old column maps or field->set_query_id was correct) Especially when it comes to triggers, there may be cases where the old code would cause lost/wrong values for NDB and/or InnoDB tables. - Split thd->options flag OPTION_STATUS_NO_TRANS_UPDATE to two flags: OPTION_STATUS_NO_TRANS_UPDATE and OPTION_KEEP_LOG. This allowed me to remove some wrong warnings about: "Some non-transactional changed tables couldn't be rolled back" - Fixed handling of INSERT .. SELECT and CREATE ... SELECT that wrongly reset (thd->options & OPTION_STATUS_NO_TRANS_UPDATE) which caused us to loose some warnings about "Some non-transactional changed tables couldn't be rolled back") - Fixed use of uninitialized memory in ha_ndbcluster.cc::delete_table() which could cause delete_table to report random failures. - Fixed core dumps for some tests when running with --debug - Added missing FN_LIBCHAR in mysql_rm_tmp_tables() (This has probably caused us to not properly remove temporary files after crash) - slow_logs was not properly initialized, which could maybe cause extra/lost entries in slow log. - If we get an duplicate row on insert, change column map to read and write all columns while retrying the operation. This is required by the definition of REPLACE and also ensures that fields that are only part of UPDATE are properly handled. This fixed a bug in NDB and REPLACE where REPLACE wrongly copied some column values from the replaced row. - For table handler that doesn't support NULL in keys, we would give an error when creating a primary key with NULL fields, even after the fields has been automaticly converted to NOT NULL. - Creating a primary key on a SPATIAL key, would fail if field was not declared as NOT NULL. Cleanups: - Removed not used condition argument to setup_tables - Removed not needed item function reset_query_id_processor(). - Field->add_index is removed. Now this is instead maintained in (field->flags & FIELD_IN_ADD_INDEX) - Field->fieldnr is removed (use field->field_index instead) - New argument to filesort() to indicate that it should return a set of row pointers (not used columns). This allowed me to remove some references to sql_command in filesort and should also enable us to return column results in some cases where we couldn't before. - Changed column bitmap handling in opt_range.cc to be aligned with TABLE bitmap, which allowed me to use bitmap functions instead of looping over all fields to create some needed bitmaps. (Faster and smaller code) - Broke up found too long lines - Moved some variable declaration at start of function for better code readability. - Removed some not used arguments from functions. (setup_fields(), mysql_prepare_insert_check_table()) - setup_fields() now takes an enum instead of an int for marking columns usage. - For internal temporary tables, use handler::write_row(), handler::delete_row() and handler::update_row() instead of handler::ha_xxxx() for faster execution. - Changed some constants to enum's and define's. - Using separate column read and write sets allows for easier checking of timestamp field was set by statement. - Remove calls to free_io_cache() as this is now done automaticly in ha_reset() - Don't build table->normalized_path as this is now identical to table->path (after bar's fixes to convert filenames) - Fixed some missed DBUG_PRINT(.."%lx") to use "0x%lx" to make it easier to do comparision with the 'convert-dbug-for-diff' tool. Things left to do in 5.1: - We wrongly log failed CREATE TABLE ... SELECT in some cases when using row based logging (as shown by testcase binlog_row_mix_innodb_myisam.result) Mats has promised to look into this. - Test that my fix for CREATE TABLE ... SELECT is indeed correct. (I added several test cases for this, but in this case it's better that someone else also tests this throughly). Lars has promosed to do this.
20 years ago
This changeset is largely a handler cleanup changeset (WL#3281), but includes fixes and cleanups that was found necessary while testing the handler changes Changes that requires code changes in other code of other storage engines. (Note that all changes are very straightforward and one should find all issues by compiling a --debug build and fixing all compiler errors and all asserts in field.cc while running the test suite), - New optional handler function introduced: reset() This is called after every DML statement to make it easy for a handler to statement specific cleanups. (The only case it's not called is if force the file to be closed) - handler::extra(HA_EXTRA_RESET) is removed. Code that was there before should be moved to handler::reset() - table->read_set contains a bitmap over all columns that are needed in the query. read_row() and similar functions only needs to read these columns - table->write_set contains a bitmap over all columns that will be updated in the query. write_row() and update_row() only needs to update these columns. The above bitmaps should now be up to date in all context (including ALTER TABLE, filesort()). The handler is informed of any changes to the bitmap after fix_fields() by calling the virtual function handler::column_bitmaps_signal(). If the handler does caching of these bitmaps (instead of using table->read_set, table->write_set), it should redo the caching in this code. as the signal() may be sent several times, it's probably best to set a variable in the signal and redo the caching on read_row() / write_row() if the variable was set. - Removed the read_set and write_set bitmap objects from the handler class - Removed all column bit handling functions from the handler class. (Now one instead uses the normal bitmap functions in my_bitmap.c instead of handler dedicated bitmap functions) - field->query_id is removed. One should instead instead check table->read_set and table->write_set if a field is used in the query. - handler::extra(HA_EXTRA_RETRIVE_ALL_COLS) and handler::extra(HA_EXTRA_RETRIEVE_PRIMARY_KEY) are removed. One should now instead use table->read_set to check for which columns to retrieve. - If a handler needs to call Field->val() or Field->store() on columns that are not used in the query, one should install a temporary all-columns-used map while doing so. For this, we provide the following functions: my_bitmap_map *old_map= dbug_tmp_use_all_columns(table, table->read_set); field->val(); dbug_tmp_restore_column_map(table->read_set, old_map); and similar for the write map: my_bitmap_map *old_map= dbug_tmp_use_all_columns(table, table->write_set); field->val(); dbug_tmp_restore_column_map(table->write_set, old_map); If this is not done, you will sooner or later hit a DBUG_ASSERT in the field store() / val() functions. (For not DBUG binaries, the dbug_tmp_restore_column_map() and dbug_tmp_restore_column_map() are inline dummy functions and should be optimized away be the compiler). - If one needs to temporary set the column map for all binaries (and not just to avoid the DBUG_ASSERT() in the Field::store() / Field::val() methods) one should use the functions tmp_use_all_columns() and tmp_restore_column_map() instead of the above dbug_ variants. - All 'status' fields in the handler base class (like records, data_file_length etc) are now stored in a 'stats' struct. This makes it easier to know what status variables are provided by the base handler. This requires some trivial variable names in the extra() function. - New virtual function handler::records(). This is called to optimize COUNT(*) if (handler::table_flags() & HA_HAS_RECORDS()) is true. (stats.records is not supposed to be an exact value. It's only has to be 'reasonable enough' for the optimizer to be able to choose a good optimization path). - Non virtual handler::init() function added for caching of virtual constants from engine. - Removed has_transactions() virtual method. Now one should instead return HA_NO_TRANSACTIONS in table_flags() if the table handler DOES NOT support transactions. - The 'xxxx_create_handler()' function now has a MEM_ROOT_root argument that is to be used with 'new handler_name()' to allocate the handler in the right area. The xxxx_create_handler() function is also responsible for any initialization of the object before returning. For example, one should change: static handler *myisam_create_handler(TABLE_SHARE *table) { return new ha_myisam(table); } -> static handler *myisam_create_handler(TABLE_SHARE *table, MEM_ROOT *mem_root) { return new (mem_root) ha_myisam(table); } - New optional virtual function: use_hidden_primary_key(). This is called in case of an update/delete when (table_flags() and HA_PRIMARY_KEY_REQUIRED_FOR_DELETE) is defined but we don't have a primary key. This allows the handler to take precisions in remembering any hidden primary key to able to update/delete any found row. The default handler marks all columns to be read. - handler::table_flags() now returns a ulonglong (to allow for more flags). - New/changed table_flags() - HA_HAS_RECORDS Set if ::records() is supported - HA_NO_TRANSACTIONS Set if engine doesn't support transactions - HA_PRIMARY_KEY_REQUIRED_FOR_DELETE Set if we should mark all primary key columns for read when reading rows as part of a DELETE statement. If there is no primary key, all columns are marked for read. - HA_PARTIAL_COLUMN_READ Set if engine will not read all columns in some cases (based on table->read_set) - HA_PRIMARY_KEY_ALLOW_RANDOM_ACCESS Renamed to HA_PRIMARY_KEY_REQUIRED_FOR_POSITION. - HA_DUPP_POS Renamed to HA_DUPLICATE_POS - HA_REQUIRES_KEY_COLUMNS_FOR_DELETE Set this if we should mark ALL key columns for read when when reading rows as part of a DELETE statement. In case of an update we will mark all keys for read for which key part changed value. - HA_STATS_RECORDS_IS_EXACT Set this if stats.records is exact. (This saves us some extra records() calls when optimizing COUNT(*)) - Removed table_flags() - HA_NOT_EXACT_COUNT Now one should instead use HA_HAS_RECORDS if handler::records() gives an exact count() and HA_STATS_RECORDS_IS_EXACT if stats.records is exact. - HA_READ_RND_SAME Removed (no one supported this one) - Removed not needed functions ha_retrieve_all_cols() and ha_retrieve_all_pk() - Renamed handler::dupp_pos to handler::dup_pos - Removed not used variable handler::sortkey Upper level handler changes: - ha_reset() now does some overall checks and calls ::reset() - ha_table_flags() added. This is a cached version of table_flags(). The cache is updated on engine creation time and updated on open. MySQL level changes (not obvious from the above): - DBUG_ASSERT() added to check that column usage matches what is set in the column usage bit maps. (This found a LOT of bugs in current column marking code). - In 5.1 before, all used columns was marked in read_set and only updated columns was marked in write_set. Now we only mark columns for which we need a value in read_set. - Column bitmaps are created in open_binary_frm() and open_table_from_share(). (Before this was in table.cc) - handler::table_flags() calls are replaced with handler::ha_table_flags() - For calling field->val() you must have the corresponding bit set in table->read_set. For calling field->store() you must have the corresponding bit set in table->write_set. (There are asserts in all store()/val() functions to catch wrong usage) - thd->set_query_id is renamed to thd->mark_used_columns and instead of setting this to an integer value, this has now the values: MARK_COLUMNS_NONE, MARK_COLUMNS_READ, MARK_COLUMNS_WRITE Changed also all variables named 'set_query_id' to mark_used_columns. - In filesort() we now inform the handler of exactly which columns are needed doing the sort and choosing the rows. - The TABLE_SHARE object has a 'all_set' column bitmap one can use when one needs a column bitmap with all columns set. (This is used for table->use_all_columns() and other places) - The TABLE object has 3 column bitmaps: - def_read_set Default bitmap for columns to be read - def_write_set Default bitmap for columns to be written - tmp_set Can be used as a temporary bitmap when needed. The table object has also two pointer to bitmaps read_set and write_set that the handler should use to find out which columns are used in which way. - count() optimization now calls handler::records() instead of using handler->stats.records (if (table_flags() & HA_HAS_RECORDS) is true). - Added extra argument to Item::walk() to indicate if we should also traverse sub queries. - Added TABLE parameter to cp_buffer_from_ref() - Don't close tables created with CREATE ... SELECT but keep them in the table cache. (Faster usage of newly created tables). New interfaces: - table->clear_column_bitmaps() to initialize the bitmaps for tables at start of new statements. - table->column_bitmaps_set() to set up new column bitmaps and signal the handler about this. - table->column_bitmaps_set_no_signal() for some few cases where we need to setup new column bitmaps but don't signal the handler (as the handler has already been signaled about these before). Used for the momement only in opt_range.cc when doing ROR scans. - table->use_all_columns() to install a bitmap where all columns are marked as use in the read and the write set. - table->default_column_bitmaps() to install the normal read and write column bitmaps, but not signaling the handler about this. This is mainly used when creating TABLE instances. - table->mark_columns_needed_for_delete(), table->mark_columns_needed_for_delete() and table->mark_columns_needed_for_insert() to allow us to put additional columns in column usage maps if handler so requires. (The handler indicates what it neads in handler->table_flags()) - table->prepare_for_position() to allow us to tell handler that it needs to read primary key parts to be able to store them in future table->position() calls. (This replaces the table->file->ha_retrieve_all_pk function) - table->mark_auto_increment_column() to tell handler are going to update columns part of any auto_increment key. - table->mark_columns_used_by_index() to mark all columns that is part of an index. It will also send extra(HA_EXTRA_KEYREAD) to handler to allow it to quickly know that it only needs to read colums that are part of the key. (The handler can also use the column map for detecting this, but simpler/faster handler can just monitor the extra() call). - table->mark_columns_used_by_index_no_reset() to in addition to other columns, also mark all columns that is used by the given key. - table->restore_column_maps_after_mark_index() to restore to default column maps after a call to table->mark_columns_used_by_index(). - New item function register_field_in_read_map(), for marking used columns in table->read_map. Used by filesort() to mark all used columns - Maintain in TABLE->merge_keys set of all keys that are used in query. (Simplices some optimization loops) - Maintain Field->part_of_key_not_clustered which is like Field->part_of_key but the field in the clustered key is not assumed to be part of all index. (used in opt_range.cc for faster loops) - dbug_tmp_use_all_columns(), dbug_tmp_restore_column_map() tmp_use_all_columns() and tmp_restore_column_map() functions to temporally mark all columns as usable. The 'dbug_' version is primarily intended inside a handler when it wants to just call Field:store() & Field::val() functions, but don't need the column maps set for any other usage. (ie:: bitmap_is_set() is never called) - We can't use compare_records() to skip updates for handlers that returns a partial column set and the read_set doesn't cover all columns in the write set. The reason for this is that if we have a column marked only for write we can't in the MySQL level know if the value changed or not. The reason this worked before was that MySQL marked all to be written columns as also to be read. The new 'optimal' bitmaps exposed this 'hidden bug'. - open_table_from_share() does not anymore setup temporary MEM_ROOT object as a thread specific variable for the handler. Instead we send the to-be-used MEMROOT to get_new_handler(). (Simpler, faster code) Bugs fixed: - Column marking was not done correctly in a lot of cases. (ALTER TABLE, when using triggers, auto_increment fields etc) (Could potentially result in wrong values inserted in table handlers relying on that the old column maps or field->set_query_id was correct) Especially when it comes to triggers, there may be cases where the old code would cause lost/wrong values for NDB and/or InnoDB tables. - Split thd->options flag OPTION_STATUS_NO_TRANS_UPDATE to two flags: OPTION_STATUS_NO_TRANS_UPDATE and OPTION_KEEP_LOG. This allowed me to remove some wrong warnings about: "Some non-transactional changed tables couldn't be rolled back" - Fixed handling of INSERT .. SELECT and CREATE ... SELECT that wrongly reset (thd->options & OPTION_STATUS_NO_TRANS_UPDATE) which caused us to loose some warnings about "Some non-transactional changed tables couldn't be rolled back") - Fixed use of uninitialized memory in ha_ndbcluster.cc::delete_table() which could cause delete_table to report random failures. - Fixed core dumps for some tests when running with --debug - Added missing FN_LIBCHAR in mysql_rm_tmp_tables() (This has probably caused us to not properly remove temporary files after crash) - slow_logs was not properly initialized, which could maybe cause extra/lost entries in slow log. - If we get an duplicate row on insert, change column map to read and write all columns while retrying the operation. This is required by the definition of REPLACE and also ensures that fields that are only part of UPDATE are properly handled. This fixed a bug in NDB and REPLACE where REPLACE wrongly copied some column values from the replaced row. - For table handler that doesn't support NULL in keys, we would give an error when creating a primary key with NULL fields, even after the fields has been automaticly converted to NOT NULL. - Creating a primary key on a SPATIAL key, would fail if field was not declared as NOT NULL. Cleanups: - Removed not used condition argument to setup_tables - Removed not needed item function reset_query_id_processor(). - Field->add_index is removed. Now this is instead maintained in (field->flags & FIELD_IN_ADD_INDEX) - Field->fieldnr is removed (use field->field_index instead) - New argument to filesort() to indicate that it should return a set of row pointers (not used columns). This allowed me to remove some references to sql_command in filesort and should also enable us to return column results in some cases where we couldn't before. - Changed column bitmap handling in opt_range.cc to be aligned with TABLE bitmap, which allowed me to use bitmap functions instead of looping over all fields to create some needed bitmaps. (Faster and smaller code) - Broke up found too long lines - Moved some variable declaration at start of function for better code readability. - Removed some not used arguments from functions. (setup_fields(), mysql_prepare_insert_check_table()) - setup_fields() now takes an enum instead of an int for marking columns usage. - For internal temporary tables, use handler::write_row(), handler::delete_row() and handler::update_row() instead of handler::ha_xxxx() for faster execution. - Changed some constants to enum's and define's. - Using separate column read and write sets allows for easier checking of timestamp field was set by statement. - Remove calls to free_io_cache() as this is now done automaticly in ha_reset() - Don't build table->normalized_path as this is now identical to table->path (after bar's fixes to convert filenames) - Fixed some missed DBUG_PRINT(.."%lx") to use "0x%lx" to make it easier to do comparision with the 'convert-dbug-for-diff' tool. Things left to do in 5.1: - We wrongly log failed CREATE TABLE ... SELECT in some cases when using row based logging (as shown by testcase binlog_row_mix_innodb_myisam.result) Mats has promised to look into this. - Test that my fix for CREATE TABLE ... SELECT is indeed correct. (I added several test cases for this, but in this case it's better that someone else also tests this throughly). Lars has promosed to do this.
20 years ago
Implement new type-of-operation-aware metadata locks. Add a wait-for graph based deadlock detector to the MDL subsystem. Fixes bug #46272 "MySQL 5.4.4, new MDL: unnecessary deadlock" and bug #37346 "innodb does not detect deadlock between update and alter table". The first bug manifested itself as an unwarranted abort of a transaction with ER_LOCK_DEADLOCK error by a concurrent ALTER statement, when this transaction tried to repeat use of a table, which it has already used in a similar fashion before ALTER started. The second bug showed up as a deadlock between table-level locks and InnoDB row locks, which was "detected" only after innodb_lock_wait_timeout timeout. A transaction would start using the table and modify a few rows. Then ALTER TABLE would come in, and start copying rows into a temporary table. Eventually it would stumble on the modified records and get blocked on a row lock. The first transaction would try to do more updates, and get blocked on thr_lock.c lock. This situation of circular wait would only get resolved by a timeout. Both these bugs stemmed from inadequate solutions to the problem of deadlocks occurring between different locking subsystems. In the first case we tried to avoid deadlocks between metadata locking and table-level locking subsystems, when upgrading shared metadata lock to exclusive one. Transactions holding the shared lock on the table and waiting for some table-level lock used to be aborted too aggressively. We also allowed ALTER TABLE to start in presence of transactions that modify the subject table. ALTER TABLE acquires TL_WRITE_ALLOW_READ lock at start, and that block all writes against the table (naturally, we don't want any writes to be lost when switching the old and the new table). TL_WRITE_ALLOW_READ lock, in turn, would block the started transaction on thr_lock.c lock, should they do more updates. This, again, lead to the need to abort such transactions. The second bug occurred simply because we didn't have any mechanism to detect deadlocks between the table-level locks in thr_lock.c and row-level locks in InnoDB, other than innodb_lock_wait_timeout. This patch solves both these problems by moving lock conflicts which are causing these deadlocks into the metadata locking subsystem, thus making it possible to avoid or detect such deadlocks inside MDL. To do this we introduce new type-of-operation-aware metadata locks, which allow MDL subsystem to know not only the fact that transaction has used or is going to use some object but also what kind of operation it has carried out or going to carry out on the object. This, along with the addition of a special kind of upgradable metadata lock, allows ALTER TABLE to wait until all transactions which has updated the table to go away. This solves the second issue. Another special type of upgradable metadata lock is acquired by LOCK TABLE WRITE. This second lock type allows to solve the first issue, since abortion of table-level locks in event of DDL under LOCK TABLES becomes also unnecessary. Below follows the list of incompatible changes introduced by this patch: - From now on, ALTER TABLE and CREATE/DROP TRIGGER SQL (i.e. those statements that acquire TL_WRITE_ALLOW_READ lock) wait for all transactions which has *updated* the table to complete. - From now on, LOCK TABLES ... WRITE, REPAIR/OPTIMIZE TABLE (i.e. all statements which acquire TL_WRITE table-level lock) wait for all transaction which *updated or read* from the table to complete. As a consequence, innodb_table_locks=0 option no longer applies to LOCK TABLES ... WRITE. - DROP DATABASE, DROP TABLE, RENAME TABLE no longer abort statements or transactions which use tables being dropped or renamed, and instead wait for these transactions to complete. - Since LOCK TABLES WRITE now takes a special metadata lock, not compatible with with reads or writes against the subject table and transaction-wide, thr_lock.c deadlock avoidance algorithm that used to ensure absence of deadlocks between LOCK TABLES WRITE and other statements is no longer sufficient, even for MyISAM. The wait-for graph based deadlock detector of MDL subsystem may sometimes be necessary and is involved. This may lead to ER_LOCK_DEADLOCK error produced for multi-statement transactions even if these only use MyISAM: session 1: session 2: begin; update t1 ... lock table t2 write, t1 write; -- gets a lock on t2, blocks on t1 update t2 ... (ER_LOCK_DEADLOCK) - Finally, support of LOW_PRIORITY option for LOCK TABLES ... WRITE was abandoned. LOCK TABLE ... LOW_PRIORITY WRITE from now on has the same priority as the usual LOCK TABLE ... WRITE. SELECT HIGH PRIORITY no longer trumps LOCK TABLE ... WRITE in the wait queue. - We do not take upgradable metadata locks on implicitly locked tables. So if one has, say, a view v1 that uses table t1, and issues: LOCK TABLE v1 WRITE; FLUSH TABLE t1; -- (or just 'FLUSH TABLES'), an error is produced. In order to be able to perform DDL on a table under LOCK TABLES, the table must be locked explicitly in the LOCK TABLES list.
16 years ago
22 years ago
Implement new type-of-operation-aware metadata locks. Add a wait-for graph based deadlock detector to the MDL subsystem. Fixes bug #46272 "MySQL 5.4.4, new MDL: unnecessary deadlock" and bug #37346 "innodb does not detect deadlock between update and alter table". The first bug manifested itself as an unwarranted abort of a transaction with ER_LOCK_DEADLOCK error by a concurrent ALTER statement, when this transaction tried to repeat use of a table, which it has already used in a similar fashion before ALTER started. The second bug showed up as a deadlock between table-level locks and InnoDB row locks, which was "detected" only after innodb_lock_wait_timeout timeout. A transaction would start using the table and modify a few rows. Then ALTER TABLE would come in, and start copying rows into a temporary table. Eventually it would stumble on the modified records and get blocked on a row lock. The first transaction would try to do more updates, and get blocked on thr_lock.c lock. This situation of circular wait would only get resolved by a timeout. Both these bugs stemmed from inadequate solutions to the problem of deadlocks occurring between different locking subsystems. In the first case we tried to avoid deadlocks between metadata locking and table-level locking subsystems, when upgrading shared metadata lock to exclusive one. Transactions holding the shared lock on the table and waiting for some table-level lock used to be aborted too aggressively. We also allowed ALTER TABLE to start in presence of transactions that modify the subject table. ALTER TABLE acquires TL_WRITE_ALLOW_READ lock at start, and that block all writes against the table (naturally, we don't want any writes to be lost when switching the old and the new table). TL_WRITE_ALLOW_READ lock, in turn, would block the started transaction on thr_lock.c lock, should they do more updates. This, again, lead to the need to abort such transactions. The second bug occurred simply because we didn't have any mechanism to detect deadlocks between the table-level locks in thr_lock.c and row-level locks in InnoDB, other than innodb_lock_wait_timeout. This patch solves both these problems by moving lock conflicts which are causing these deadlocks into the metadata locking subsystem, thus making it possible to avoid or detect such deadlocks inside MDL. To do this we introduce new type-of-operation-aware metadata locks, which allow MDL subsystem to know not only the fact that transaction has used or is going to use some object but also what kind of operation it has carried out or going to carry out on the object. This, along with the addition of a special kind of upgradable metadata lock, allows ALTER TABLE to wait until all transactions which has updated the table to go away. This solves the second issue. Another special type of upgradable metadata lock is acquired by LOCK TABLE WRITE. This second lock type allows to solve the first issue, since abortion of table-level locks in event of DDL under LOCK TABLES becomes also unnecessary. Below follows the list of incompatible changes introduced by this patch: - From now on, ALTER TABLE and CREATE/DROP TRIGGER SQL (i.e. those statements that acquire TL_WRITE_ALLOW_READ lock) wait for all transactions which has *updated* the table to complete. - From now on, LOCK TABLES ... WRITE, REPAIR/OPTIMIZE TABLE (i.e. all statements which acquire TL_WRITE table-level lock) wait for all transaction which *updated or read* from the table to complete. As a consequence, innodb_table_locks=0 option no longer applies to LOCK TABLES ... WRITE. - DROP DATABASE, DROP TABLE, RENAME TABLE no longer abort statements or transactions which use tables being dropped or renamed, and instead wait for these transactions to complete. - Since LOCK TABLES WRITE now takes a special metadata lock, not compatible with with reads or writes against the subject table and transaction-wide, thr_lock.c deadlock avoidance algorithm that used to ensure absence of deadlocks between LOCK TABLES WRITE and other statements is no longer sufficient, even for MyISAM. The wait-for graph based deadlock detector of MDL subsystem may sometimes be necessary and is involved. This may lead to ER_LOCK_DEADLOCK error produced for multi-statement transactions even if these only use MyISAM: session 1: session 2: begin; update t1 ... lock table t2 write, t1 write; -- gets a lock on t2, blocks on t1 update t2 ... (ER_LOCK_DEADLOCK) - Finally, support of LOW_PRIORITY option for LOCK TABLES ... WRITE was abandoned. LOCK TABLE ... LOW_PRIORITY WRITE from now on has the same priority as the usual LOCK TABLE ... WRITE. SELECT HIGH PRIORITY no longer trumps LOCK TABLE ... WRITE in the wait queue. - We do not take upgradable metadata locks on implicitly locked tables. So if one has, say, a view v1 that uses table t1, and issues: LOCK TABLE v1 WRITE; FLUSH TABLE t1; -- (or just 'FLUSH TABLES'), an error is produced. In order to be able to perform DDL on a table under LOCK TABLES, the table must be locked explicitly in the LOCK TABLES list.
16 years ago
Backport of revno 2630.28.10, 2630.28.31, 2630.28.26, 2630.33.1, 2630.39.1, 2630.28.29, 2630.34.3, 2630.34.2, 2630.34.1, 2630.29.29, 2630.29.28, 2630.31.1, 2630.28.13, 2630.28.10, 2617.23.14 and some other minor revisions. This patch implements: WL#4264 "Backup: Stabilize Service Interface" -- all the server prerequisites except si_objects.{h,cc} themselves (they can be just copied over, when needed). WL#4435: Support OUT-parameters in prepared statements. (and all issues in the initial patches for these two tasks, that were discovered in pushbuild and during testing). Bug#39519: mysql_stmt_close() should flush all data associated with the statement. After execution of a prepared statement, send OUT parameters of the invoked stored procedure, if any, to the client. When using the binary protocol, send the parameters in an additional result set over the wire. When using the text protocol, assign out parameters to the user variables from the CALL(@var1, @var2, ...) specification. The following refactoring has been made: - Protocol::send_fields() was renamed to Protocol::send_result_set_metadata(); - A new Protocol::send_result_set_row() was introduced to incapsulate common functionality for sending row data. - Signature of Protocol::prepare_for_send() was changed: this operation does not need a list of items, the number of items is fully sufficient. The following backward incompatible changes have been made: - CLIENT_MULTI_RESULTS is now enabled by default in the client; - CLIENT_PS_MULTI_RESUTLS is now enabled by default in the client.
16 years ago
Implement new type-of-operation-aware metadata locks. Add a wait-for graph based deadlock detector to the MDL subsystem. Fixes bug #46272 "MySQL 5.4.4, new MDL: unnecessary deadlock" and bug #37346 "innodb does not detect deadlock between update and alter table". The first bug manifested itself as an unwarranted abort of a transaction with ER_LOCK_DEADLOCK error by a concurrent ALTER statement, when this transaction tried to repeat use of a table, which it has already used in a similar fashion before ALTER started. The second bug showed up as a deadlock between table-level locks and InnoDB row locks, which was "detected" only after innodb_lock_wait_timeout timeout. A transaction would start using the table and modify a few rows. Then ALTER TABLE would come in, and start copying rows into a temporary table. Eventually it would stumble on the modified records and get blocked on a row lock. The first transaction would try to do more updates, and get blocked on thr_lock.c lock. This situation of circular wait would only get resolved by a timeout. Both these bugs stemmed from inadequate solutions to the problem of deadlocks occurring between different locking subsystems. In the first case we tried to avoid deadlocks between metadata locking and table-level locking subsystems, when upgrading shared metadata lock to exclusive one. Transactions holding the shared lock on the table and waiting for some table-level lock used to be aborted too aggressively. We also allowed ALTER TABLE to start in presence of transactions that modify the subject table. ALTER TABLE acquires TL_WRITE_ALLOW_READ lock at start, and that block all writes against the table (naturally, we don't want any writes to be lost when switching the old and the new table). TL_WRITE_ALLOW_READ lock, in turn, would block the started transaction on thr_lock.c lock, should they do more updates. This, again, lead to the need to abort such transactions. The second bug occurred simply because we didn't have any mechanism to detect deadlocks between the table-level locks in thr_lock.c and row-level locks in InnoDB, other than innodb_lock_wait_timeout. This patch solves both these problems by moving lock conflicts which are causing these deadlocks into the metadata locking subsystem, thus making it possible to avoid or detect such deadlocks inside MDL. To do this we introduce new type-of-operation-aware metadata locks, which allow MDL subsystem to know not only the fact that transaction has used or is going to use some object but also what kind of operation it has carried out or going to carry out on the object. This, along with the addition of a special kind of upgradable metadata lock, allows ALTER TABLE to wait until all transactions which has updated the table to go away. This solves the second issue. Another special type of upgradable metadata lock is acquired by LOCK TABLE WRITE. This second lock type allows to solve the first issue, since abortion of table-level locks in event of DDL under LOCK TABLES becomes also unnecessary. Below follows the list of incompatible changes introduced by this patch: - From now on, ALTER TABLE and CREATE/DROP TRIGGER SQL (i.e. those statements that acquire TL_WRITE_ALLOW_READ lock) wait for all transactions which has *updated* the table to complete. - From now on, LOCK TABLES ... WRITE, REPAIR/OPTIMIZE TABLE (i.e. all statements which acquire TL_WRITE table-level lock) wait for all transaction which *updated or read* from the table to complete. As a consequence, innodb_table_locks=0 option no longer applies to LOCK TABLES ... WRITE. - DROP DATABASE, DROP TABLE, RENAME TABLE no longer abort statements or transactions which use tables being dropped or renamed, and instead wait for these transactions to complete. - Since LOCK TABLES WRITE now takes a special metadata lock, not compatible with with reads or writes against the subject table and transaction-wide, thr_lock.c deadlock avoidance algorithm that used to ensure absence of deadlocks between LOCK TABLES WRITE and other statements is no longer sufficient, even for MyISAM. The wait-for graph based deadlock detector of MDL subsystem may sometimes be necessary and is involved. This may lead to ER_LOCK_DEADLOCK error produced for multi-statement transactions even if these only use MyISAM: session 1: session 2: begin; update t1 ... lock table t2 write, t1 write; -- gets a lock on t2, blocks on t1 update t2 ... (ER_LOCK_DEADLOCK) - Finally, support of LOW_PRIORITY option for LOCK TABLES ... WRITE was abandoned. LOCK TABLE ... LOW_PRIORITY WRITE from now on has the same priority as the usual LOCK TABLE ... WRITE. SELECT HIGH PRIORITY no longer trumps LOCK TABLE ... WRITE in the wait queue. - We do not take upgradable metadata locks on implicitly locked tables. So if one has, say, a view v1 that uses table t1, and issues: LOCK TABLE v1 WRITE; FLUSH TABLE t1; -- (or just 'FLUSH TABLES'), an error is produced. In order to be able to perform DDL on a table under LOCK TABLES, the table must be locked explicitly in the LOCK TABLES list.
16 years ago
This changeset is largely a handler cleanup changeset (WL#3281), but includes fixes and cleanups that was found necessary while testing the handler changes Changes that requires code changes in other code of other storage engines. (Note that all changes are very straightforward and one should find all issues by compiling a --debug build and fixing all compiler errors and all asserts in field.cc while running the test suite), - New optional handler function introduced: reset() This is called after every DML statement to make it easy for a handler to statement specific cleanups. (The only case it's not called is if force the file to be closed) - handler::extra(HA_EXTRA_RESET) is removed. Code that was there before should be moved to handler::reset() - table->read_set contains a bitmap over all columns that are needed in the query. read_row() and similar functions only needs to read these columns - table->write_set contains a bitmap over all columns that will be updated in the query. write_row() and update_row() only needs to update these columns. The above bitmaps should now be up to date in all context (including ALTER TABLE, filesort()). The handler is informed of any changes to the bitmap after fix_fields() by calling the virtual function handler::column_bitmaps_signal(). If the handler does caching of these bitmaps (instead of using table->read_set, table->write_set), it should redo the caching in this code. as the signal() may be sent several times, it's probably best to set a variable in the signal and redo the caching on read_row() / write_row() if the variable was set. - Removed the read_set and write_set bitmap objects from the handler class - Removed all column bit handling functions from the handler class. (Now one instead uses the normal bitmap functions in my_bitmap.c instead of handler dedicated bitmap functions) - field->query_id is removed. One should instead instead check table->read_set and table->write_set if a field is used in the query. - handler::extra(HA_EXTRA_RETRIVE_ALL_COLS) and handler::extra(HA_EXTRA_RETRIEVE_PRIMARY_KEY) are removed. One should now instead use table->read_set to check for which columns to retrieve. - If a handler needs to call Field->val() or Field->store() on columns that are not used in the query, one should install a temporary all-columns-used map while doing so. For this, we provide the following functions: my_bitmap_map *old_map= dbug_tmp_use_all_columns(table, table->read_set); field->val(); dbug_tmp_restore_column_map(table->read_set, old_map); and similar for the write map: my_bitmap_map *old_map= dbug_tmp_use_all_columns(table, table->write_set); field->val(); dbug_tmp_restore_column_map(table->write_set, old_map); If this is not done, you will sooner or later hit a DBUG_ASSERT in the field store() / val() functions. (For not DBUG binaries, the dbug_tmp_restore_column_map() and dbug_tmp_restore_column_map() are inline dummy functions and should be optimized away be the compiler). - If one needs to temporary set the column map for all binaries (and not just to avoid the DBUG_ASSERT() in the Field::store() / Field::val() methods) one should use the functions tmp_use_all_columns() and tmp_restore_column_map() instead of the above dbug_ variants. - All 'status' fields in the handler base class (like records, data_file_length etc) are now stored in a 'stats' struct. This makes it easier to know what status variables are provided by the base handler. This requires some trivial variable names in the extra() function. - New virtual function handler::records(). This is called to optimize COUNT(*) if (handler::table_flags() & HA_HAS_RECORDS()) is true. (stats.records is not supposed to be an exact value. It's only has to be 'reasonable enough' for the optimizer to be able to choose a good optimization path). - Non virtual handler::init() function added for caching of virtual constants from engine. - Removed has_transactions() virtual method. Now one should instead return HA_NO_TRANSACTIONS in table_flags() if the table handler DOES NOT support transactions. - The 'xxxx_create_handler()' function now has a MEM_ROOT_root argument that is to be used with 'new handler_name()' to allocate the handler in the right area. The xxxx_create_handler() function is also responsible for any initialization of the object before returning. For example, one should change: static handler *myisam_create_handler(TABLE_SHARE *table) { return new ha_myisam(table); } -> static handler *myisam_create_handler(TABLE_SHARE *table, MEM_ROOT *mem_root) { return new (mem_root) ha_myisam(table); } - New optional virtual function: use_hidden_primary_key(). This is called in case of an update/delete when (table_flags() and HA_PRIMARY_KEY_REQUIRED_FOR_DELETE) is defined but we don't have a primary key. This allows the handler to take precisions in remembering any hidden primary key to able to update/delete any found row. The default handler marks all columns to be read. - handler::table_flags() now returns a ulonglong (to allow for more flags). - New/changed table_flags() - HA_HAS_RECORDS Set if ::records() is supported - HA_NO_TRANSACTIONS Set if engine doesn't support transactions - HA_PRIMARY_KEY_REQUIRED_FOR_DELETE Set if we should mark all primary key columns for read when reading rows as part of a DELETE statement. If there is no primary key, all columns are marked for read. - HA_PARTIAL_COLUMN_READ Set if engine will not read all columns in some cases (based on table->read_set) - HA_PRIMARY_KEY_ALLOW_RANDOM_ACCESS Renamed to HA_PRIMARY_KEY_REQUIRED_FOR_POSITION. - HA_DUPP_POS Renamed to HA_DUPLICATE_POS - HA_REQUIRES_KEY_COLUMNS_FOR_DELETE Set this if we should mark ALL key columns for read when when reading rows as part of a DELETE statement. In case of an update we will mark all keys for read for which key part changed value. - HA_STATS_RECORDS_IS_EXACT Set this if stats.records is exact. (This saves us some extra records() calls when optimizing COUNT(*)) - Removed table_flags() - HA_NOT_EXACT_COUNT Now one should instead use HA_HAS_RECORDS if handler::records() gives an exact count() and HA_STATS_RECORDS_IS_EXACT if stats.records is exact. - HA_READ_RND_SAME Removed (no one supported this one) - Removed not needed functions ha_retrieve_all_cols() and ha_retrieve_all_pk() - Renamed handler::dupp_pos to handler::dup_pos - Removed not used variable handler::sortkey Upper level handler changes: - ha_reset() now does some overall checks and calls ::reset() - ha_table_flags() added. This is a cached version of table_flags(). The cache is updated on engine creation time and updated on open. MySQL level changes (not obvious from the above): - DBUG_ASSERT() added to check that column usage matches what is set in the column usage bit maps. (This found a LOT of bugs in current column marking code). - In 5.1 before, all used columns was marked in read_set and only updated columns was marked in write_set. Now we only mark columns for which we need a value in read_set. - Column bitmaps are created in open_binary_frm() and open_table_from_share(). (Before this was in table.cc) - handler::table_flags() calls are replaced with handler::ha_table_flags() - For calling field->val() you must have the corresponding bit set in table->read_set. For calling field->store() you must have the corresponding bit set in table->write_set. (There are asserts in all store()/val() functions to catch wrong usage) - thd->set_query_id is renamed to thd->mark_used_columns and instead of setting this to an integer value, this has now the values: MARK_COLUMNS_NONE, MARK_COLUMNS_READ, MARK_COLUMNS_WRITE Changed also all variables named 'set_query_id' to mark_used_columns. - In filesort() we now inform the handler of exactly which columns are needed doing the sort and choosing the rows. - The TABLE_SHARE object has a 'all_set' column bitmap one can use when one needs a column bitmap with all columns set. (This is used for table->use_all_columns() and other places) - The TABLE object has 3 column bitmaps: - def_read_set Default bitmap for columns to be read - def_write_set Default bitmap for columns to be written - tmp_set Can be used as a temporary bitmap when needed. The table object has also two pointer to bitmaps read_set and write_set that the handler should use to find out which columns are used in which way. - count() optimization now calls handler::records() instead of using handler->stats.records (if (table_flags() & HA_HAS_RECORDS) is true). - Added extra argument to Item::walk() to indicate if we should also traverse sub queries. - Added TABLE parameter to cp_buffer_from_ref() - Don't close tables created with CREATE ... SELECT but keep them in the table cache. (Faster usage of newly created tables). New interfaces: - table->clear_column_bitmaps() to initialize the bitmaps for tables at start of new statements. - table->column_bitmaps_set() to set up new column bitmaps and signal the handler about this. - table->column_bitmaps_set_no_signal() for some few cases where we need to setup new column bitmaps but don't signal the handler (as the handler has already been signaled about these before). Used for the momement only in opt_range.cc when doing ROR scans. - table->use_all_columns() to install a bitmap where all columns are marked as use in the read and the write set. - table->default_column_bitmaps() to install the normal read and write column bitmaps, but not signaling the handler about this. This is mainly used when creating TABLE instances. - table->mark_columns_needed_for_delete(), table->mark_columns_needed_for_delete() and table->mark_columns_needed_for_insert() to allow us to put additional columns in column usage maps if handler so requires. (The handler indicates what it neads in handler->table_flags()) - table->prepare_for_position() to allow us to tell handler that it needs to read primary key parts to be able to store them in future table->position() calls. (This replaces the table->file->ha_retrieve_all_pk function) - table->mark_auto_increment_column() to tell handler are going to update columns part of any auto_increment key. - table->mark_columns_used_by_index() to mark all columns that is part of an index. It will also send extra(HA_EXTRA_KEYREAD) to handler to allow it to quickly know that it only needs to read colums that are part of the key. (The handler can also use the column map for detecting this, but simpler/faster handler can just monitor the extra() call). - table->mark_columns_used_by_index_no_reset() to in addition to other columns, also mark all columns that is used by the given key. - table->restore_column_maps_after_mark_index() to restore to default column maps after a call to table->mark_columns_used_by_index(). - New item function register_field_in_read_map(), for marking used columns in table->read_map. Used by filesort() to mark all used columns - Maintain in TABLE->merge_keys set of all keys that are used in query. (Simplices some optimization loops) - Maintain Field->part_of_key_not_clustered which is like Field->part_of_key but the field in the clustered key is not assumed to be part of all index. (used in opt_range.cc for faster loops) - dbug_tmp_use_all_columns(), dbug_tmp_restore_column_map() tmp_use_all_columns() and tmp_restore_column_map() functions to temporally mark all columns as usable. The 'dbug_' version is primarily intended inside a handler when it wants to just call Field:store() & Field::val() functions, but don't need the column maps set for any other usage. (ie:: bitmap_is_set() is never called) - We can't use compare_records() to skip updates for handlers that returns a partial column set and the read_set doesn't cover all columns in the write set. The reason for this is that if we have a column marked only for write we can't in the MySQL level know if the value changed or not. The reason this worked before was that MySQL marked all to be written columns as also to be read. The new 'optimal' bitmaps exposed this 'hidden bug'. - open_table_from_share() does not anymore setup temporary MEM_ROOT object as a thread specific variable for the handler. Instead we send the to-be-used MEMROOT to get_new_handler(). (Simpler, faster code) Bugs fixed: - Column marking was not done correctly in a lot of cases. (ALTER TABLE, when using triggers, auto_increment fields etc) (Could potentially result in wrong values inserted in table handlers relying on that the old column maps or field->set_query_id was correct) Especially when it comes to triggers, there may be cases where the old code would cause lost/wrong values for NDB and/or InnoDB tables. - Split thd->options flag OPTION_STATUS_NO_TRANS_UPDATE to two flags: OPTION_STATUS_NO_TRANS_UPDATE and OPTION_KEEP_LOG. This allowed me to remove some wrong warnings about: "Some non-transactional changed tables couldn't be rolled back" - Fixed handling of INSERT .. SELECT and CREATE ... SELECT that wrongly reset (thd->options & OPTION_STATUS_NO_TRANS_UPDATE) which caused us to loose some warnings about "Some non-transactional changed tables couldn't be rolled back") - Fixed use of uninitialized memory in ha_ndbcluster.cc::delete_table() which could cause delete_table to report random failures. - Fixed core dumps for some tests when running with --debug - Added missing FN_LIBCHAR in mysql_rm_tmp_tables() (This has probably caused us to not properly remove temporary files after crash) - slow_logs was not properly initialized, which could maybe cause extra/lost entries in slow log. - If we get an duplicate row on insert, change column map to read and write all columns while retrying the operation. This is required by the definition of REPLACE and also ensures that fields that are only part of UPDATE are properly handled. This fixed a bug in NDB and REPLACE where REPLACE wrongly copied some column values from the replaced row. - For table handler that doesn't support NULL in keys, we would give an error when creating a primary key with NULL fields, even after the fields has been automaticly converted to NOT NULL. - Creating a primary key on a SPATIAL key, would fail if field was not declared as NOT NULL. Cleanups: - Removed not used condition argument to setup_tables - Removed not needed item function reset_query_id_processor(). - Field->add_index is removed. Now this is instead maintained in (field->flags & FIELD_IN_ADD_INDEX) - Field->fieldnr is removed (use field->field_index instead) - New argument to filesort() to indicate that it should return a set of row pointers (not used columns). This allowed me to remove some references to sql_command in filesort and should also enable us to return column results in some cases where we couldn't before. - Changed column bitmap handling in opt_range.cc to be aligned with TABLE bitmap, which allowed me to use bitmap functions instead of looping over all fields to create some needed bitmaps. (Faster and smaller code) - Broke up found too long lines - Moved some variable declaration at start of function for better code readability. - Removed some not used arguments from functions. (setup_fields(), mysql_prepare_insert_check_table()) - setup_fields() now takes an enum instead of an int for marking columns usage. - For internal temporary tables, use handler::write_row(), handler::delete_row() and handler::update_row() instead of handler::ha_xxxx() for faster execution. - Changed some constants to enum's and define's. - Using separate column read and write sets allows for easier checking of timestamp field was set by statement. - Remove calls to free_io_cache() as this is now done automaticly in ha_reset() - Don't build table->normalized_path as this is now identical to table->path (after bar's fixes to convert filenames) - Fixed some missed DBUG_PRINT(.."%lx") to use "0x%lx" to make it easier to do comparision with the 'convert-dbug-for-diff' tool. Things left to do in 5.1: - We wrongly log failed CREATE TABLE ... SELECT in some cases when using row based logging (as shown by testcase binlog_row_mix_innodb_myisam.result) Mats has promised to look into this. - Test that my fix for CREATE TABLE ... SELECT is indeed correct. (I added several test cases for this, but in this case it's better that someone else also tests this throughly). Lars has promosed to do this.
20 years ago
Implement new type-of-operation-aware metadata locks. Add a wait-for graph based deadlock detector to the MDL subsystem. Fixes bug #46272 "MySQL 5.4.4, new MDL: unnecessary deadlock" and bug #37346 "innodb does not detect deadlock between update and alter table". The first bug manifested itself as an unwarranted abort of a transaction with ER_LOCK_DEADLOCK error by a concurrent ALTER statement, when this transaction tried to repeat use of a table, which it has already used in a similar fashion before ALTER started. The second bug showed up as a deadlock between table-level locks and InnoDB row locks, which was "detected" only after innodb_lock_wait_timeout timeout. A transaction would start using the table and modify a few rows. Then ALTER TABLE would come in, and start copying rows into a temporary table. Eventually it would stumble on the modified records and get blocked on a row lock. The first transaction would try to do more updates, and get blocked on thr_lock.c lock. This situation of circular wait would only get resolved by a timeout. Both these bugs stemmed from inadequate solutions to the problem of deadlocks occurring between different locking subsystems. In the first case we tried to avoid deadlocks between metadata locking and table-level locking subsystems, when upgrading shared metadata lock to exclusive one. Transactions holding the shared lock on the table and waiting for some table-level lock used to be aborted too aggressively. We also allowed ALTER TABLE to start in presence of transactions that modify the subject table. ALTER TABLE acquires TL_WRITE_ALLOW_READ lock at start, and that block all writes against the table (naturally, we don't want any writes to be lost when switching the old and the new table). TL_WRITE_ALLOW_READ lock, in turn, would block the started transaction on thr_lock.c lock, should they do more updates. This, again, lead to the need to abort such transactions. The second bug occurred simply because we didn't have any mechanism to detect deadlocks between the table-level locks in thr_lock.c and row-level locks in InnoDB, other than innodb_lock_wait_timeout. This patch solves both these problems by moving lock conflicts which are causing these deadlocks into the metadata locking subsystem, thus making it possible to avoid or detect such deadlocks inside MDL. To do this we introduce new type-of-operation-aware metadata locks, which allow MDL subsystem to know not only the fact that transaction has used or is going to use some object but also what kind of operation it has carried out or going to carry out on the object. This, along with the addition of a special kind of upgradable metadata lock, allows ALTER TABLE to wait until all transactions which has updated the table to go away. This solves the second issue. Another special type of upgradable metadata lock is acquired by LOCK TABLE WRITE. This second lock type allows to solve the first issue, since abortion of table-level locks in event of DDL under LOCK TABLES becomes also unnecessary. Below follows the list of incompatible changes introduced by this patch: - From now on, ALTER TABLE and CREATE/DROP TRIGGER SQL (i.e. those statements that acquire TL_WRITE_ALLOW_READ lock) wait for all transactions which has *updated* the table to complete. - From now on, LOCK TABLES ... WRITE, REPAIR/OPTIMIZE TABLE (i.e. all statements which acquire TL_WRITE table-level lock) wait for all transaction which *updated or read* from the table to complete. As a consequence, innodb_table_locks=0 option no longer applies to LOCK TABLES ... WRITE. - DROP DATABASE, DROP TABLE, RENAME TABLE no longer abort statements or transactions which use tables being dropped or renamed, and instead wait for these transactions to complete. - Since LOCK TABLES WRITE now takes a special metadata lock, not compatible with with reads or writes against the subject table and transaction-wide, thr_lock.c deadlock avoidance algorithm that used to ensure absence of deadlocks between LOCK TABLES WRITE and other statements is no longer sufficient, even for MyISAM. The wait-for graph based deadlock detector of MDL subsystem may sometimes be necessary and is involved. This may lead to ER_LOCK_DEADLOCK error produced for multi-statement transactions even if these only use MyISAM: session 1: session 2: begin; update t1 ... lock table t2 write, t1 write; -- gets a lock on t2, blocks on t1 update t2 ... (ER_LOCK_DEADLOCK) - Finally, support of LOW_PRIORITY option for LOCK TABLES ... WRITE was abandoned. LOCK TABLE ... LOW_PRIORITY WRITE from now on has the same priority as the usual LOCK TABLE ... WRITE. SELECT HIGH PRIORITY no longer trumps LOCK TABLE ... WRITE in the wait queue. - We do not take upgradable metadata locks on implicitly locked tables. So if one has, say, a view v1 that uses table t1, and issues: LOCK TABLE v1 WRITE; FLUSH TABLE t1; -- (or just 'FLUSH TABLES'), an error is produced. In order to be able to perform DDL on a table under LOCK TABLES, the table must be locked explicitly in the LOCK TABLES list.
16 years ago
Implement new type-of-operation-aware metadata locks. Add a wait-for graph based deadlock detector to the MDL subsystem. Fixes bug #46272 "MySQL 5.4.4, new MDL: unnecessary deadlock" and bug #37346 "innodb does not detect deadlock between update and alter table". The first bug manifested itself as an unwarranted abort of a transaction with ER_LOCK_DEADLOCK error by a concurrent ALTER statement, when this transaction tried to repeat use of a table, which it has already used in a similar fashion before ALTER started. The second bug showed up as a deadlock between table-level locks and InnoDB row locks, which was "detected" only after innodb_lock_wait_timeout timeout. A transaction would start using the table and modify a few rows. Then ALTER TABLE would come in, and start copying rows into a temporary table. Eventually it would stumble on the modified records and get blocked on a row lock. The first transaction would try to do more updates, and get blocked on thr_lock.c lock. This situation of circular wait would only get resolved by a timeout. Both these bugs stemmed from inadequate solutions to the problem of deadlocks occurring between different locking subsystems. In the first case we tried to avoid deadlocks between metadata locking and table-level locking subsystems, when upgrading shared metadata lock to exclusive one. Transactions holding the shared lock on the table and waiting for some table-level lock used to be aborted too aggressively. We also allowed ALTER TABLE to start in presence of transactions that modify the subject table. ALTER TABLE acquires TL_WRITE_ALLOW_READ lock at start, and that block all writes against the table (naturally, we don't want any writes to be lost when switching the old and the new table). TL_WRITE_ALLOW_READ lock, in turn, would block the started transaction on thr_lock.c lock, should they do more updates. This, again, lead to the need to abort such transactions. The second bug occurred simply because we didn't have any mechanism to detect deadlocks between the table-level locks in thr_lock.c and row-level locks in InnoDB, other than innodb_lock_wait_timeout. This patch solves both these problems by moving lock conflicts which are causing these deadlocks into the metadata locking subsystem, thus making it possible to avoid or detect such deadlocks inside MDL. To do this we introduce new type-of-operation-aware metadata locks, which allow MDL subsystem to know not only the fact that transaction has used or is going to use some object but also what kind of operation it has carried out or going to carry out on the object. This, along with the addition of a special kind of upgradable metadata lock, allows ALTER TABLE to wait until all transactions which has updated the table to go away. This solves the second issue. Another special type of upgradable metadata lock is acquired by LOCK TABLE WRITE. This second lock type allows to solve the first issue, since abortion of table-level locks in event of DDL under LOCK TABLES becomes also unnecessary. Below follows the list of incompatible changes introduced by this patch: - From now on, ALTER TABLE and CREATE/DROP TRIGGER SQL (i.e. those statements that acquire TL_WRITE_ALLOW_READ lock) wait for all transactions which has *updated* the table to complete. - From now on, LOCK TABLES ... WRITE, REPAIR/OPTIMIZE TABLE (i.e. all statements which acquire TL_WRITE table-level lock) wait for all transaction which *updated or read* from the table to complete. As a consequence, innodb_table_locks=0 option no longer applies to LOCK TABLES ... WRITE. - DROP DATABASE, DROP TABLE, RENAME TABLE no longer abort statements or transactions which use tables being dropped or renamed, and instead wait for these transactions to complete. - Since LOCK TABLES WRITE now takes a special metadata lock, not compatible with with reads or writes against the subject table and transaction-wide, thr_lock.c deadlock avoidance algorithm that used to ensure absence of deadlocks between LOCK TABLES WRITE and other statements is no longer sufficient, even for MyISAM. The wait-for graph based deadlock detector of MDL subsystem may sometimes be necessary and is involved. This may lead to ER_LOCK_DEADLOCK error produced for multi-statement transactions even if these only use MyISAM: session 1: session 2: begin; update t1 ... lock table t2 write, t1 write; -- gets a lock on t2, blocks on t1 update t2 ... (ER_LOCK_DEADLOCK) - Finally, support of LOW_PRIORITY option for LOCK TABLES ... WRITE was abandoned. LOCK TABLE ... LOW_PRIORITY WRITE from now on has the same priority as the usual LOCK TABLE ... WRITE. SELECT HIGH PRIORITY no longer trumps LOCK TABLE ... WRITE in the wait queue. - We do not take upgradable metadata locks on implicitly locked tables. So if one has, say, a view v1 that uses table t1, and issues: LOCK TABLE v1 WRITE; FLUSH TABLE t1; -- (or just 'FLUSH TABLES'), an error is produced. In order to be able to perform DDL on a table under LOCK TABLES, the table must be locked explicitly in the LOCK TABLES list.
16 years ago
Implement new type-of-operation-aware metadata locks. Add a wait-for graph based deadlock detector to the MDL subsystem. Fixes bug #46272 "MySQL 5.4.4, new MDL: unnecessary deadlock" and bug #37346 "innodb does not detect deadlock between update and alter table". The first bug manifested itself as an unwarranted abort of a transaction with ER_LOCK_DEADLOCK error by a concurrent ALTER statement, when this transaction tried to repeat use of a table, which it has already used in a similar fashion before ALTER started. The second bug showed up as a deadlock between table-level locks and InnoDB row locks, which was "detected" only after innodb_lock_wait_timeout timeout. A transaction would start using the table and modify a few rows. Then ALTER TABLE would come in, and start copying rows into a temporary table. Eventually it would stumble on the modified records and get blocked on a row lock. The first transaction would try to do more updates, and get blocked on thr_lock.c lock. This situation of circular wait would only get resolved by a timeout. Both these bugs stemmed from inadequate solutions to the problem of deadlocks occurring between different locking subsystems. In the first case we tried to avoid deadlocks between metadata locking and table-level locking subsystems, when upgrading shared metadata lock to exclusive one. Transactions holding the shared lock on the table and waiting for some table-level lock used to be aborted too aggressively. We also allowed ALTER TABLE to start in presence of transactions that modify the subject table. ALTER TABLE acquires TL_WRITE_ALLOW_READ lock at start, and that block all writes against the table (naturally, we don't want any writes to be lost when switching the old and the new table). TL_WRITE_ALLOW_READ lock, in turn, would block the started transaction on thr_lock.c lock, should they do more updates. This, again, lead to the need to abort such transactions. The second bug occurred simply because we didn't have any mechanism to detect deadlocks between the table-level locks in thr_lock.c and row-level locks in InnoDB, other than innodb_lock_wait_timeout. This patch solves both these problems by moving lock conflicts which are causing these deadlocks into the metadata locking subsystem, thus making it possible to avoid or detect such deadlocks inside MDL. To do this we introduce new type-of-operation-aware metadata locks, which allow MDL subsystem to know not only the fact that transaction has used or is going to use some object but also what kind of operation it has carried out or going to carry out on the object. This, along with the addition of a special kind of upgradable metadata lock, allows ALTER TABLE to wait until all transactions which has updated the table to go away. This solves the second issue. Another special type of upgradable metadata lock is acquired by LOCK TABLE WRITE. This second lock type allows to solve the first issue, since abortion of table-level locks in event of DDL under LOCK TABLES becomes also unnecessary. Below follows the list of incompatible changes introduced by this patch: - From now on, ALTER TABLE and CREATE/DROP TRIGGER SQL (i.e. those statements that acquire TL_WRITE_ALLOW_READ lock) wait for all transactions which has *updated* the table to complete. - From now on, LOCK TABLES ... WRITE, REPAIR/OPTIMIZE TABLE (i.e. all statements which acquire TL_WRITE table-level lock) wait for all transaction which *updated or read* from the table to complete. As a consequence, innodb_table_locks=0 option no longer applies to LOCK TABLES ... WRITE. - DROP DATABASE, DROP TABLE, RENAME TABLE no longer abort statements or transactions which use tables being dropped or renamed, and instead wait for these transactions to complete. - Since LOCK TABLES WRITE now takes a special metadata lock, not compatible with with reads or writes against the subject table and transaction-wide, thr_lock.c deadlock avoidance algorithm that used to ensure absence of deadlocks between LOCK TABLES WRITE and other statements is no longer sufficient, even for MyISAM. The wait-for graph based deadlock detector of MDL subsystem may sometimes be necessary and is involved. This may lead to ER_LOCK_DEADLOCK error produced for multi-statement transactions even if these only use MyISAM: session 1: session 2: begin; update t1 ... lock table t2 write, t1 write; -- gets a lock on t2, blocks on t1 update t2 ... (ER_LOCK_DEADLOCK) - Finally, support of LOW_PRIORITY option for LOCK TABLES ... WRITE was abandoned. LOCK TABLE ... LOW_PRIORITY WRITE from now on has the same priority as the usual LOCK TABLE ... WRITE. SELECT HIGH PRIORITY no longer trumps LOCK TABLE ... WRITE in the wait queue. - We do not take upgradable metadata locks on implicitly locked tables. So if one has, say, a view v1 that uses table t1, and issues: LOCK TABLE v1 WRITE; FLUSH TABLE t1; -- (or just 'FLUSH TABLES'), an error is produced. In order to be able to perform DDL on a table under LOCK TABLES, the table must be locked explicitly in the LOCK TABLES list.
16 years ago
22 years ago
Fix for: Bug #20662 "Infinite loop in CREATE TABLE IF NOT EXISTS ... SELECT with locked tables" Bug #20903 "Crash when using CREATE TABLE .. SELECT and triggers" Bug #24738 "CREATE TABLE ... SELECT is not isolated properly" Bug #24508 "Inconsistent results of CREATE TABLE ... SELECT when temporary table exists" Deadlock occured when one tried to execute CREATE TABLE IF NOT EXISTS ... SELECT statement under LOCK TABLES which held read lock on target table. Attempt to execute the same statement for already existing target table with triggers caused server crashes. Also concurrent execution of CREATE TABLE ... SELECT statement and other statements involving target table suffered from various races (some of which might've led to deadlocks). Finally, attempt to execute CREATE TABLE ... SELECT in case when a temporary table with same name was already present led to the insertion of data into this temporary table and creation of empty non-temporary table. All above problems stemmed from the old implementation of CREATE TABLE ... SELECT in which we created, opened and locked target table without any special protection in a separate step and not with the rest of tables used by this statement. This underminded deadlock-avoidance approach used in server and created window for races. It also excluded target table from prelocking causing problems with trigger execution. The patch solves these problems by implementing new approach to handling of CREATE TABLE ... SELECT for base tables. We try to open and lock table to be created at the same time as the rest of tables used by this statement. If such table does not exist at this moment we create and place in the table cache special placeholder for it which prevents its creation or any other usage by other threads. We still use old approach for creation of temporary tables. Note that we have separate fix for 5.0 since there we use slightly different less intrusive approach.
19 years ago
Fix for: Bug #20662 "Infinite loop in CREATE TABLE IF NOT EXISTS ... SELECT with locked tables" Bug #20903 "Crash when using CREATE TABLE .. SELECT and triggers" Bug #24738 "CREATE TABLE ... SELECT is not isolated properly" Bug #24508 "Inconsistent results of CREATE TABLE ... SELECT when temporary table exists" Deadlock occured when one tried to execute CREATE TABLE IF NOT EXISTS ... SELECT statement under LOCK TABLES which held read lock on target table. Attempt to execute the same statement for already existing target table with triggers caused server crashes. Also concurrent execution of CREATE TABLE ... SELECT statement and other statements involving target table suffered from various races (some of which might've led to deadlocks). Finally, attempt to execute CREATE TABLE ... SELECT in case when a temporary table with same name was already present led to the insertion of data into this temporary table and creation of empty non-temporary table. All above problems stemmed from the old implementation of CREATE TABLE ... SELECT in which we created, opened and locked target table without any special protection in a separate step and not with the rest of tables used by this statement. This underminded deadlock-avoidance approach used in server and created window for races. It also excluded target table from prelocking causing problems with trigger execution. The patch solves these problems by implementing new approach to handling of CREATE TABLE ... SELECT for base tables. We try to open and lock table to be created at the same time as the rest of tables used by this statement. If such table does not exist at this moment we create and place in the table cache special placeholder for it which prevents its creation or any other usage by other threads. We still use old approach for creation of temporary tables. Note that we have separate fix for 5.0 since there we use slightly different less intrusive approach.
19 years ago
Implement new type-of-operation-aware metadata locks. Add a wait-for graph based deadlock detector to the MDL subsystem. Fixes bug #46272 "MySQL 5.4.4, new MDL: unnecessary deadlock" and bug #37346 "innodb does not detect deadlock between update and alter table". The first bug manifested itself as an unwarranted abort of a transaction with ER_LOCK_DEADLOCK error by a concurrent ALTER statement, when this transaction tried to repeat use of a table, which it has already used in a similar fashion before ALTER started. The second bug showed up as a deadlock between table-level locks and InnoDB row locks, which was "detected" only after innodb_lock_wait_timeout timeout. A transaction would start using the table and modify a few rows. Then ALTER TABLE would come in, and start copying rows into a temporary table. Eventually it would stumble on the modified records and get blocked on a row lock. The first transaction would try to do more updates, and get blocked on thr_lock.c lock. This situation of circular wait would only get resolved by a timeout. Both these bugs stemmed from inadequate solutions to the problem of deadlocks occurring between different locking subsystems. In the first case we tried to avoid deadlocks between metadata locking and table-level locking subsystems, when upgrading shared metadata lock to exclusive one. Transactions holding the shared lock on the table and waiting for some table-level lock used to be aborted too aggressively. We also allowed ALTER TABLE to start in presence of transactions that modify the subject table. ALTER TABLE acquires TL_WRITE_ALLOW_READ lock at start, and that block all writes against the table (naturally, we don't want any writes to be lost when switching the old and the new table). TL_WRITE_ALLOW_READ lock, in turn, would block the started transaction on thr_lock.c lock, should they do more updates. This, again, lead to the need to abort such transactions. The second bug occurred simply because we didn't have any mechanism to detect deadlocks between the table-level locks in thr_lock.c and row-level locks in InnoDB, other than innodb_lock_wait_timeout. This patch solves both these problems by moving lock conflicts which are causing these deadlocks into the metadata locking subsystem, thus making it possible to avoid or detect such deadlocks inside MDL. To do this we introduce new type-of-operation-aware metadata locks, which allow MDL subsystem to know not only the fact that transaction has used or is going to use some object but also what kind of operation it has carried out or going to carry out on the object. This, along with the addition of a special kind of upgradable metadata lock, allows ALTER TABLE to wait until all transactions which has updated the table to go away. This solves the second issue. Another special type of upgradable metadata lock is acquired by LOCK TABLE WRITE. This second lock type allows to solve the first issue, since abortion of table-level locks in event of DDL under LOCK TABLES becomes also unnecessary. Below follows the list of incompatible changes introduced by this patch: - From now on, ALTER TABLE and CREATE/DROP TRIGGER SQL (i.e. those statements that acquire TL_WRITE_ALLOW_READ lock) wait for all transactions which has *updated* the table to complete. - From now on, LOCK TABLES ... WRITE, REPAIR/OPTIMIZE TABLE (i.e. all statements which acquire TL_WRITE table-level lock) wait for all transaction which *updated or read* from the table to complete. As a consequence, innodb_table_locks=0 option no longer applies to LOCK TABLES ... WRITE. - DROP DATABASE, DROP TABLE, RENAME TABLE no longer abort statements or transactions which use tables being dropped or renamed, and instead wait for these transactions to complete. - Since LOCK TABLES WRITE now takes a special metadata lock, not compatible with with reads or writes against the subject table and transaction-wide, thr_lock.c deadlock avoidance algorithm that used to ensure absence of deadlocks between LOCK TABLES WRITE and other statements is no longer sufficient, even for MyISAM. The wait-for graph based deadlock detector of MDL subsystem may sometimes be necessary and is involved. This may lead to ER_LOCK_DEADLOCK error produced for multi-statement transactions even if these only use MyISAM: session 1: session 2: begin; update t1 ... lock table t2 write, t1 write; -- gets a lock on t2, blocks on t1 update t2 ... (ER_LOCK_DEADLOCK) - Finally, support of LOW_PRIORITY option for LOCK TABLES ... WRITE was abandoned. LOCK TABLE ... LOW_PRIORITY WRITE from now on has the same priority as the usual LOCK TABLE ... WRITE. SELECT HIGH PRIORITY no longer trumps LOCK TABLE ... WRITE in the wait queue. - We do not take upgradable metadata locks on implicitly locked tables. So if one has, say, a view v1 that uses table t1, and issues: LOCK TABLE v1 WRITE; FLUSH TABLE t1; -- (or just 'FLUSH TABLES'), an error is produced. In order to be able to perform DDL on a table under LOCK TABLES, the table must be locked explicitly in the LOCK TABLES list.
16 years ago
Fix for: Bug #20662 "Infinite loop in CREATE TABLE IF NOT EXISTS ... SELECT with locked tables" Bug #20903 "Crash when using CREATE TABLE .. SELECT and triggers" Bug #24738 "CREATE TABLE ... SELECT is not isolated properly" Bug #24508 "Inconsistent results of CREATE TABLE ... SELECT when temporary table exists" Deadlock occured when one tried to execute CREATE TABLE IF NOT EXISTS ... SELECT statement under LOCK TABLES which held read lock on target table. Attempt to execute the same statement for already existing target table with triggers caused server crashes. Also concurrent execution of CREATE TABLE ... SELECT statement and other statements involving target table suffered from various races (some of which might've led to deadlocks). Finally, attempt to execute CREATE TABLE ... SELECT in case when a temporary table with same name was already present led to the insertion of data into this temporary table and creation of empty non-temporary table. All above problems stemmed from the old implementation of CREATE TABLE ... SELECT in which we created, opened and locked target table without any special protection in a separate step and not with the rest of tables used by this statement. This underminded deadlock-avoidance approach used in server and created window for races. It also excluded target table from prelocking causing problems with trigger execution. The patch solves these problems by implementing new approach to handling of CREATE TABLE ... SELECT for base tables. We try to open and lock table to be created at the same time as the rest of tables used by this statement. If such table does not exist at this moment we create and place in the table cache special placeholder for it which prevents its creation or any other usage by other threads. We still use old approach for creation of temporary tables. Note that we have separate fix for 5.0 since there we use slightly different less intrusive approach.
19 years ago
Fix for: Bug #20662 "Infinite loop in CREATE TABLE IF NOT EXISTS ... SELECT with locked tables" Bug #20903 "Crash when using CREATE TABLE .. SELECT and triggers" Bug #24738 "CREATE TABLE ... SELECT is not isolated properly" Bug #24508 "Inconsistent results of CREATE TABLE ... SELECT when temporary table exists" Deadlock occured when one tried to execute CREATE TABLE IF NOT EXISTS ... SELECT statement under LOCK TABLES which held read lock on target table. Attempt to execute the same statement for already existing target table with triggers caused server crashes. Also concurrent execution of CREATE TABLE ... SELECT statement and other statements involving target table suffered from various races (some of which might've led to deadlocks). Finally, attempt to execute CREATE TABLE ... SELECT in case when a temporary table with same name was already present led to the insertion of data into this temporary table and creation of empty non-temporary table. All above problems stemmed from the old implementation of CREATE TABLE ... SELECT in which we created, opened and locked target table without any special protection in a separate step and not with the rest of tables used by this statement. This underminded deadlock-avoidance approach used in server and created window for races. It also excluded target table from prelocking causing problems with trigger execution. The patch solves these problems by implementing new approach to handling of CREATE TABLE ... SELECT for base tables. We try to open and lock table to be created at the same time as the rest of tables used by this statement. If such table does not exist at this moment we create and place in the table cache special placeholder for it which prevents its creation or any other usage by other threads. We still use old approach for creation of temporary tables. Note that we have separate fix for 5.0 since there we use slightly different less intrusive approach.
19 years ago
Fix for: Bug #20662 "Infinite loop in CREATE TABLE IF NOT EXISTS ... SELECT with locked tables" Bug #20903 "Crash when using CREATE TABLE .. SELECT and triggers" Bug #24738 "CREATE TABLE ... SELECT is not isolated properly" Bug #24508 "Inconsistent results of CREATE TABLE ... SELECT when temporary table exists" Deadlock occured when one tried to execute CREATE TABLE IF NOT EXISTS ... SELECT statement under LOCK TABLES which held read lock on target table. Attempt to execute the same statement for already existing target table with triggers caused server crashes. Also concurrent execution of CREATE TABLE ... SELECT statement and other statements involving target table suffered from various races (some of which might've led to deadlocks). Finally, attempt to execute CREATE TABLE ... SELECT in case when a temporary table with same name was already present led to the insertion of data into this temporary table and creation of empty non-temporary table. All above problems stemmed from the old implementation of CREATE TABLE ... SELECT in which we created, opened and locked target table without any special protection in a separate step and not with the rest of tables used by this statement. This underminded deadlock-avoidance approach used in server and created window for races. It also excluded target table from prelocking causing problems with trigger execution. The patch solves these problems by implementing new approach to handling of CREATE TABLE ... SELECT for base tables. We try to open and lock table to be created at the same time as the rest of tables used by this statement. If such table does not exist at this moment we create and place in the table cache special placeholder for it which prevents its creation or any other usage by other threads. We still use old approach for creation of temporary tables. Note that we have separate fix for 5.0 since there we use slightly different less intrusive approach.
19 years ago
Implement new type-of-operation-aware metadata locks. Add a wait-for graph based deadlock detector to the MDL subsystem. Fixes bug #46272 "MySQL 5.4.4, new MDL: unnecessary deadlock" and bug #37346 "innodb does not detect deadlock between update and alter table". The first bug manifested itself as an unwarranted abort of a transaction with ER_LOCK_DEADLOCK error by a concurrent ALTER statement, when this transaction tried to repeat use of a table, which it has already used in a similar fashion before ALTER started. The second bug showed up as a deadlock between table-level locks and InnoDB row locks, which was "detected" only after innodb_lock_wait_timeout timeout. A transaction would start using the table and modify a few rows. Then ALTER TABLE would come in, and start copying rows into a temporary table. Eventually it would stumble on the modified records and get blocked on a row lock. The first transaction would try to do more updates, and get blocked on thr_lock.c lock. This situation of circular wait would only get resolved by a timeout. Both these bugs stemmed from inadequate solutions to the problem of deadlocks occurring between different locking subsystems. In the first case we tried to avoid deadlocks between metadata locking and table-level locking subsystems, when upgrading shared metadata lock to exclusive one. Transactions holding the shared lock on the table and waiting for some table-level lock used to be aborted too aggressively. We also allowed ALTER TABLE to start in presence of transactions that modify the subject table. ALTER TABLE acquires TL_WRITE_ALLOW_READ lock at start, and that block all writes against the table (naturally, we don't want any writes to be lost when switching the old and the new table). TL_WRITE_ALLOW_READ lock, in turn, would block the started transaction on thr_lock.c lock, should they do more updates. This, again, lead to the need to abort such transactions. The second bug occurred simply because we didn't have any mechanism to detect deadlocks between the table-level locks in thr_lock.c and row-level locks in InnoDB, other than innodb_lock_wait_timeout. This patch solves both these problems by moving lock conflicts which are causing these deadlocks into the metadata locking subsystem, thus making it possible to avoid or detect such deadlocks inside MDL. To do this we introduce new type-of-operation-aware metadata locks, which allow MDL subsystem to know not only the fact that transaction has used or is going to use some object but also what kind of operation it has carried out or going to carry out on the object. This, along with the addition of a special kind of upgradable metadata lock, allows ALTER TABLE to wait until all transactions which has updated the table to go away. This solves the second issue. Another special type of upgradable metadata lock is acquired by LOCK TABLE WRITE. This second lock type allows to solve the first issue, since abortion of table-level locks in event of DDL under LOCK TABLES becomes also unnecessary. Below follows the list of incompatible changes introduced by this patch: - From now on, ALTER TABLE and CREATE/DROP TRIGGER SQL (i.e. those statements that acquire TL_WRITE_ALLOW_READ lock) wait for all transactions which has *updated* the table to complete. - From now on, LOCK TABLES ... WRITE, REPAIR/OPTIMIZE TABLE (i.e. all statements which acquire TL_WRITE table-level lock) wait for all transaction which *updated or read* from the table to complete. As a consequence, innodb_table_locks=0 option no longer applies to LOCK TABLES ... WRITE. - DROP DATABASE, DROP TABLE, RENAME TABLE no longer abort statements or transactions which use tables being dropped or renamed, and instead wait for these transactions to complete. - Since LOCK TABLES WRITE now takes a special metadata lock, not compatible with with reads or writes against the subject table and transaction-wide, thr_lock.c deadlock avoidance algorithm that used to ensure absence of deadlocks between LOCK TABLES WRITE and other statements is no longer sufficient, even for MyISAM. The wait-for graph based deadlock detector of MDL subsystem may sometimes be necessary and is involved. This may lead to ER_LOCK_DEADLOCK error produced for multi-statement transactions even if these only use MyISAM: session 1: session 2: begin; update t1 ... lock table t2 write, t1 write; -- gets a lock on t2, blocks on t1 update t2 ... (ER_LOCK_DEADLOCK) - Finally, support of LOW_PRIORITY option for LOCK TABLES ... WRITE was abandoned. LOCK TABLE ... LOW_PRIORITY WRITE from now on has the same priority as the usual LOCK TABLE ... WRITE. SELECT HIGH PRIORITY no longer trumps LOCK TABLE ... WRITE in the wait queue. - We do not take upgradable metadata locks on implicitly locked tables. So if one has, say, a view v1 that uses table t1, and issues: LOCK TABLE v1 WRITE; FLUSH TABLE t1; -- (or just 'FLUSH TABLES'), an error is produced. In order to be able to perform DDL on a table under LOCK TABLES, the table must be locked explicitly in the LOCK TABLES list.
16 years ago
Implement new type-of-operation-aware metadata locks. Add a wait-for graph based deadlock detector to the MDL subsystem. Fixes bug #46272 "MySQL 5.4.4, new MDL: unnecessary deadlock" and bug #37346 "innodb does not detect deadlock between update and alter table". The first bug manifested itself as an unwarranted abort of a transaction with ER_LOCK_DEADLOCK error by a concurrent ALTER statement, when this transaction tried to repeat use of a table, which it has already used in a similar fashion before ALTER started. The second bug showed up as a deadlock between table-level locks and InnoDB row locks, which was "detected" only after innodb_lock_wait_timeout timeout. A transaction would start using the table and modify a few rows. Then ALTER TABLE would come in, and start copying rows into a temporary table. Eventually it would stumble on the modified records and get blocked on a row lock. The first transaction would try to do more updates, and get blocked on thr_lock.c lock. This situation of circular wait would only get resolved by a timeout. Both these bugs stemmed from inadequate solutions to the problem of deadlocks occurring between different locking subsystems. In the first case we tried to avoid deadlocks between metadata locking and table-level locking subsystems, when upgrading shared metadata lock to exclusive one. Transactions holding the shared lock on the table and waiting for some table-level lock used to be aborted too aggressively. We also allowed ALTER TABLE to start in presence of transactions that modify the subject table. ALTER TABLE acquires TL_WRITE_ALLOW_READ lock at start, and that block all writes against the table (naturally, we don't want any writes to be lost when switching the old and the new table). TL_WRITE_ALLOW_READ lock, in turn, would block the started transaction on thr_lock.c lock, should they do more updates. This, again, lead to the need to abort such transactions. The second bug occurred simply because we didn't have any mechanism to detect deadlocks between the table-level locks in thr_lock.c and row-level locks in InnoDB, other than innodb_lock_wait_timeout. This patch solves both these problems by moving lock conflicts which are causing these deadlocks into the metadata locking subsystem, thus making it possible to avoid or detect such deadlocks inside MDL. To do this we introduce new type-of-operation-aware metadata locks, which allow MDL subsystem to know not only the fact that transaction has used or is going to use some object but also what kind of operation it has carried out or going to carry out on the object. This, along with the addition of a special kind of upgradable metadata lock, allows ALTER TABLE to wait until all transactions which has updated the table to go away. This solves the second issue. Another special type of upgradable metadata lock is acquired by LOCK TABLE WRITE. This second lock type allows to solve the first issue, since abortion of table-level locks in event of DDL under LOCK TABLES becomes also unnecessary. Below follows the list of incompatible changes introduced by this patch: - From now on, ALTER TABLE and CREATE/DROP TRIGGER SQL (i.e. those statements that acquire TL_WRITE_ALLOW_READ lock) wait for all transactions which has *updated* the table to complete. - From now on, LOCK TABLES ... WRITE, REPAIR/OPTIMIZE TABLE (i.e. all statements which acquire TL_WRITE table-level lock) wait for all transaction which *updated or read* from the table to complete. As a consequence, innodb_table_locks=0 option no longer applies to LOCK TABLES ... WRITE. - DROP DATABASE, DROP TABLE, RENAME TABLE no longer abort statements or transactions which use tables being dropped or renamed, and instead wait for these transactions to complete. - Since LOCK TABLES WRITE now takes a special metadata lock, not compatible with with reads or writes against the subject table and transaction-wide, thr_lock.c deadlock avoidance algorithm that used to ensure absence of deadlocks between LOCK TABLES WRITE and other statements is no longer sufficient, even for MyISAM. The wait-for graph based deadlock detector of MDL subsystem may sometimes be necessary and is involved. This may lead to ER_LOCK_DEADLOCK error produced for multi-statement transactions even if these only use MyISAM: session 1: session 2: begin; update t1 ... lock table t2 write, t1 write; -- gets a lock on t2, blocks on t1 update t2 ... (ER_LOCK_DEADLOCK) - Finally, support of LOW_PRIORITY option for LOCK TABLES ... WRITE was abandoned. LOCK TABLE ... LOW_PRIORITY WRITE from now on has the same priority as the usual LOCK TABLE ... WRITE. SELECT HIGH PRIORITY no longer trumps LOCK TABLE ... WRITE in the wait queue. - We do not take upgradable metadata locks on implicitly locked tables. So if one has, say, a view v1 that uses table t1, and issues: LOCK TABLE v1 WRITE; FLUSH TABLE t1; -- (or just 'FLUSH TABLES'), an error is produced. In order to be able to perform DDL on a table under LOCK TABLES, the table must be locked explicitly in the LOCK TABLES list.
16 years ago
21 years ago
21 years ago
21 years ago
21 years ago
WL#3817: Simplify string / memory area types and make things more consistent (first part) The following type conversions was done: - Changed byte to uchar - Changed gptr to uchar* - Change my_string to char * - Change my_size_t to size_t - Change size_s to size_t Removed declaration of byte, gptr, my_string, my_size_t and size_s. Following function parameter changes was done: - All string functions in mysys/strings was changed to use size_t instead of uint for string lengths. - All read()/write() functions changed to use size_t (including vio). - All protocoll functions changed to use size_t instead of uint - Functions that used a pointer to a string length was changed to use size_t* - Changed malloc(), free() and related functions from using gptr to use void * as this requires fewer casts in the code and is more in line with how the standard functions work. - Added extra length argument to dirname_part() to return the length of the created string. - Changed (at least) following functions to take uchar* as argument: - db_dump() - my_net_write() - net_write_command() - net_store_data() - DBUG_DUMP() - decimal2bin() & bin2decimal() - Changed my_compress() and my_uncompress() to use size_t. Changed one argument to my_uncompress() from a pointer to a value as we only return one value (makes function easier to use). - Changed type of 'pack_data' argument to packfrm() to avoid casts. - Changed in readfrm() and writefrom(), ha_discover and handler::discover() the type for argument 'frmdata' to uchar** to avoid casts. - Changed most Field functions to use uchar* instead of char* (reduced a lot of casts). - Changed field->val_xxx(xxx, new_ptr) to take const pointers. Other changes: - Removed a lot of not needed casts - Added a few new cast required by other changes - Added some cast to my_multi_malloc() arguments for safety (as string lengths needs to be uint, not size_t). - Fixed all calls to hash-get-key functions to use size_t*. (Needed to be done explicitely as this conflict was often hided by casting the function to hash_get_key). - Changed some buffers to memory regions to uchar* to avoid casts. - Changed some string lengths from uint to size_t. - Changed field->ptr to be uchar* instead of char*. This allowed us to get rid of a lot of casts. - Some changes from true -> TRUE, false -> FALSE, unsigned char -> uchar - Include zlib.h in some files as we needed declaration of crc32() - Changed MY_FILE_ERROR to be (size_t) -1. - Changed many variables to hold the result of my_read() / my_write() to be size_t. This was needed to properly detect errors (which are returned as (size_t) -1). - Removed some very old VMS code - Changed packfrm()/unpackfrm() to not be depending on uint size (portability fix) - Removed windows specific code to restore cursor position as this causes slowdown on windows and we should not mix read() and pread() calls anyway as this is not thread safe. Updated function comment to reflect this. Changed function that depended on original behavior of my_pwrite() to itself restore the cursor position (one such case). - Added some missing checking of return value of malloc(). - Changed definition of MOD_PAD_CHAR_TO_FULL_LENGTH to avoid 'long' overflow. - Changed type of table_def::m_size from my_size_t to ulong to reflect that m_size is the number of elements in the array, not a string/memory length. - Moved THD::max_row_length() to table.cc (as it's not depending on THD). Inlined max_row_length_blob() into this function. - More function comments - Fixed some compiler warnings when compiled without partitions. - Removed setting of LEX_STRING() arguments in declaration (portability fix). - Some trivial indentation/variable name changes. - Some trivial code simplifications: - Replaced some calls to alloc_root + memcpy to use strmake_root()/strdup_root(). - Changed some calls from memdup() to strmake() (Safety fix) - Simpler loops in client-simple.c
19 years ago
21 years ago
23 years ago
23 years ago
23 years ago
Backport of revno 2630.28.10, 2630.28.31, 2630.28.26, 2630.33.1, 2630.39.1, 2630.28.29, 2630.34.3, 2630.34.2, 2630.34.1, 2630.29.29, 2630.29.28, 2630.31.1, 2630.28.13, 2630.28.10, 2617.23.14 and some other minor revisions. This patch implements: WL#4264 "Backup: Stabilize Service Interface" -- all the server prerequisites except si_objects.{h,cc} themselves (they can be just copied over, when needed). WL#4435: Support OUT-parameters in prepared statements. (and all issues in the initial patches for these two tasks, that were discovered in pushbuild and during testing). Bug#39519: mysql_stmt_close() should flush all data associated with the statement. After execution of a prepared statement, send OUT parameters of the invoked stored procedure, if any, to the client. When using the binary protocol, send the parameters in an additional result set over the wire. When using the text protocol, assign out parameters to the user variables from the CALL(@var1, @var2, ...) specification. The following refactoring has been made: - Protocol::send_fields() was renamed to Protocol::send_result_set_metadata(); - A new Protocol::send_result_set_row() was introduced to incapsulate common functionality for sending row data. - Signature of Protocol::prepare_for_send() was changed: this operation does not need a list of items, the number of items is fully sufficient. The following backward incompatible changes have been made: - CLIENT_MULTI_RESULTS is now enabled by default in the client; - CLIENT_PS_MULTI_RESUTLS is now enabled by default in the client.
16 years ago
WL#3817: Simplify string / memory area types and make things more consistent (first part) The following type conversions was done: - Changed byte to uchar - Changed gptr to uchar* - Change my_string to char * - Change my_size_t to size_t - Change size_s to size_t Removed declaration of byte, gptr, my_string, my_size_t and size_s. Following function parameter changes was done: - All string functions in mysys/strings was changed to use size_t instead of uint for string lengths. - All read()/write() functions changed to use size_t (including vio). - All protocoll functions changed to use size_t instead of uint - Functions that used a pointer to a string length was changed to use size_t* - Changed malloc(), free() and related functions from using gptr to use void * as this requires fewer casts in the code and is more in line with how the standard functions work. - Added extra length argument to dirname_part() to return the length of the created string. - Changed (at least) following functions to take uchar* as argument: - db_dump() - my_net_write() - net_write_command() - net_store_data() - DBUG_DUMP() - decimal2bin() & bin2decimal() - Changed my_compress() and my_uncompress() to use size_t. Changed one argument to my_uncompress() from a pointer to a value as we only return one value (makes function easier to use). - Changed type of 'pack_data' argument to packfrm() to avoid casts. - Changed in readfrm() and writefrom(), ha_discover and handler::discover() the type for argument 'frmdata' to uchar** to avoid casts. - Changed most Field functions to use uchar* instead of char* (reduced a lot of casts). - Changed field->val_xxx(xxx, new_ptr) to take const pointers. Other changes: - Removed a lot of not needed casts - Added a few new cast required by other changes - Added some cast to my_multi_malloc() arguments for safety (as string lengths needs to be uint, not size_t). - Fixed all calls to hash-get-key functions to use size_t*. (Needed to be done explicitely as this conflict was often hided by casting the function to hash_get_key). - Changed some buffers to memory regions to uchar* to avoid casts. - Changed some string lengths from uint to size_t. - Changed field->ptr to be uchar* instead of char*. This allowed us to get rid of a lot of casts. - Some changes from true -> TRUE, false -> FALSE, unsigned char -> uchar - Include zlib.h in some files as we needed declaration of crc32() - Changed MY_FILE_ERROR to be (size_t) -1. - Changed many variables to hold the result of my_read() / my_write() to be size_t. This was needed to properly detect errors (which are returned as (size_t) -1). - Removed some very old VMS code - Changed packfrm()/unpackfrm() to not be depending on uint size (portability fix) - Removed windows specific code to restore cursor position as this causes slowdown on windows and we should not mix read() and pread() calls anyway as this is not thread safe. Updated function comment to reflect this. Changed function that depended on original behavior of my_pwrite() to itself restore the cursor position (one such case). - Added some missing checking of return value of malloc(). - Changed definition of MOD_PAD_CHAR_TO_FULL_LENGTH to avoid 'long' overflow. - Changed type of table_def::m_size from my_size_t to ulong to reflect that m_size is the number of elements in the array, not a string/memory length. - Moved THD::max_row_length() to table.cc (as it's not depending on THD). Inlined max_row_length_blob() into this function. - More function comments - Fixed some compiler warnings when compiled without partitions. - Removed setting of LEX_STRING() arguments in declaration (portability fix). - Some trivial indentation/variable name changes. - Some trivial code simplifications: - Replaced some calls to alloc_root + memcpy to use strmake_root()/strdup_root(). - Changed some calls from memdup() to strmake() (Safety fix) - Simpler loops in client-simple.c
19 years ago
A fix for Bug#26750 "valgrind leak in sp_head" (and post-review fixes). The legend: on a replication slave, in case a trigger creation was filtered out because of application of replicate-do-table/ replicate-ignore-table rule, the parsed definition of a trigger was not cleaned up properly. LEX::sphead member was left around and leaked memory. Until the actual implementation of support of replicate-ignore-table rules for triggers by the patch for Bug 24478 it was never the case that "case SQLCOM_CREATE_TRIGGER" was not executed once a trigger was parsed, so the deletion of lex->sphead there worked and the memory did not leak. The fix: The real cause of the bug is that there is no 1 or 2 places where we can clean up the main LEX after parse. And the reason we can not have just one or two places where we clean up the LEX is asymmetric behaviour of MYSQLparse in case of success or error. One of the root causes of this behaviour is the code in Item::Item() constructor. There, a newly created item adds itself to THD::free_list - a single-linked list of Items used in a statement. Yuck. This code is unaware that we may have more than one statement active at a time, and always assumes that the free_list of the current statement is located in THD::free_list. One day we need to be able to explicitly allocate an item in a given Query_arena. Thus, when parsing a definition of a stored procedure, like CREATE PROCEDURE p1() BEGIN SELECT a FROM t1; SELECT b FROM t1; END; we actually need to reset THD::mem_root, THD::free_list and THD::lex to parse the nested procedure statement (SELECT *). The actual reset and restore is implemented in semantic actions attached to sp_proc_stmt grammar rule. The problem is that in case of a parsing error inside a nested statement Bison generated parser would abort immediately, without executing the restore part of the semantic action. This would leave THD in an in-the-middle-of-parsing state. This is why we couldn't have had a single place where we clean up the LEX after MYSQLparse - in case of an error we needed to do a clean up immediately, in case of success a clean up could have been delayed. This left the door open for a memory leak. One of the following possibilities were considered when working on a fix: - patch the replication logic to do the clean up. Rejected as breaks module borders, replication code should not need to know the gory details of clean up procedure after CREATE TRIGGER. - wrap MYSQLparse with a function that would do a clean up. Rejected as ideally we should fix the problem when it happens, not adjust for it outside of the problematic code. - make sure MYSQLparse cleans up after itself by invoking the clean up functionality in the appropriate places before return. Implemented in this patch. - use %destructor rule for sp_proc_stmt to restore THD - cleaner than the prevoius approach, but rejected because needs a careful analysis of the side effects, and this patch is for 5.0, and long term we need to use the next alternative anyway - make sure that sp_proc_stmt doesn't juggle with THD - this is a large work that will affect many modules. Cleanup: move main_lex and main_mem_root from Statement to its only two descendants Prepared_statement and THD. This ensures that when a Statement instance was created for purposes of statement backup, we do not involve LEX constructor/destructor, which is fairly expensive. In order to track that the transformation produces equivalent functionality please check the respective constructors and destructors of Statement, Prepared_statement and THD - these members were used only there. This cleanup is unrelated to the patch.
19 years ago
20 years ago
20 years ago
20 years ago
20 years ago
Apply and review: 3655 Jon Olav Hauglid 2009-10-19 Bug #30977 Concurrent statement using stored function and DROP FUNCTION breaks SBR Bug #48246 assert in close_thread_table Implement a fix for: Bug #41804 purge stored procedure cache causes mysterious hang for many minutes Bug #49972 Crash in prepared statements The problem was that concurrent execution of DML statements that use stored functions and DDL statements that drop/modify the same function might result in incorrect binary log in statement (and mixed) mode and therefore break replication. This patch fixes the problem by introducing metadata locking for stored procedures and functions. This is similar to what is done in Bug#25144 for views. Procedures and functions now are locked using metadata locks until the transaction is either committed or rolled back. This prevents other statements from modifying the procedure/function while it is being executed. This provides commit ordering - guaranteeing serializability across multiple transactions and thus fixes the reported binlog problem. Note that we do not take locks for top-level CALLs. This means that procedures called directly are not protected from changes by simultaneous DDL operations so they are executed at the state they had at the time of the CALL. By not taking locks for top-level CALLs, we still allow transactions to be started inside procedures. This patch also changes stored procedure cache invalidation. Upon a change of cache version, we no longer invalidate the entire cache, but only those routines which we use, only when a statement is executed that uses them. This patch also changes the logic of prepared statement validation. A stored procedure used by a prepared statement is now validated only once a metadata lock has been acquired. A version mismatch causes a flush of the obsolete routine from the cache and statement reprepare. Incompatible changes: 1) ER_LOCK_DEADLOCK is reported for a transaction trying to access a procedure/function that is locked by a DDL operation in another connection. 2) Procedure/function DDL operations are now prohibited in LOCK TABLES mode as exclusive locks must be taken all at once and LOCK TABLES provides no way to specifiy procedures/functions to be locked. Test cases have been added to sp-lock.test and rpl_sp.test. Work on this bug has very much been a team effort and this patch includes and is based on contributions from Davi Arnaut, Dmitry Lenev, Magne Mæhre and Konstantin Osipov.
16 years ago
Fixed compiler warnings Fixed compile-pentium64 scripts Fixed wrong estimate of update_with_key_prefix in sql-bench Merge bk-internal.mysql.com:/home/bk/mysql-5.1 into mysql.com:/home/my/mysql-5.1 Fixed unsafe define of uint4korr() Fixed that --extern works with mysql-test-run.pl Small trivial cleanups This also fixes a bug in counting number of rows that are updated when we have many simultanous queries Move all connection handling and command exectuion main loop from sql_parse.cc to sql_connection.cc Split handle_one_connection() into reusable sub functions. Split create_new_thread() into reusable sub functions. Added thread_scheduler; Preliminary interface code for future thread_handling code. Use 'my_thread_id' for internal thread id's Make thr_alarm_kill() to depend on thread_id instead of thread Make thr_abort_locks_for_thread() depend on thread_id instead of thread In store_globals(), set my_thread_var->id to be thd->thread_id. Use my_thread_var->id as basis for my_thread_name() The above changes makes the connection we have between THD and threads more soft. Added a lot of DBUG_PRINT() and DBUG_ASSERT() functions Fixed compiler warnings Fixed core dumps when running with --debug Removed setting of signal masks (was never used) Made event code call pthread_exit() (portability fix) Fixed that event code doesn't call DBUG_xxx functions before my_thread_init() is called. Made handling of thread_id and thd->variables.pseudo_thread_id uniform. Removed one common 'not freed memory' warning from mysqltest Fixed a couple of usage of not initialized warnings (unlikely cases) Suppress compiler warnings from bdb and (for the moment) warnings from ndb
19 years ago
WL#3817: Simplify string / memory area types and make things more consistent (first part) The following type conversions was done: - Changed byte to uchar - Changed gptr to uchar* - Change my_string to char * - Change my_size_t to size_t - Change size_s to size_t Removed declaration of byte, gptr, my_string, my_size_t and size_s. Following function parameter changes was done: - All string functions in mysys/strings was changed to use size_t instead of uint for string lengths. - All read()/write() functions changed to use size_t (including vio). - All protocoll functions changed to use size_t instead of uint - Functions that used a pointer to a string length was changed to use size_t* - Changed malloc(), free() and related functions from using gptr to use void * as this requires fewer casts in the code and is more in line with how the standard functions work. - Added extra length argument to dirname_part() to return the length of the created string. - Changed (at least) following functions to take uchar* as argument: - db_dump() - my_net_write() - net_write_command() - net_store_data() - DBUG_DUMP() - decimal2bin() & bin2decimal() - Changed my_compress() and my_uncompress() to use size_t. Changed one argument to my_uncompress() from a pointer to a value as we only return one value (makes function easier to use). - Changed type of 'pack_data' argument to packfrm() to avoid casts. - Changed in readfrm() and writefrom(), ha_discover and handler::discover() the type for argument 'frmdata' to uchar** to avoid casts. - Changed most Field functions to use uchar* instead of char* (reduced a lot of casts). - Changed field->val_xxx(xxx, new_ptr) to take const pointers. Other changes: - Removed a lot of not needed casts - Added a few new cast required by other changes - Added some cast to my_multi_malloc() arguments for safety (as string lengths needs to be uint, not size_t). - Fixed all calls to hash-get-key functions to use size_t*. (Needed to be done explicitely as this conflict was often hided by casting the function to hash_get_key). - Changed some buffers to memory regions to uchar* to avoid casts. - Changed some string lengths from uint to size_t. - Changed field->ptr to be uchar* instead of char*. This allowed us to get rid of a lot of casts. - Some changes from true -> TRUE, false -> FALSE, unsigned char -> uchar - Include zlib.h in some files as we needed declaration of crc32() - Changed MY_FILE_ERROR to be (size_t) -1. - Changed many variables to hold the result of my_read() / my_write() to be size_t. This was needed to properly detect errors (which are returned as (size_t) -1). - Removed some very old VMS code - Changed packfrm()/unpackfrm() to not be depending on uint size (portability fix) - Removed windows specific code to restore cursor position as this causes slowdown on windows and we should not mix read() and pread() calls anyway as this is not thread safe. Updated function comment to reflect this. Changed function that depended on original behavior of my_pwrite() to itself restore the cursor position (one such case). - Added some missing checking of return value of malloc(). - Changed definition of MOD_PAD_CHAR_TO_FULL_LENGTH to avoid 'long' overflow. - Changed type of table_def::m_size from my_size_t to ulong to reflect that m_size is the number of elements in the array, not a string/memory length. - Moved THD::max_row_length() to table.cc (as it's not depending on THD). Inlined max_row_length_blob() into this function. - More function comments - Fixed some compiler warnings when compiled without partitions. - Removed setting of LEX_STRING() arguments in declaration (portability fix). - Some trivial indentation/variable name changes. - Some trivial code simplifications: - Replaced some calls to alloc_root + memcpy to use strmake_root()/strdup_root(). - Changed some calls from memdup() to strmake() (Safety fix) - Simpler loops in client-simple.c
19 years ago
WL#3817: Simplify string / memory area types and make things more consistent (first part) The following type conversions was done: - Changed byte to uchar - Changed gptr to uchar* - Change my_string to char * - Change my_size_t to size_t - Change size_s to size_t Removed declaration of byte, gptr, my_string, my_size_t and size_s. Following function parameter changes was done: - All string functions in mysys/strings was changed to use size_t instead of uint for string lengths. - All read()/write() functions changed to use size_t (including vio). - All protocoll functions changed to use size_t instead of uint - Functions that used a pointer to a string length was changed to use size_t* - Changed malloc(), free() and related functions from using gptr to use void * as this requires fewer casts in the code and is more in line with how the standard functions work. - Added extra length argument to dirname_part() to return the length of the created string. - Changed (at least) following functions to take uchar* as argument: - db_dump() - my_net_write() - net_write_command() - net_store_data() - DBUG_DUMP() - decimal2bin() & bin2decimal() - Changed my_compress() and my_uncompress() to use size_t. Changed one argument to my_uncompress() from a pointer to a value as we only return one value (makes function easier to use). - Changed type of 'pack_data' argument to packfrm() to avoid casts. - Changed in readfrm() and writefrom(), ha_discover and handler::discover() the type for argument 'frmdata' to uchar** to avoid casts. - Changed most Field functions to use uchar* instead of char* (reduced a lot of casts). - Changed field->val_xxx(xxx, new_ptr) to take const pointers. Other changes: - Removed a lot of not needed casts - Added a few new cast required by other changes - Added some cast to my_multi_malloc() arguments for safety (as string lengths needs to be uint, not size_t). - Fixed all calls to hash-get-key functions to use size_t*. (Needed to be done explicitely as this conflict was often hided by casting the function to hash_get_key). - Changed some buffers to memory regions to uchar* to avoid casts. - Changed some string lengths from uint to size_t. - Changed field->ptr to be uchar* instead of char*. This allowed us to get rid of a lot of casts. - Some changes from true -> TRUE, false -> FALSE, unsigned char -> uchar - Include zlib.h in some files as we needed declaration of crc32() - Changed MY_FILE_ERROR to be (size_t) -1. - Changed many variables to hold the result of my_read() / my_write() to be size_t. This was needed to properly detect errors (which are returned as (size_t) -1). - Removed some very old VMS code - Changed packfrm()/unpackfrm() to not be depending on uint size (portability fix) - Removed windows specific code to restore cursor position as this causes slowdown on windows and we should not mix read() and pread() calls anyway as this is not thread safe. Updated function comment to reflect this. Changed function that depended on original behavior of my_pwrite() to itself restore the cursor position (one such case). - Added some missing checking of return value of malloc(). - Changed definition of MOD_PAD_CHAR_TO_FULL_LENGTH to avoid 'long' overflow. - Changed type of table_def::m_size from my_size_t to ulong to reflect that m_size is the number of elements in the array, not a string/memory length. - Moved THD::max_row_length() to table.cc (as it's not depending on THD). Inlined max_row_length_blob() into this function. - More function comments - Fixed some compiler warnings when compiled without partitions. - Removed setting of LEX_STRING() arguments in declaration (portability fix). - Some trivial indentation/variable name changes. - Some trivial code simplifications: - Replaced some calls to alloc_root + memcpy to use strmake_root()/strdup_root(). - Changed some calls from memdup() to strmake() (Safety fix) - Simpler loops in client-simple.c
19 years ago
23 years ago
23 years ago
23 years ago
23 years ago
23 years ago
23 years ago
23 years ago
23 years ago
19 years ago
Backport of revno 2630.28.10, 2630.28.31, 2630.28.26, 2630.33.1, 2630.39.1, 2630.28.29, 2630.34.3, 2630.34.2, 2630.34.1, 2630.29.29, 2630.29.28, 2630.31.1, 2630.28.13, 2630.28.10, 2617.23.14 and some other minor revisions. This patch implements: WL#4264 "Backup: Stabilize Service Interface" -- all the server prerequisites except si_objects.{h,cc} themselves (they can be just copied over, when needed). WL#4435: Support OUT-parameters in prepared statements. (and all issues in the initial patches for these two tasks, that were discovered in pushbuild and during testing). Bug#39519: mysql_stmt_close() should flush all data associated with the statement. After execution of a prepared statement, send OUT parameters of the invoked stored procedure, if any, to the client. When using the binary protocol, send the parameters in an additional result set over the wire. When using the text protocol, assign out parameters to the user variables from the CALL(@var1, @var2, ...) specification. The following refactoring has been made: - Protocol::send_fields() was renamed to Protocol::send_result_set_metadata(); - A new Protocol::send_result_set_row() was introduced to incapsulate common functionality for sending row data. - Signature of Protocol::prepare_for_send() was changed: this operation does not need a list of items, the number of items is fully sufficient. The following backward incompatible changes have been made: - CLIENT_MULTI_RESULTS is now enabled by default in the client; - CLIENT_PS_MULTI_RESUTLS is now enabled by default in the client.
16 years ago
Backport of revno 2630.28.10, 2630.28.31, 2630.28.26, 2630.33.1, 2630.39.1, 2630.28.29, 2630.34.3, 2630.34.2, 2630.34.1, 2630.29.29, 2630.29.28, 2630.31.1, 2630.28.13, 2630.28.10, 2617.23.14 and some other minor revisions. This patch implements: WL#4264 "Backup: Stabilize Service Interface" -- all the server prerequisites except si_objects.{h,cc} themselves (they can be just copied over, when needed). WL#4435: Support OUT-parameters in prepared statements. (and all issues in the initial patches for these two tasks, that were discovered in pushbuild and during testing). Bug#39519: mysql_stmt_close() should flush all data associated with the statement. After execution of a prepared statement, send OUT parameters of the invoked stored procedure, if any, to the client. When using the binary protocol, send the parameters in an additional result set over the wire. When using the text protocol, assign out parameters to the user variables from the CALL(@var1, @var2, ...) specification. The following refactoring has been made: - Protocol::send_fields() was renamed to Protocol::send_result_set_metadata(); - A new Protocol::send_result_set_row() was introduced to incapsulate common functionality for sending row data. - Signature of Protocol::prepare_for_send() was changed: this operation does not need a list of items, the number of items is fully sufficient. The following backward incompatible changes have been made: - CLIENT_MULTI_RESULTS is now enabled by default in the client; - CLIENT_PS_MULTI_RESUTLS is now enabled by default in the client.
16 years ago
Backport of revno 2630.28.10, 2630.28.31, 2630.28.26, 2630.33.1, 2630.39.1, 2630.28.29, 2630.34.3, 2630.34.2, 2630.34.1, 2630.29.29, 2630.29.28, 2630.31.1, 2630.28.13, 2630.28.10, 2617.23.14 and some other minor revisions. This patch implements: WL#4264 "Backup: Stabilize Service Interface" -- all the server prerequisites except si_objects.{h,cc} themselves (they can be just copied over, when needed). WL#4435: Support OUT-parameters in prepared statements. (and all issues in the initial patches for these two tasks, that were discovered in pushbuild and during testing). Bug#39519: mysql_stmt_close() should flush all data associated with the statement. After execution of a prepared statement, send OUT parameters of the invoked stored procedure, if any, to the client. When using the binary protocol, send the parameters in an additional result set over the wire. When using the text protocol, assign out parameters to the user variables from the CALL(@var1, @var2, ...) specification. The following refactoring has been made: - Protocol::send_fields() was renamed to Protocol::send_result_set_metadata(); - A new Protocol::send_result_set_row() was introduced to incapsulate common functionality for sending row data. - Signature of Protocol::prepare_for_send() was changed: this operation does not need a list of items, the number of items is fully sufficient. The following backward incompatible changes have been made: - CLIENT_MULTI_RESULTS is now enabled by default in the client; - CLIENT_PS_MULTI_RESUTLS is now enabled by default in the client.
16 years ago
Backport of revno 2630.28.10, 2630.28.31, 2630.28.26, 2630.33.1, 2630.39.1, 2630.28.29, 2630.34.3, 2630.34.2, 2630.34.1, 2630.29.29, 2630.29.28, 2630.31.1, 2630.28.13, 2630.28.10, 2617.23.14 and some other minor revisions. This patch implements: WL#4264 "Backup: Stabilize Service Interface" -- all the server prerequisites except si_objects.{h,cc} themselves (they can be just copied over, when needed). WL#4435: Support OUT-parameters in prepared statements. (and all issues in the initial patches for these two tasks, that were discovered in pushbuild and during testing). Bug#39519: mysql_stmt_close() should flush all data associated with the statement. After execution of a prepared statement, send OUT parameters of the invoked stored procedure, if any, to the client. When using the binary protocol, send the parameters in an additional result set over the wire. When using the text protocol, assign out parameters to the user variables from the CALL(@var1, @var2, ...) specification. The following refactoring has been made: - Protocol::send_fields() was renamed to Protocol::send_result_set_metadata(); - A new Protocol::send_result_set_row() was introduced to incapsulate common functionality for sending row data. - Signature of Protocol::prepare_for_send() was changed: this operation does not need a list of items, the number of items is fully sufficient. The following backward incompatible changes have been made: - CLIENT_MULTI_RESULTS is now enabled by default in the client; - CLIENT_PS_MULTI_RESUTLS is now enabled by default in the client.
16 years ago
Backport of revno 2630.28.10, 2630.28.31, 2630.28.26, 2630.33.1, 2630.39.1, 2630.28.29, 2630.34.3, 2630.34.2, 2630.34.1, 2630.29.29, 2630.29.28, 2630.31.1, 2630.28.13, 2630.28.10, 2617.23.14 and some other minor revisions. This patch implements: WL#4264 "Backup: Stabilize Service Interface" -- all the server prerequisites except si_objects.{h,cc} themselves (they can be just copied over, when needed). WL#4435: Support OUT-parameters in prepared statements. (and all issues in the initial patches for these two tasks, that were discovered in pushbuild and during testing). Bug#39519: mysql_stmt_close() should flush all data associated with the statement. After execution of a prepared statement, send OUT parameters of the invoked stored procedure, if any, to the client. When using the binary protocol, send the parameters in an additional result set over the wire. When using the text protocol, assign out parameters to the user variables from the CALL(@var1, @var2, ...) specification. The following refactoring has been made: - Protocol::send_fields() was renamed to Protocol::send_result_set_metadata(); - A new Protocol::send_result_set_row() was introduced to incapsulate common functionality for sending row data. - Signature of Protocol::prepare_for_send() was changed: this operation does not need a list of items, the number of items is fully sufficient. The following backward incompatible changes have been made: - CLIENT_MULTI_RESULTS is now enabled by default in the client; - CLIENT_PS_MULTI_RESUTLS is now enabled by default in the client.
16 years ago
Backport of revno 2630.28.10, 2630.28.31, 2630.28.26, 2630.33.1, 2630.39.1, 2630.28.29, 2630.34.3, 2630.34.2, 2630.34.1, 2630.29.29, 2630.29.28, 2630.31.1, 2630.28.13, 2630.28.10, 2617.23.14 and some other minor revisions. This patch implements: WL#4264 "Backup: Stabilize Service Interface" -- all the server prerequisites except si_objects.{h,cc} themselves (they can be just copied over, when needed). WL#4435: Support OUT-parameters in prepared statements. (and all issues in the initial patches for these two tasks, that were discovered in pushbuild and during testing). Bug#39519: mysql_stmt_close() should flush all data associated with the statement. After execution of a prepared statement, send OUT parameters of the invoked stored procedure, if any, to the client. When using the binary protocol, send the parameters in an additional result set over the wire. When using the text protocol, assign out parameters to the user variables from the CALL(@var1, @var2, ...) specification. The following refactoring has been made: - Protocol::send_fields() was renamed to Protocol::send_result_set_metadata(); - A new Protocol::send_result_set_row() was introduced to incapsulate common functionality for sending row data. - Signature of Protocol::prepare_for_send() was changed: this operation does not need a list of items, the number of items is fully sufficient. The following backward incompatible changes have been made: - CLIENT_MULTI_RESULTS is now enabled by default in the client; - CLIENT_PS_MULTI_RESUTLS is now enabled by default in the client.
16 years ago
A fix for Bug#26750 "valgrind leak in sp_head" (and post-review fixes). The legend: on a replication slave, in case a trigger creation was filtered out because of application of replicate-do-table/ replicate-ignore-table rule, the parsed definition of a trigger was not cleaned up properly. LEX::sphead member was left around and leaked memory. Until the actual implementation of support of replicate-ignore-table rules for triggers by the patch for Bug 24478 it was never the case that "case SQLCOM_CREATE_TRIGGER" was not executed once a trigger was parsed, so the deletion of lex->sphead there worked and the memory did not leak. The fix: The real cause of the bug is that there is no 1 or 2 places where we can clean up the main LEX after parse. And the reason we can not have just one or two places where we clean up the LEX is asymmetric behaviour of MYSQLparse in case of success or error. One of the root causes of this behaviour is the code in Item::Item() constructor. There, a newly created item adds itself to THD::free_list - a single-linked list of Items used in a statement. Yuck. This code is unaware that we may have more than one statement active at a time, and always assumes that the free_list of the current statement is located in THD::free_list. One day we need to be able to explicitly allocate an item in a given Query_arena. Thus, when parsing a definition of a stored procedure, like CREATE PROCEDURE p1() BEGIN SELECT a FROM t1; SELECT b FROM t1; END; we actually need to reset THD::mem_root, THD::free_list and THD::lex to parse the nested procedure statement (SELECT *). The actual reset and restore is implemented in semantic actions attached to sp_proc_stmt grammar rule. The problem is that in case of a parsing error inside a nested statement Bison generated parser would abort immediately, without executing the restore part of the semantic action. This would leave THD in an in-the-middle-of-parsing state. This is why we couldn't have had a single place where we clean up the LEX after MYSQLparse - in case of an error we needed to do a clean up immediately, in case of success a clean up could have been delayed. This left the door open for a memory leak. One of the following possibilities were considered when working on a fix: - patch the replication logic to do the clean up. Rejected as breaks module borders, replication code should not need to know the gory details of clean up procedure after CREATE TRIGGER. - wrap MYSQLparse with a function that would do a clean up. Rejected as ideally we should fix the problem when it happens, not adjust for it outside of the problematic code. - make sure MYSQLparse cleans up after itself by invoking the clean up functionality in the appropriate places before return. Implemented in this patch. - use %destructor rule for sp_proc_stmt to restore THD - cleaner than the prevoius approach, but rejected because needs a careful analysis of the side effects, and this patch is for 5.0, and long term we need to use the next alternative anyway - make sure that sp_proc_stmt doesn't juggle with THD - this is a large work that will affect many modules. Cleanup: move main_lex and main_mem_root from Statement to its only two descendants Prepared_statement and THD. This ensures that when a Statement instance was created for purposes of statement backup, we do not involve LEX constructor/destructor, which is fairly expensive. In order to track that the transformation produces equivalent functionality please check the respective constructors and destructors of Statement, Prepared_statement and THD - these members were used only there. This cleanup is unrelated to the patch.
19 years ago
Apply and review: 3655 Jon Olav Hauglid 2009-10-19 Bug #30977 Concurrent statement using stored function and DROP FUNCTION breaks SBR Bug #48246 assert in close_thread_table Implement a fix for: Bug #41804 purge stored procedure cache causes mysterious hang for many minutes Bug #49972 Crash in prepared statements The problem was that concurrent execution of DML statements that use stored functions and DDL statements that drop/modify the same function might result in incorrect binary log in statement (and mixed) mode and therefore break replication. This patch fixes the problem by introducing metadata locking for stored procedures and functions. This is similar to what is done in Bug#25144 for views. Procedures and functions now are locked using metadata locks until the transaction is either committed or rolled back. This prevents other statements from modifying the procedure/function while it is being executed. This provides commit ordering - guaranteeing serializability across multiple transactions and thus fixes the reported binlog problem. Note that we do not take locks for top-level CALLs. This means that procedures called directly are not protected from changes by simultaneous DDL operations so they are executed at the state they had at the time of the CALL. By not taking locks for top-level CALLs, we still allow transactions to be started inside procedures. This patch also changes stored procedure cache invalidation. Upon a change of cache version, we no longer invalidate the entire cache, but only those routines which we use, only when a statement is executed that uses them. This patch also changes the logic of prepared statement validation. A stored procedure used by a prepared statement is now validated only once a metadata lock has been acquired. A version mismatch causes a flush of the obsolete routine from the cache and statement reprepare. Incompatible changes: 1) ER_LOCK_DEADLOCK is reported for a transaction trying to access a procedure/function that is locked by a DDL operation in another connection. 2) Procedure/function DDL operations are now prohibited in LOCK TABLES mode as exclusive locks must be taken all at once and LOCK TABLES provides no way to specifiy procedures/functions to be locked. Test cases have been added to sp-lock.test and rpl_sp.test. Work on this bug has very much been a team effort and this patch includes and is based on contributions from Davi Arnaut, Dmitry Lenev, Magne Mæhre and Konstantin Osipov.
16 years ago
A fix for Bug#26750 "valgrind leak in sp_head" (and post-review fixes). The legend: on a replication slave, in case a trigger creation was filtered out because of application of replicate-do-table/ replicate-ignore-table rule, the parsed definition of a trigger was not cleaned up properly. LEX::sphead member was left around and leaked memory. Until the actual implementation of support of replicate-ignore-table rules for triggers by the patch for Bug 24478 it was never the case that "case SQLCOM_CREATE_TRIGGER" was not executed once a trigger was parsed, so the deletion of lex->sphead there worked and the memory did not leak. The fix: The real cause of the bug is that there is no 1 or 2 places where we can clean up the main LEX after parse. And the reason we can not have just one or two places where we clean up the LEX is asymmetric behaviour of MYSQLparse in case of success or error. One of the root causes of this behaviour is the code in Item::Item() constructor. There, a newly created item adds itself to THD::free_list - a single-linked list of Items used in a statement. Yuck. This code is unaware that we may have more than one statement active at a time, and always assumes that the free_list of the current statement is located in THD::free_list. One day we need to be able to explicitly allocate an item in a given Query_arena. Thus, when parsing a definition of a stored procedure, like CREATE PROCEDURE p1() BEGIN SELECT a FROM t1; SELECT b FROM t1; END; we actually need to reset THD::mem_root, THD::free_list and THD::lex to parse the nested procedure statement (SELECT *). The actual reset and restore is implemented in semantic actions attached to sp_proc_stmt grammar rule. The problem is that in case of a parsing error inside a nested statement Bison generated parser would abort immediately, without executing the restore part of the semantic action. This would leave THD in an in-the-middle-of-parsing state. This is why we couldn't have had a single place where we clean up the LEX after MYSQLparse - in case of an error we needed to do a clean up immediately, in case of success a clean up could have been delayed. This left the door open for a memory leak. One of the following possibilities were considered when working on a fix: - patch the replication logic to do the clean up. Rejected as breaks module borders, replication code should not need to know the gory details of clean up procedure after CREATE TRIGGER. - wrap MYSQLparse with a function that would do a clean up. Rejected as ideally we should fix the problem when it happens, not adjust for it outside of the problematic code. - make sure MYSQLparse cleans up after itself by invoking the clean up functionality in the appropriate places before return. Implemented in this patch. - use %destructor rule for sp_proc_stmt to restore THD - cleaner than the prevoius approach, but rejected because needs a careful analysis of the side effects, and this patch is for 5.0, and long term we need to use the next alternative anyway - make sure that sp_proc_stmt doesn't juggle with THD - this is a large work that will affect many modules. Cleanup: move main_lex and main_mem_root from Statement to its only two descendants Prepared_statement and THD. This ensures that when a Statement instance was created for purposes of statement backup, we do not involve LEX constructor/destructor, which is fairly expensive. In order to track that the transformation produces equivalent functionality please check the respective constructors and destructors of Statement, Prepared_statement and THD - these members were used only there. This cleanup is unrelated to the patch.
19 years ago
A fix for Bug#26750 "valgrind leak in sp_head" (and post-review fixes). The legend: on a replication slave, in case a trigger creation was filtered out because of application of replicate-do-table/ replicate-ignore-table rule, the parsed definition of a trigger was not cleaned up properly. LEX::sphead member was left around and leaked memory. Until the actual implementation of support of replicate-ignore-table rules for triggers by the patch for Bug 24478 it was never the case that "case SQLCOM_CREATE_TRIGGER" was not executed once a trigger was parsed, so the deletion of lex->sphead there worked and the memory did not leak. The fix: The real cause of the bug is that there is no 1 or 2 places where we can clean up the main LEX after parse. And the reason we can not have just one or two places where we clean up the LEX is asymmetric behaviour of MYSQLparse in case of success or error. One of the root causes of this behaviour is the code in Item::Item() constructor. There, a newly created item adds itself to THD::free_list - a single-linked list of Items used in a statement. Yuck. This code is unaware that we may have more than one statement active at a time, and always assumes that the free_list of the current statement is located in THD::free_list. One day we need to be able to explicitly allocate an item in a given Query_arena. Thus, when parsing a definition of a stored procedure, like CREATE PROCEDURE p1() BEGIN SELECT a FROM t1; SELECT b FROM t1; END; we actually need to reset THD::mem_root, THD::free_list and THD::lex to parse the nested procedure statement (SELECT *). The actual reset and restore is implemented in semantic actions attached to sp_proc_stmt grammar rule. The problem is that in case of a parsing error inside a nested statement Bison generated parser would abort immediately, without executing the restore part of the semantic action. This would leave THD in an in-the-middle-of-parsing state. This is why we couldn't have had a single place where we clean up the LEX after MYSQLparse - in case of an error we needed to do a clean up immediately, in case of success a clean up could have been delayed. This left the door open for a memory leak. One of the following possibilities were considered when working on a fix: - patch the replication logic to do the clean up. Rejected as breaks module borders, replication code should not need to know the gory details of clean up procedure after CREATE TRIGGER. - wrap MYSQLparse with a function that would do a clean up. Rejected as ideally we should fix the problem when it happens, not adjust for it outside of the problematic code. - make sure MYSQLparse cleans up after itself by invoking the clean up functionality in the appropriate places before return. Implemented in this patch. - use %destructor rule for sp_proc_stmt to restore THD - cleaner than the prevoius approach, but rejected because needs a careful analysis of the side effects, and this patch is for 5.0, and long term we need to use the next alternative anyway - make sure that sp_proc_stmt doesn't juggle with THD - this is a large work that will affect many modules. Cleanup: move main_lex and main_mem_root from Statement to its only two descendants Prepared_statement and THD. This ensures that when a Statement instance was created for purposes of statement backup, we do not involve LEX constructor/destructor, which is fairly expensive. In order to track that the transformation produces equivalent functionality please check the respective constructors and destructors of Statement, Prepared_statement and THD - these members were used only there. This cleanup is unrelated to the patch.
19 years ago
WL#3817: Simplify string / memory area types and make things more consistent (first part) The following type conversions was done: - Changed byte to uchar - Changed gptr to uchar* - Change my_string to char * - Change my_size_t to size_t - Change size_s to size_t Removed declaration of byte, gptr, my_string, my_size_t and size_s. Following function parameter changes was done: - All string functions in mysys/strings was changed to use size_t instead of uint for string lengths. - All read()/write() functions changed to use size_t (including vio). - All protocoll functions changed to use size_t instead of uint - Functions that used a pointer to a string length was changed to use size_t* - Changed malloc(), free() and related functions from using gptr to use void * as this requires fewer casts in the code and is more in line with how the standard functions work. - Added extra length argument to dirname_part() to return the length of the created string. - Changed (at least) following functions to take uchar* as argument: - db_dump() - my_net_write() - net_write_command() - net_store_data() - DBUG_DUMP() - decimal2bin() & bin2decimal() - Changed my_compress() and my_uncompress() to use size_t. Changed one argument to my_uncompress() from a pointer to a value as we only return one value (makes function easier to use). - Changed type of 'pack_data' argument to packfrm() to avoid casts. - Changed in readfrm() and writefrom(), ha_discover and handler::discover() the type for argument 'frmdata' to uchar** to avoid casts. - Changed most Field functions to use uchar* instead of char* (reduced a lot of casts). - Changed field->val_xxx(xxx, new_ptr) to take const pointers. Other changes: - Removed a lot of not needed casts - Added a few new cast required by other changes - Added some cast to my_multi_malloc() arguments for safety (as string lengths needs to be uint, not size_t). - Fixed all calls to hash-get-key functions to use size_t*. (Needed to be done explicitely as this conflict was often hided by casting the function to hash_get_key). - Changed some buffers to memory regions to uchar* to avoid casts. - Changed some string lengths from uint to size_t. - Changed field->ptr to be uchar* instead of char*. This allowed us to get rid of a lot of casts. - Some changes from true -> TRUE, false -> FALSE, unsigned char -> uchar - Include zlib.h in some files as we needed declaration of crc32() - Changed MY_FILE_ERROR to be (size_t) -1. - Changed many variables to hold the result of my_read() / my_write() to be size_t. This was needed to properly detect errors (which are returned as (size_t) -1). - Removed some very old VMS code - Changed packfrm()/unpackfrm() to not be depending on uint size (portability fix) - Removed windows specific code to restore cursor position as this causes slowdown on windows and we should not mix read() and pread() calls anyway as this is not thread safe. Updated function comment to reflect this. Changed function that depended on original behavior of my_pwrite() to itself restore the cursor position (one such case). - Added some missing checking of return value of malloc(). - Changed definition of MOD_PAD_CHAR_TO_FULL_LENGTH to avoid 'long' overflow. - Changed type of table_def::m_size from my_size_t to ulong to reflect that m_size is the number of elements in the array, not a string/memory length. - Moved THD::max_row_length() to table.cc (as it's not depending on THD). Inlined max_row_length_blob() into this function. - More function comments - Fixed some compiler warnings when compiled without partitions. - Removed setting of LEX_STRING() arguments in declaration (portability fix). - Some trivial indentation/variable name changes. - Some trivial code simplifications: - Replaced some calls to alloc_root + memcpy to use strmake_root()/strdup_root(). - Changed some calls from memdup() to strmake() (Safety fix) - Simpler loops in client-simple.c
19 years ago
Backport of revno ## 2617.31.1, 2617.31.3, 2617.31.4, 2617.31.5, 2617.31.12, 2617.31.15, 2617.31.15, 2617.31.16, 2617.43.1 - initial changeset that introduced the fix for Bug#989 and follow up fixes for all test suite failures introduced in the initial changeset. ------------------------------------------------------------ revno: 2617.31.1 committer: Davi Arnaut <Davi.Arnaut@Sun.COM> branch nick: 4284-6.0 timestamp: Fri 2009-03-06 19:17:00 -0300 message: Bug#989: If DROP TABLE while there's an active transaction, wrong binlog order WL#4284: Transactional DDL locking Currently the MySQL server does not keep metadata locks on schema objects for the duration of a transaction, thus failing to guarantee the integrity of the schema objects being used during the transaction and to protect then from concurrent DDL operations. This also poses a problem for replication as a DDL operation might be replicated even thought there are active transactions using the object being modified. The solution is to defer the release of metadata locks until a active transaction is either committed or rolled back. This prevents other statements from modifying the table for the entire duration of the transaction. This provides commitment ordering for guaranteeing serializability across multiple transactions. - Incompatible change: If MySQL's metadata locking system encounters a lock conflict, the usual schema is to use the try and back-off technique to avoid deadlocks -- this schema consists in releasing all locks and trying to acquire them all in one go. But in a transactional context this algorithm can't be utilized as its not possible to release locks acquired during the course of the transaction without breaking the transaction commitments. To avoid deadlocks in this case, the ER_LOCK_DEADLOCK will be returned if a lock conflict is encountered during a transaction. Let's consider an example: A transaction has two statements that modify table t1, then table t2, and then commits. The first statement of the transaction will acquire a shared metadata lock on table t1, and it will be kept utill COMMIT to ensure serializability. At the moment when the second statement attempts to acquire a shared metadata lock on t2, a concurrent ALTER or DROP statement might have locked t2 exclusively. The prescription of the current locking protocol is that the acquirer of the shared lock backs off -- gives up all his current locks and retries. This implies that the entire multi-statement transaction has to be rolled back. - Incompatible change: FLUSH commands such as FLUSH PRIVILEGES and FLUSH TABLES WITH READ LOCK won't cause locked tables to be implicitly unlocked anymore.
16 years ago
Backport of revno 2630.28.10, 2630.28.31, 2630.28.26, 2630.33.1, 2630.39.1, 2630.28.29, 2630.34.3, 2630.34.2, 2630.34.1, 2630.29.29, 2630.29.28, 2630.31.1, 2630.28.13, 2630.28.10, 2617.23.14 and some other minor revisions. This patch implements: WL#4264 "Backup: Stabilize Service Interface" -- all the server prerequisites except si_objects.{h,cc} themselves (they can be just copied over, when needed). WL#4435: Support OUT-parameters in prepared statements. (and all issues in the initial patches for these two tasks, that were discovered in pushbuild and during testing). Bug#39519: mysql_stmt_close() should flush all data associated with the statement. After execution of a prepared statement, send OUT parameters of the invoked stored procedure, if any, to the client. When using the binary protocol, send the parameters in an additional result set over the wire. When using the text protocol, assign out parameters to the user variables from the CALL(@var1, @var2, ...) specification. The following refactoring has been made: - Protocol::send_fields() was renamed to Protocol::send_result_set_metadata(); - A new Protocol::send_result_set_row() was introduced to incapsulate common functionality for sending row data. - Signature of Protocol::prepare_for_send() was changed: this operation does not need a list of items, the number of items is fully sufficient. The following backward incompatible changes have been made: - CLIENT_MULTI_RESULTS is now enabled by default in the client; - CLIENT_PS_MULTI_RESUTLS is now enabled by default in the client.
16 years ago
Backport of revno ## 2617.31.1, 2617.31.3, 2617.31.4, 2617.31.5, 2617.31.12, 2617.31.15, 2617.31.15, 2617.31.16, 2617.43.1 - initial changeset that introduced the fix for Bug#989 and follow up fixes for all test suite failures introduced in the initial changeset. ------------------------------------------------------------ revno: 2617.31.1 committer: Davi Arnaut <Davi.Arnaut@Sun.COM> branch nick: 4284-6.0 timestamp: Fri 2009-03-06 19:17:00 -0300 message: Bug#989: If DROP TABLE while there's an active transaction, wrong binlog order WL#4284: Transactional DDL locking Currently the MySQL server does not keep metadata locks on schema objects for the duration of a transaction, thus failing to guarantee the integrity of the schema objects being used during the transaction and to protect then from concurrent DDL operations. This also poses a problem for replication as a DDL operation might be replicated even thought there are active transactions using the object being modified. The solution is to defer the release of metadata locks until a active transaction is either committed or rolled back. This prevents other statements from modifying the table for the entire duration of the transaction. This provides commitment ordering for guaranteeing serializability across multiple transactions. - Incompatible change: If MySQL's metadata locking system encounters a lock conflict, the usual schema is to use the try and back-off technique to avoid deadlocks -- this schema consists in releasing all locks and trying to acquire them all in one go. But in a transactional context this algorithm can't be utilized as its not possible to release locks acquired during the course of the transaction without breaking the transaction commitments. To avoid deadlocks in this case, the ER_LOCK_DEADLOCK will be returned if a lock conflict is encountered during a transaction. Let's consider an example: A transaction has two statements that modify table t1, then table t2, and then commits. The first statement of the transaction will acquire a shared metadata lock on table t1, and it will be kept utill COMMIT to ensure serializability. At the moment when the second statement attempts to acquire a shared metadata lock on t2, a concurrent ALTER or DROP statement might have locked t2 exclusively. The prescription of the current locking protocol is that the acquirer of the shared lock backs off -- gives up all his current locks and retries. This implies that the entire multi-statement transaction has to be rolled back. - Incompatible change: FLUSH commands such as FLUSH PRIVILEGES and FLUSH TABLES WITH READ LOCK won't cause locked tables to be implicitly unlocked anymore.
16 years ago
A prerequisite patch for the fix for Bug#46224 "HANDLER statements within a transaction might lead to deadlocks". Introduce a notion of a sentinel to MDL_context. A sentinel is a ticket that separates all tickets in the context into two groups: before and after it. Currently we can have (and need) only one designated sentinel -- it separates all locks taken by LOCK TABLE or HANDLER statement, which must survive COMMIT and ROLLBACK and all other locks, which must be released at COMMIT or ROLLBACK. The tricky part is maintaining the sentinel up to date when someone release its corresponding ticket. This can happen, e.g. if someone issues DROP TABLE under LOCK TABLES (generally, see all calls to release_all_locks_for_name()). MDL_context::release_ticket() is modified to take care of it. ****** A fix and a test case for Bug#46224 "HANDLER statements within a transaction might lead to deadlocks". An attempt to mix HANDLER SQL statements, which are transaction- agnostic, an open multi-statement transaction, and DDL against the involved tables (in a concurrent connection) could lead to a deadlock. The deadlock would occur when HANDLER OPEN or HANDLER READ would have to wait on a conflicting metadata lock. If the connection that issued HANDLER statement also had other metadata locks (say, acquired in scope of a transaction), a classical deadlock situation of mutual wait could occur. Incompatible change: entering LOCK TABLES mode automatically closes all open HANDLERs in the current connection. Incompatible change: previously an attempt to wait on a lock in a connection that has an open HANDLER statement could wait indefinitely/deadlock. After this patch, an error ER_LOCK_DEADLOCK is produced. The idea of the fix is to merge thd->handler_mdl_context with the main mdl_context of the connection, used for transactional locks. This makes deadlock detection possible, since all waits with locks are "visible" and available to analysis in a single MDL context of the connection. Since HANDLER locks and transactional locks have a different life cycle -- HANDLERs are explicitly open and closed, and so are HANDLER locks, explicitly acquired and released, whereas transactional locks "accumulate" till the end of a transaction and are released only with COMMIT, ROLLBACK and ROLLBACK TO SAVEPOINT, a concept of "sentinel" was introduced to MDL_context. All locks, HANDLER and others, reside in the same linked list. However, a selected element of the list separates locks with different life cycle. HANDLER locks always reside at the end of the list, after the sentinel. Transactional locks are prepended to the beginning of the list, before the sentinel. Thus, ROLLBACK, COMMIT or ROLLBACK TO SAVEPOINT, only release those locks that reside before the sentinel. HANDLER locks must be released explicitly as part of HANDLER CLOSE statement, or an implicit close. The same approach with sentinel is also employed for LOCK TABLES locks. Since HANDLER and LOCK TABLES statement has never worked together, the implementation is made simple and only maintains one sentinel, which is used either for HANDLER locks, or for LOCK TABLES locks.
16 years ago
Backport of revno ## 2617.31.1, 2617.31.3, 2617.31.4, 2617.31.5, 2617.31.12, 2617.31.15, 2617.31.15, 2617.31.16, 2617.43.1 - initial changeset that introduced the fix for Bug#989 and follow up fixes for all test suite failures introduced in the initial changeset. ------------------------------------------------------------ revno: 2617.31.1 committer: Davi Arnaut <Davi.Arnaut@Sun.COM> branch nick: 4284-6.0 timestamp: Fri 2009-03-06 19:17:00 -0300 message: Bug#989: If DROP TABLE while there's an active transaction, wrong binlog order WL#4284: Transactional DDL locking Currently the MySQL server does not keep metadata locks on schema objects for the duration of a transaction, thus failing to guarantee the integrity of the schema objects being used during the transaction and to protect then from concurrent DDL operations. This also poses a problem for replication as a DDL operation might be replicated even thought there are active transactions using the object being modified. The solution is to defer the release of metadata locks until a active transaction is either committed or rolled back. This prevents other statements from modifying the table for the entire duration of the transaction. This provides commitment ordering for guaranteeing serializability across multiple transactions. - Incompatible change: If MySQL's metadata locking system encounters a lock conflict, the usual schema is to use the try and back-off technique to avoid deadlocks -- this schema consists in releasing all locks and trying to acquire them all in one go. But in a transactional context this algorithm can't be utilized as its not possible to release locks acquired during the course of the transaction without breaking the transaction commitments. To avoid deadlocks in this case, the ER_LOCK_DEADLOCK will be returned if a lock conflict is encountered during a transaction. Let's consider an example: A transaction has two statements that modify table t1, then table t2, and then commits. The first statement of the transaction will acquire a shared metadata lock on table t1, and it will be kept utill COMMIT to ensure serializability. At the moment when the second statement attempts to acquire a shared metadata lock on t2, a concurrent ALTER or DROP statement might have locked t2 exclusively. The prescription of the current locking protocol is that the acquirer of the shared lock backs off -- gives up all his current locks and retries. This implies that the entire multi-statement transaction has to be rolled back. - Incompatible change: FLUSH commands such as FLUSH PRIVILEGES and FLUSH TABLES WITH READ LOCK won't cause locked tables to be implicitly unlocked anymore.
16 years ago
A fix for Bug#26750 "valgrind leak in sp_head" (and post-review fixes). The legend: on a replication slave, in case a trigger creation was filtered out because of application of replicate-do-table/ replicate-ignore-table rule, the parsed definition of a trigger was not cleaned up properly. LEX::sphead member was left around and leaked memory. Until the actual implementation of support of replicate-ignore-table rules for triggers by the patch for Bug 24478 it was never the case that "case SQLCOM_CREATE_TRIGGER" was not executed once a trigger was parsed, so the deletion of lex->sphead there worked and the memory did not leak. The fix: The real cause of the bug is that there is no 1 or 2 places where we can clean up the main LEX after parse. And the reason we can not have just one or two places where we clean up the LEX is asymmetric behaviour of MYSQLparse in case of success or error. One of the root causes of this behaviour is the code in Item::Item() constructor. There, a newly created item adds itself to THD::free_list - a single-linked list of Items used in a statement. Yuck. This code is unaware that we may have more than one statement active at a time, and always assumes that the free_list of the current statement is located in THD::free_list. One day we need to be able to explicitly allocate an item in a given Query_arena. Thus, when parsing a definition of a stored procedure, like CREATE PROCEDURE p1() BEGIN SELECT a FROM t1; SELECT b FROM t1; END; we actually need to reset THD::mem_root, THD::free_list and THD::lex to parse the nested procedure statement (SELECT *). The actual reset and restore is implemented in semantic actions attached to sp_proc_stmt grammar rule. The problem is that in case of a parsing error inside a nested statement Bison generated parser would abort immediately, without executing the restore part of the semantic action. This would leave THD in an in-the-middle-of-parsing state. This is why we couldn't have had a single place where we clean up the LEX after MYSQLparse - in case of an error we needed to do a clean up immediately, in case of success a clean up could have been delayed. This left the door open for a memory leak. One of the following possibilities were considered when working on a fix: - patch the replication logic to do the clean up. Rejected as breaks module borders, replication code should not need to know the gory details of clean up procedure after CREATE TRIGGER. - wrap MYSQLparse with a function that would do a clean up. Rejected as ideally we should fix the problem when it happens, not adjust for it outside of the problematic code. - make sure MYSQLparse cleans up after itself by invoking the clean up functionality in the appropriate places before return. Implemented in this patch. - use %destructor rule for sp_proc_stmt to restore THD - cleaner than the prevoius approach, but rejected because needs a careful analysis of the side effects, and this patch is for 5.0, and long term we need to use the next alternative anyway - make sure that sp_proc_stmt doesn't juggle with THD - this is a large work that will affect many modules. Cleanup: move main_lex and main_mem_root from Statement to its only two descendants Prepared_statement and THD. This ensures that when a Statement instance was created for purposes of statement backup, we do not involve LEX constructor/destructor, which is fairly expensive. In order to track that the transformation produces equivalent functionality please check the respective constructors and destructors of Statement, Prepared_statement and THD - these members were used only there. This cleanup is unrelated to the patch.
19 years ago
A fix for Bug#26750 "valgrind leak in sp_head" (and post-review fixes). The legend: on a replication slave, in case a trigger creation was filtered out because of application of replicate-do-table/ replicate-ignore-table rule, the parsed definition of a trigger was not cleaned up properly. LEX::sphead member was left around and leaked memory. Until the actual implementation of support of replicate-ignore-table rules for triggers by the patch for Bug 24478 it was never the case that "case SQLCOM_CREATE_TRIGGER" was not executed once a trigger was parsed, so the deletion of lex->sphead there worked and the memory did not leak. The fix: The real cause of the bug is that there is no 1 or 2 places where we can clean up the main LEX after parse. And the reason we can not have just one or two places where we clean up the LEX is asymmetric behaviour of MYSQLparse in case of success or error. One of the root causes of this behaviour is the code in Item::Item() constructor. There, a newly created item adds itself to THD::free_list - a single-linked list of Items used in a statement. Yuck. This code is unaware that we may have more than one statement active at a time, and always assumes that the free_list of the current statement is located in THD::free_list. One day we need to be able to explicitly allocate an item in a given Query_arena. Thus, when parsing a definition of a stored procedure, like CREATE PROCEDURE p1() BEGIN SELECT a FROM t1; SELECT b FROM t1; END; we actually need to reset THD::mem_root, THD::free_list and THD::lex to parse the nested procedure statement (SELECT *). The actual reset and restore is implemented in semantic actions attached to sp_proc_stmt grammar rule. The problem is that in case of a parsing error inside a nested statement Bison generated parser would abort immediately, without executing the restore part of the semantic action. This would leave THD in an in-the-middle-of-parsing state. This is why we couldn't have had a single place where we clean up the LEX after MYSQLparse - in case of an error we needed to do a clean up immediately, in case of success a clean up could have been delayed. This left the door open for a memory leak. One of the following possibilities were considered when working on a fix: - patch the replication logic to do the clean up. Rejected as breaks module borders, replication code should not need to know the gory details of clean up procedure after CREATE TRIGGER. - wrap MYSQLparse with a function that would do a clean up. Rejected as ideally we should fix the problem when it happens, not adjust for it outside of the problematic code. - make sure MYSQLparse cleans up after itself by invoking the clean up functionality in the appropriate places before return. Implemented in this patch. - use %destructor rule for sp_proc_stmt to restore THD - cleaner than the prevoius approach, but rejected because needs a careful analysis of the side effects, and this patch is for 5.0, and long term we need to use the next alternative anyway - make sure that sp_proc_stmt doesn't juggle with THD - this is a large work that will affect many modules. Cleanup: move main_lex and main_mem_root from Statement to its only two descendants Prepared_statement and THD. This ensures that when a Statement instance was created for purposes of statement backup, we do not involve LEX constructor/destructor, which is fairly expensive. In order to track that the transformation produces equivalent functionality please check the respective constructors and destructors of Statement, Prepared_statement and THD - these members were used only there. This cleanup is unrelated to the patch.
19 years ago
Backport of revno 2630.28.10, 2630.28.31, 2630.28.26, 2630.33.1, 2630.39.1, 2630.28.29, 2630.34.3, 2630.34.2, 2630.34.1, 2630.29.29, 2630.29.28, 2630.31.1, 2630.28.13, 2630.28.10, 2617.23.14 and some other minor revisions. This patch implements: WL#4264 "Backup: Stabilize Service Interface" -- all the server prerequisites except si_objects.{h,cc} themselves (they can be just copied over, when needed). WL#4435: Support OUT-parameters in prepared statements. (and all issues in the initial patches for these two tasks, that were discovered in pushbuild and during testing). Bug#39519: mysql_stmt_close() should flush all data associated with the statement. After execution of a prepared statement, send OUT parameters of the invoked stored procedure, if any, to the client. When using the binary protocol, send the parameters in an additional result set over the wire. When using the text protocol, assign out parameters to the user variables from the CALL(@var1, @var2, ...) specification. The following refactoring has been made: - Protocol::send_fields() was renamed to Protocol::send_result_set_metadata(); - A new Protocol::send_result_set_row() was introduced to incapsulate common functionality for sending row data. - Signature of Protocol::prepare_for_send() was changed: this operation does not need a list of items, the number of items is fully sufficient. The following backward incompatible changes have been made: - CLIENT_MULTI_RESULTS is now enabled by default in the client; - CLIENT_PS_MULTI_RESUTLS is now enabled by default in the client.
16 years ago
Backport of revno 2630.28.10, 2630.28.31, 2630.28.26, 2630.33.1, 2630.39.1, 2630.28.29, 2630.34.3, 2630.34.2, 2630.34.1, 2630.29.29, 2630.29.28, 2630.31.1, 2630.28.13, 2630.28.10, 2617.23.14 and some other minor revisions. This patch implements: WL#4264 "Backup: Stabilize Service Interface" -- all the server prerequisites except si_objects.{h,cc} themselves (they can be just copied over, when needed). WL#4435: Support OUT-parameters in prepared statements. (and all issues in the initial patches for these two tasks, that were discovered in pushbuild and during testing). Bug#39519: mysql_stmt_close() should flush all data associated with the statement. After execution of a prepared statement, send OUT parameters of the invoked stored procedure, if any, to the client. When using the binary protocol, send the parameters in an additional result set over the wire. When using the text protocol, assign out parameters to the user variables from the CALL(@var1, @var2, ...) specification. The following refactoring has been made: - Protocol::send_fields() was renamed to Protocol::send_result_set_metadata(); - A new Protocol::send_result_set_row() was introduced to incapsulate common functionality for sending row data. - Signature of Protocol::prepare_for_send() was changed: this operation does not need a list of items, the number of items is fully sufficient. The following backward incompatible changes have been made: - CLIENT_MULTI_RESULTS is now enabled by default in the client; - CLIENT_PS_MULTI_RESUTLS is now enabled by default in the client.
16 years ago
Backport of revno 2630.28.10, 2630.28.31, 2630.28.26, 2630.33.1, 2630.39.1, 2630.28.29, 2630.34.3, 2630.34.2, 2630.34.1, 2630.29.29, 2630.29.28, 2630.31.1, 2630.28.13, 2630.28.10, 2617.23.14 and some other minor revisions. This patch implements: WL#4264 "Backup: Stabilize Service Interface" -- all the server prerequisites except si_objects.{h,cc} themselves (they can be just copied over, when needed). WL#4435: Support OUT-parameters in prepared statements. (and all issues in the initial patches for these two tasks, that were discovered in pushbuild and during testing). Bug#39519: mysql_stmt_close() should flush all data associated with the statement. After execution of a prepared statement, send OUT parameters of the invoked stored procedure, if any, to the client. When using the binary protocol, send the parameters in an additional result set over the wire. When using the text protocol, assign out parameters to the user variables from the CALL(@var1, @var2, ...) specification. The following refactoring has been made: - Protocol::send_fields() was renamed to Protocol::send_result_set_metadata(); - A new Protocol::send_result_set_row() was introduced to incapsulate common functionality for sending row data. - Signature of Protocol::prepare_for_send() was changed: this operation does not need a list of items, the number of items is fully sufficient. The following backward incompatible changes have been made: - CLIENT_MULTI_RESULTS is now enabled by default in the client; - CLIENT_PS_MULTI_RESUTLS is now enabled by default in the client.
16 years ago
Apply and review: 3655 Jon Olav Hauglid 2009-10-19 Bug #30977 Concurrent statement using stored function and DROP FUNCTION breaks SBR Bug #48246 assert in close_thread_table Implement a fix for: Bug #41804 purge stored procedure cache causes mysterious hang for many minutes Bug #49972 Crash in prepared statements The problem was that concurrent execution of DML statements that use stored functions and DDL statements that drop/modify the same function might result in incorrect binary log in statement (and mixed) mode and therefore break replication. This patch fixes the problem by introducing metadata locking for stored procedures and functions. This is similar to what is done in Bug#25144 for views. Procedures and functions now are locked using metadata locks until the transaction is either committed or rolled back. This prevents other statements from modifying the procedure/function while it is being executed. This provides commit ordering - guaranteeing serializability across multiple transactions and thus fixes the reported binlog problem. Note that we do not take locks for top-level CALLs. This means that procedures called directly are not protected from changes by simultaneous DDL operations so they are executed at the state they had at the time of the CALL. By not taking locks for top-level CALLs, we still allow transactions to be started inside procedures. This patch also changes stored procedure cache invalidation. Upon a change of cache version, we no longer invalidate the entire cache, but only those routines which we use, only when a statement is executed that uses them. This patch also changes the logic of prepared statement validation. A stored procedure used by a prepared statement is now validated only once a metadata lock has been acquired. A version mismatch causes a flush of the obsolete routine from the cache and statement reprepare. Incompatible changes: 1) ER_LOCK_DEADLOCK is reported for a transaction trying to access a procedure/function that is locked by a DDL operation in another connection. 2) Procedure/function DDL operations are now prohibited in LOCK TABLES mode as exclusive locks must be taken all at once and LOCK TABLES provides no way to specifiy procedures/functions to be locked. Test cases have been added to sp-lock.test and rpl_sp.test. Work on this bug has very much been a team effort and this patch includes and is based on contributions from Davi Arnaut, Dmitry Lenev, Magne Mæhre and Konstantin Osipov.
16 years ago
Backport of revno 2630.28.10, 2630.28.31, 2630.28.26, 2630.33.1, 2630.39.1, 2630.28.29, 2630.34.3, 2630.34.2, 2630.34.1, 2630.29.29, 2630.29.28, 2630.31.1, 2630.28.13, 2630.28.10, 2617.23.14 and some other minor revisions. This patch implements: WL#4264 "Backup: Stabilize Service Interface" -- all the server prerequisites except si_objects.{h,cc} themselves (they can be just copied over, when needed). WL#4435: Support OUT-parameters in prepared statements. (and all issues in the initial patches for these two tasks, that were discovered in pushbuild and during testing). Bug#39519: mysql_stmt_close() should flush all data associated with the statement. After execution of a prepared statement, send OUT parameters of the invoked stored procedure, if any, to the client. When using the binary protocol, send the parameters in an additional result set over the wire. When using the text protocol, assign out parameters to the user variables from the CALL(@var1, @var2, ...) specification. The following refactoring has been made: - Protocol::send_fields() was renamed to Protocol::send_result_set_metadata(); - A new Protocol::send_result_set_row() was introduced to incapsulate common functionality for sending row data. - Signature of Protocol::prepare_for_send() was changed: this operation does not need a list of items, the number of items is fully sufficient. The following backward incompatible changes have been made: - CLIENT_MULTI_RESULTS is now enabled by default in the client; - CLIENT_PS_MULTI_RESUTLS is now enabled by default in the client.
16 years ago
Backport of revno 2630.28.10, 2630.28.31, 2630.28.26, 2630.33.1, 2630.39.1, 2630.28.29, 2630.34.3, 2630.34.2, 2630.34.1, 2630.29.29, 2630.29.28, 2630.31.1, 2630.28.13, 2630.28.10, 2617.23.14 and some other minor revisions. This patch implements: WL#4264 "Backup: Stabilize Service Interface" -- all the server prerequisites except si_objects.{h,cc} themselves (they can be just copied over, when needed). WL#4435: Support OUT-parameters in prepared statements. (and all issues in the initial patches for these two tasks, that were discovered in pushbuild and during testing). Bug#39519: mysql_stmt_close() should flush all data associated with the statement. After execution of a prepared statement, send OUT parameters of the invoked stored procedure, if any, to the client. When using the binary protocol, send the parameters in an additional result set over the wire. When using the text protocol, assign out parameters to the user variables from the CALL(@var1, @var2, ...) specification. The following refactoring has been made: - Protocol::send_fields() was renamed to Protocol::send_result_set_metadata(); - A new Protocol::send_result_set_row() was introduced to incapsulate common functionality for sending row data. - Signature of Protocol::prepare_for_send() was changed: this operation does not need a list of items, the number of items is fully sufficient. The following backward incompatible changes have been made: - CLIENT_MULTI_RESULTS is now enabled by default in the client; - CLIENT_PS_MULTI_RESUTLS is now enabled by default in the client.
16 years ago
  1. /* Copyright (C) 1995-2002 MySQL AB, 2008-2009 Sun Microsystems, Inc
  2. This program is free software; you can redistribute it and/or modify
  3. it under the terms of the GNU General Public License as published by
  4. the Free Software Foundation; version 2 of the License.
  5. This program is distributed in the hope that it will be useful,
  6. but WITHOUT ANY WARRANTY; without even the implied warranty of
  7. MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
  8. GNU General Public License for more details.
  9. You should have received a copy of the GNU General Public License
  10. along with this program; if not, write to the Free Software
  11. Foundation, Inc., 59 Temple Place, Suite 330, Boston, MA 02111-1307 USA */
  12. /**
  13. @file
  14. This file contains the implementation of prepared statements.
  15. When one prepares a statement:
  16. - Server gets the query from client with command 'COM_STMT_PREPARE';
  17. in the following format:
  18. [COM_STMT_PREPARE:1] [query]
  19. - Parse the query and recognize any parameter markers '?' and
  20. store its information list in lex->param_list
  21. - Allocate a new statement for this prepare; and keep this in
  22. 'thd->stmt_map'.
  23. - Without executing the query, return back to client the total
  24. number of parameters along with result-set metadata information
  25. (if any) in the following format:
  26. @verbatim
  27. [STMT_ID:4]
  28. [Column_count:2]
  29. [Param_count:2]
  30. [Params meta info (stubs only for now)] (if Param_count > 0)
  31. [Columns meta info] (if Column_count > 0)
  32. @endverbatim
  33. During prepare the tables used in a statement are opened, but no
  34. locks are acquired. Table opening will block any DDL during the
  35. operation, and we do not need any locks as we neither read nor
  36. modify any data during prepare. Tables are closed after prepare
  37. finishes.
  38. When one executes a statement:
  39. - Server gets the command 'COM_STMT_EXECUTE' to execute the
  40. previously prepared query. If there are any parameter markers, then the
  41. client will send the data in the following format:
  42. @verbatim
  43. [COM_STMT_EXECUTE:1]
  44. [STMT_ID:4]
  45. [NULL_BITS:(param_count+7)/8)]
  46. [TYPES_SUPPLIED_BY_CLIENT(0/1):1]
  47. [[length]data]
  48. [[length]data] .. [[length]data].
  49. @endverbatim
  50. (Note: Except for string/binary types; all other types will not be
  51. supplied with length field)
  52. - If it is a first execute or types of parameters were altered by client,
  53. then setup the conversion routines.
  54. - Assign parameter items from the supplied data.
  55. - Execute the query without re-parsing and send back the results
  56. to client
  57. During execution of prepared statement tables are opened and locked
  58. the same way they would for normal (non-prepared) statement
  59. execution. Tables are unlocked and closed after the execution.
  60. When one supplies long data for a placeholder:
  61. - Server gets the long data in pieces with command type
  62. 'COM_STMT_SEND_LONG_DATA'.
  63. - The packet recieved will have the format as:
  64. [COM_STMT_SEND_LONG_DATA:1][STMT_ID:4][parameter_number:2][data]
  65. - data from the packet is appended to the long data value buffer for this
  66. placeholder.
  67. - It's up to the client to stop supplying data chunks at any point. The
  68. server doesn't care; also, the server doesn't notify the client whether
  69. it got the data or not; if there is any error, then it will be returned
  70. at statement execute.
  71. */
  72. #include "my_global.h" /* NO_EMBEDDED_ACCESS_CHECKS */
  73. #include "sql_priv.h"
  74. #include "unireg.h"
  75. #include "sql_class.h" // set_var.h: THD
  76. #include "set_var.h"
  77. #include "sql_prepare.h"
  78. #include "sql_parse.h" // insert_precheck, update_precheck, delete_precheck
  79. #include "sql_base.h" // open_normal_and_derived_tables
  80. #include "sql_cache.h" // query_cache_*
  81. #include "sql_view.h" // create_view_precheck
  82. #include "sql_delete.h" // mysql_prepare_delete
  83. #include "sql_select.h" // for JOIN
  84. #include "sql_insert.h" // upgrade_lock_type_for_insert, mysql_prepare_insert
  85. #include "sql_update.h" // mysql_prepare_update
  86. #include "sql_db.h" // mysql_opt_change_db, mysql_change_db
  87. #include "sql_acl.h" // *_ACL
  88. #include "sql_derived.h" // mysql_derived_prepare,
  89. // mysql_handle_derived
  90. #include "sql_cursor.h"
  91. #include "sp_head.h"
  92. #include "sp.h"
  93. #include "sp_cache.h"
  94. #include "probes_mysql.h"
  95. #ifdef EMBEDDED_LIBRARY
  96. /* include MYSQL_BIND headers */
  97. #include <mysql.h>
  98. #else
  99. #include <mysql_com.h>
  100. #endif
  101. #include "lock.h" // MYSQL_OPEN_FORCE_SHARED_MDL
  102. /**
  103. A result class used to send cursor rows using the binary protocol.
  104. */
  105. class Select_fetch_protocol_binary: public select_send
  106. {
  107. Protocol_binary protocol;
  108. public:
  109. Select_fetch_protocol_binary(THD *thd);
  110. virtual bool send_result_set_metadata(List<Item> &list, uint flags);
  111. virtual bool send_data(List<Item> &items);
  112. virtual bool send_eof();
  113. #ifdef EMBEDDED_LIBRARY
  114. void begin_dataset()
  115. {
  116. protocol.begin_dataset();
  117. }
  118. #endif
  119. };
  120. /****************************************************************************/
  121. /**
  122. Prepared_statement: a statement that can contain placeholders.
  123. */
  124. class Prepared_statement: public Statement
  125. {
  126. public:
  127. enum flag_values
  128. {
  129. IS_IN_USE= 1,
  130. IS_SQL_PREPARE= 2
  131. };
  132. THD *thd;
  133. Select_fetch_protocol_binary result;
  134. Item_param **param_array;
  135. Server_side_cursor *cursor;
  136. uint param_count;
  137. uint last_errno;
  138. uint flags;
  139. char last_error[MYSQL_ERRMSG_SIZE];
  140. #ifndef EMBEDDED_LIBRARY
  141. bool (*set_params)(Prepared_statement *st, uchar *data, uchar *data_end,
  142. uchar *read_pos, String *expanded_query);
  143. #else
  144. bool (*set_params_data)(Prepared_statement *st, String *expanded_query);
  145. #endif
  146. bool (*set_params_from_vars)(Prepared_statement *stmt,
  147. List<LEX_STRING>& varnames,
  148. String *expanded_query);
  149. public:
  150. Prepared_statement(THD *thd_arg);
  151. virtual ~Prepared_statement();
  152. void setup_set_params();
  153. virtual Query_arena::Type type() const;
  154. virtual void cleanup_stmt();
  155. bool set_name(LEX_STRING *name);
  156. inline void close_cursor() { delete cursor; cursor= 0; }
  157. inline bool is_in_use() { return flags & (uint) IS_IN_USE; }
  158. inline bool is_sql_prepare() const { return flags & (uint) IS_SQL_PREPARE; }
  159. void set_sql_prepare() { flags|= (uint) IS_SQL_PREPARE; }
  160. bool prepare(const char *packet, uint packet_length);
  161. bool execute_loop(String *expanded_query,
  162. bool open_cursor,
  163. uchar *packet_arg, uchar *packet_end_arg);
  164. bool execute_server_runnable(Server_runnable *server_runnable);
  165. /* Destroy this statement */
  166. void deallocate();
  167. private:
  168. /**
  169. The memory root to allocate parsed tree elements (instances of Item,
  170. SELECT_LEX and other classes).
  171. */
  172. MEM_ROOT main_mem_root;
  173. private:
  174. bool set_db(const char *db, uint db_length);
  175. bool set_parameters(String *expanded_query,
  176. uchar *packet, uchar *packet_end);
  177. bool execute(String *expanded_query, bool open_cursor);
  178. bool reprepare();
  179. bool validate_metadata(Prepared_statement *copy);
  180. void swap_prepared_statement(Prepared_statement *copy);
  181. };
  182. /**
  183. Execute one SQL statement in an isolated context.
  184. */
  185. class Execute_sql_statement: public Server_runnable
  186. {
  187. public:
  188. Execute_sql_statement(LEX_STRING sql_text);
  189. virtual bool execute_server_code(THD *thd);
  190. private:
  191. LEX_STRING m_sql_text;
  192. };
  193. class Ed_connection;
  194. /**
  195. Protocol_local: a helper class to intercept the result
  196. of the data written to the network.
  197. */
  198. class Protocol_local :public Protocol
  199. {
  200. public:
  201. Protocol_local(THD *thd, Ed_connection *ed_connection);
  202. ~Protocol_local() { free_root(&m_rset_root, MYF(0)); }
  203. protected:
  204. virtual void prepare_for_resend();
  205. virtual bool write();
  206. virtual bool store_null();
  207. virtual bool store_tiny(longlong from);
  208. virtual bool store_short(longlong from);
  209. virtual bool store_long(longlong from);
  210. virtual bool store_longlong(longlong from, bool unsigned_flag);
  211. virtual bool store_decimal(const my_decimal *);
  212. virtual bool store(const char *from, size_t length, CHARSET_INFO *cs);
  213. virtual bool store(const char *from, size_t length,
  214. CHARSET_INFO *fromcs, CHARSET_INFO *tocs);
  215. virtual bool store(MYSQL_TIME *time);
  216. virtual bool store_date(MYSQL_TIME *time);
  217. virtual bool store_time(MYSQL_TIME *time);
  218. virtual bool store(float value, uint32 decimals, String *buffer);
  219. virtual bool store(double value, uint32 decimals, String *buffer);
  220. virtual bool store(Field *field);
  221. virtual bool send_result_set_metadata(List<Item> *list, uint flags);
  222. virtual bool send_out_parameters(List<Item_param> *sp_params);
  223. #ifdef EMBEDDED_LIBRARY
  224. void remove_last_row();
  225. #endif
  226. virtual enum enum_protocol_type type() { return PROTOCOL_LOCAL; };
  227. virtual bool send_ok(uint server_status, uint statement_warn_count,
  228. ulonglong affected_rows, ulonglong last_insert_id,
  229. const char *message);
  230. virtual bool send_eof(uint server_status, uint statement_warn_count);
  231. virtual bool send_error(uint sql_errno, const char *err_msg, const char* sqlstate);
  232. private:
  233. bool store_string(const char *str, size_t length,
  234. CHARSET_INFO *src_cs, CHARSET_INFO *dst_cs);
  235. bool store_column(const void *data, size_t length);
  236. void opt_add_row_to_rset();
  237. private:
  238. Ed_connection *m_connection;
  239. MEM_ROOT m_rset_root;
  240. List<Ed_row> *m_rset;
  241. size_t m_column_count;
  242. Ed_column *m_current_row;
  243. Ed_column *m_current_column;
  244. };
  245. /******************************************************************************
  246. Implementation
  247. ******************************************************************************/
  248. inline bool is_param_null(const uchar *pos, ulong param_no)
  249. {
  250. return pos[param_no/8] & (1 << (param_no & 7));
  251. }
  252. /**
  253. Find a prepared statement in the statement map by id.
  254. Try to find a prepared statement and set THD error if it's not found.
  255. @param thd thread handle
  256. @param id statement id
  257. @param where the place from which this function is called (for
  258. error reporting).
  259. @return
  260. 0 if the statement was not found, a pointer otherwise.
  261. */
  262. static Prepared_statement *
  263. find_prepared_statement(THD *thd, ulong id)
  264. {
  265. /*
  266. To strictly separate namespaces of SQL prepared statements and C API
  267. prepared statements find() will return 0 if there is a named prepared
  268. statement with such id.
  269. */
  270. Statement *stmt= thd->stmt_map.find(id);
  271. if (stmt == 0 || stmt->type() != Query_arena::PREPARED_STATEMENT)
  272. return NULL;
  273. return (Prepared_statement *) stmt;
  274. }
  275. /**
  276. Send prepared statement id and metadata to the client after prepare.
  277. @todo
  278. Fix this nasty upcast from List<Item_param> to List<Item>
  279. @return
  280. 0 in case of success, 1 otherwise
  281. */
  282. #ifndef EMBEDDED_LIBRARY
  283. static bool send_prep_stmt(Prepared_statement *stmt, uint columns)
  284. {
  285. NET *net= &stmt->thd->net;
  286. uchar buff[12];
  287. uint tmp;
  288. int error;
  289. THD *thd= stmt->thd;
  290. DBUG_ENTER("send_prep_stmt");
  291. buff[0]= 0; /* OK packet indicator */
  292. int4store(buff+1, stmt->id);
  293. int2store(buff+5, columns);
  294. int2store(buff+7, stmt->param_count);
  295. buff[9]= 0; // Guard against a 4.1 client
  296. tmp= min(stmt->thd->warning_info->statement_warn_count(), 65535);
  297. int2store(buff+10, tmp);
  298. /*
  299. Send types and names of placeholders to the client
  300. XXX: fix this nasty upcast from List<Item_param> to List<Item>
  301. */
  302. error= my_net_write(net, buff, sizeof(buff));
  303. if (stmt->param_count && ! error)
  304. {
  305. error= thd->protocol_text.send_result_set_metadata((List<Item> *)
  306. &stmt->lex->param_list,
  307. Protocol::SEND_EOF);
  308. }
  309. if (!error)
  310. /* Flag that a response has already been sent */
  311. thd->stmt_da->disable_status();
  312. DBUG_RETURN(error);
  313. }
  314. #else
  315. static bool send_prep_stmt(Prepared_statement *stmt,
  316. uint columns __attribute__((unused)))
  317. {
  318. THD *thd= stmt->thd;
  319. thd->client_stmt_id= stmt->id;
  320. thd->client_param_count= stmt->param_count;
  321. thd->clear_error();
  322. thd->stmt_da->disable_status();
  323. return 0;
  324. }
  325. #endif /*!EMBEDDED_LIBRARY*/
  326. #ifndef EMBEDDED_LIBRARY
  327. /**
  328. Read the length of the parameter data and return it back to
  329. the caller.
  330. Read data length, position the packet to the first byte after it,
  331. and return the length to the caller.
  332. @param packet a pointer to the data
  333. @param len remaining packet length
  334. @return
  335. Length of data piece.
  336. */
  337. static ulong get_param_length(uchar **packet, ulong len)
  338. {
  339. reg1 uchar *pos= *packet;
  340. if (len < 1)
  341. return 0;
  342. if (*pos < 251)
  343. {
  344. (*packet)++;
  345. return (ulong) *pos;
  346. }
  347. if (len < 3)
  348. return 0;
  349. if (*pos == 252)
  350. {
  351. (*packet)+=3;
  352. return (ulong) uint2korr(pos+1);
  353. }
  354. if (len < 4)
  355. return 0;
  356. if (*pos == 253)
  357. {
  358. (*packet)+=4;
  359. return (ulong) uint3korr(pos+1);
  360. }
  361. if (len < 5)
  362. return 0;
  363. (*packet)+=9; // Must be 254 when here
  364. /*
  365. In our client-server protocol all numbers bigger than 2^24
  366. stored as 8 bytes with uint8korr. Here we always know that
  367. parameter length is less than 2^4 so don't look at the second
  368. 4 bytes. But still we need to obey the protocol hence 9 in the
  369. assignment above.
  370. */
  371. return (ulong) uint4korr(pos+1);
  372. }
  373. #else
  374. #define get_param_length(packet, len) len
  375. #endif /*!EMBEDDED_LIBRARY*/
  376. /**
  377. Data conversion routines.
  378. All these functions read the data from pos, convert it to requested
  379. type and assign to param; pos is advanced to predefined length.
  380. Make a note that the NULL handling is examined at first execution
  381. (i.e. when input types altered) and for all subsequent executions
  382. we don't read any values for this.
  383. @param param parameter item
  384. @param pos input data buffer
  385. @param len length of data in the buffer
  386. */
  387. static void set_param_tiny(Item_param *param, uchar **pos, ulong len)
  388. {
  389. #ifndef EMBEDDED_LIBRARY
  390. if (len < 1)
  391. return;
  392. #endif
  393. int8 value= (int8) **pos;
  394. param->set_int(param->unsigned_flag ? (longlong) ((uint8) value) :
  395. (longlong) value, 4);
  396. *pos+= 1;
  397. }
  398. static void set_param_short(Item_param *param, uchar **pos, ulong len)
  399. {
  400. int16 value;
  401. #ifndef EMBEDDED_LIBRARY
  402. if (len < 2)
  403. return;
  404. value= sint2korr(*pos);
  405. #else
  406. shortget(value, *pos);
  407. #endif
  408. param->set_int(param->unsigned_flag ? (longlong) ((uint16) value) :
  409. (longlong) value, 6);
  410. *pos+= 2;
  411. }
  412. static void set_param_int32(Item_param *param, uchar **pos, ulong len)
  413. {
  414. int32 value;
  415. #ifndef EMBEDDED_LIBRARY
  416. if (len < 4)
  417. return;
  418. value= sint4korr(*pos);
  419. #else
  420. longget(value, *pos);
  421. #endif
  422. param->set_int(param->unsigned_flag ? (longlong) ((uint32) value) :
  423. (longlong) value, 11);
  424. *pos+= 4;
  425. }
  426. static void set_param_int64(Item_param *param, uchar **pos, ulong len)
  427. {
  428. longlong value;
  429. #ifndef EMBEDDED_LIBRARY
  430. if (len < 8)
  431. return;
  432. value= (longlong) sint8korr(*pos);
  433. #else
  434. longlongget(value, *pos);
  435. #endif
  436. param->set_int(value, 21);
  437. *pos+= 8;
  438. }
  439. static void set_param_float(Item_param *param, uchar **pos, ulong len)
  440. {
  441. float data;
  442. #ifndef EMBEDDED_LIBRARY
  443. if (len < 4)
  444. return;
  445. float4get(data,*pos);
  446. #else
  447. floatget(data, *pos);
  448. #endif
  449. param->set_double((double) data);
  450. *pos+= 4;
  451. }
  452. static void set_param_double(Item_param *param, uchar **pos, ulong len)
  453. {
  454. double data;
  455. #ifndef EMBEDDED_LIBRARY
  456. if (len < 8)
  457. return;
  458. float8get(data,*pos);
  459. #else
  460. doubleget(data, *pos);
  461. #endif
  462. param->set_double((double) data);
  463. *pos+= 8;
  464. }
  465. static void set_param_decimal(Item_param *param, uchar **pos, ulong len)
  466. {
  467. ulong length= get_param_length(pos, len);
  468. param->set_decimal((char*)*pos, length);
  469. *pos+= length;
  470. }
  471. #ifndef EMBEDDED_LIBRARY
  472. /*
  473. Read date/time/datetime parameter values from network (binary
  474. protocol). See writing counterparts of these functions in
  475. libmysql.c (store_param_{time,date,datetime}).
  476. */
  477. /**
  478. @todo
  479. Add warning 'Data truncated' here
  480. */
  481. static void set_param_time(Item_param *param, uchar **pos, ulong len)
  482. {
  483. MYSQL_TIME tm;
  484. ulong length= get_param_length(pos, len);
  485. if (length >= 8)
  486. {
  487. uchar *to= *pos;
  488. uint day;
  489. tm.neg= (bool) to[0];
  490. day= (uint) sint4korr(to+1);
  491. tm.hour= (uint) to[5] + day * 24;
  492. tm.minute= (uint) to[6];
  493. tm.second= (uint) to[7];
  494. tm.second_part= (length > 8) ? (ulong) sint4korr(to+8) : 0;
  495. if (tm.hour > 838)
  496. {
  497. /* TODO: add warning 'Data truncated' here */
  498. tm.hour= 838;
  499. tm.minute= 59;
  500. tm.second= 59;
  501. }
  502. tm.day= tm.year= tm.month= 0;
  503. }
  504. else
  505. set_zero_time(&tm, MYSQL_TIMESTAMP_TIME);
  506. param->set_time(&tm, MYSQL_TIMESTAMP_TIME,
  507. MAX_TIME_WIDTH * MY_CHARSET_BIN_MB_MAXLEN);
  508. *pos+= length;
  509. }
  510. static void set_param_datetime(Item_param *param, uchar **pos, ulong len)
  511. {
  512. MYSQL_TIME tm;
  513. ulong length= get_param_length(pos, len);
  514. if (length >= 4)
  515. {
  516. uchar *to= *pos;
  517. tm.neg= 0;
  518. tm.year= (uint) sint2korr(to);
  519. tm.month= (uint) to[2];
  520. tm.day= (uint) to[3];
  521. if (length > 4)
  522. {
  523. tm.hour= (uint) to[4];
  524. tm.minute= (uint) to[5];
  525. tm.second= (uint) to[6];
  526. }
  527. else
  528. tm.hour= tm.minute= tm.second= 0;
  529. tm.second_part= (length > 7) ? (ulong) sint4korr(to+7) : 0;
  530. }
  531. else
  532. set_zero_time(&tm, MYSQL_TIMESTAMP_DATETIME);
  533. param->set_time(&tm, MYSQL_TIMESTAMP_DATETIME,
  534. MAX_DATETIME_WIDTH * MY_CHARSET_BIN_MB_MAXLEN);
  535. *pos+= length;
  536. }
  537. static void set_param_date(Item_param *param, uchar **pos, ulong len)
  538. {
  539. MYSQL_TIME tm;
  540. ulong length= get_param_length(pos, len);
  541. if (length >= 4)
  542. {
  543. uchar *to= *pos;
  544. tm.year= (uint) sint2korr(to);
  545. tm.month= (uint) to[2];
  546. tm.day= (uint) to[3];
  547. tm.hour= tm.minute= tm.second= 0;
  548. tm.second_part= 0;
  549. tm.neg= 0;
  550. }
  551. else
  552. set_zero_time(&tm, MYSQL_TIMESTAMP_DATE);
  553. param->set_time(&tm, MYSQL_TIMESTAMP_DATE,
  554. MAX_DATE_WIDTH * MY_CHARSET_BIN_MB_MAXLEN);
  555. *pos+= length;
  556. }
  557. #else/*!EMBEDDED_LIBRARY*/
  558. /**
  559. @todo
  560. Add warning 'Data truncated' here
  561. */
  562. void set_param_time(Item_param *param, uchar **pos, ulong len)
  563. {
  564. MYSQL_TIME tm= *((MYSQL_TIME*)*pos);
  565. tm.hour+= tm.day * 24;
  566. tm.day= tm.year= tm.month= 0;
  567. if (tm.hour > 838)
  568. {
  569. /* TODO: add warning 'Data truncated' here */
  570. tm.hour= 838;
  571. tm.minute= 59;
  572. tm.second= 59;
  573. }
  574. param->set_time(&tm, MYSQL_TIMESTAMP_TIME,
  575. MAX_TIME_WIDTH * MY_CHARSET_BIN_MB_MAXLEN);
  576. }
  577. void set_param_datetime(Item_param *param, uchar **pos, ulong len)
  578. {
  579. MYSQL_TIME tm= *((MYSQL_TIME*)*pos);
  580. tm.neg= 0;
  581. param->set_time(&tm, MYSQL_TIMESTAMP_DATETIME,
  582. MAX_DATETIME_WIDTH * MY_CHARSET_BIN_MB_MAXLEN);
  583. }
  584. void set_param_date(Item_param *param, uchar **pos, ulong len)
  585. {
  586. MYSQL_TIME *to= (MYSQL_TIME*)*pos;
  587. param->set_time(to, MYSQL_TIMESTAMP_DATE,
  588. MAX_DATE_WIDTH * MY_CHARSET_BIN_MB_MAXLEN);
  589. }
  590. #endif /*!EMBEDDED_LIBRARY*/
  591. static void set_param_str(Item_param *param, uchar **pos, ulong len)
  592. {
  593. ulong length= get_param_length(pos, len);
  594. if (length > len)
  595. length= len;
  596. param->set_str((const char *)*pos, length);
  597. *pos+= length;
  598. }
  599. #undef get_param_length
  600. static void setup_one_conversion_function(THD *thd, Item_param *param,
  601. uchar param_type)
  602. {
  603. switch (param_type) {
  604. case MYSQL_TYPE_TINY:
  605. param->set_param_func= set_param_tiny;
  606. param->item_type= Item::INT_ITEM;
  607. param->item_result_type= INT_RESULT;
  608. break;
  609. case MYSQL_TYPE_SHORT:
  610. param->set_param_func= set_param_short;
  611. param->item_type= Item::INT_ITEM;
  612. param->item_result_type= INT_RESULT;
  613. break;
  614. case MYSQL_TYPE_LONG:
  615. param->set_param_func= set_param_int32;
  616. param->item_type= Item::INT_ITEM;
  617. param->item_result_type= INT_RESULT;
  618. break;
  619. case MYSQL_TYPE_LONGLONG:
  620. param->set_param_func= set_param_int64;
  621. param->item_type= Item::INT_ITEM;
  622. param->item_result_type= INT_RESULT;
  623. break;
  624. case MYSQL_TYPE_FLOAT:
  625. param->set_param_func= set_param_float;
  626. param->item_type= Item::REAL_ITEM;
  627. param->item_result_type= REAL_RESULT;
  628. break;
  629. case MYSQL_TYPE_DOUBLE:
  630. param->set_param_func= set_param_double;
  631. param->item_type= Item::REAL_ITEM;
  632. param->item_result_type= REAL_RESULT;
  633. break;
  634. case MYSQL_TYPE_DECIMAL:
  635. case MYSQL_TYPE_NEWDECIMAL:
  636. param->set_param_func= set_param_decimal;
  637. param->item_type= Item::DECIMAL_ITEM;
  638. param->item_result_type= DECIMAL_RESULT;
  639. break;
  640. case MYSQL_TYPE_TIME:
  641. param->set_param_func= set_param_time;
  642. param->item_type= Item::STRING_ITEM;
  643. param->item_result_type= STRING_RESULT;
  644. break;
  645. case MYSQL_TYPE_DATE:
  646. param->set_param_func= set_param_date;
  647. param->item_type= Item::STRING_ITEM;
  648. param->item_result_type= STRING_RESULT;
  649. break;
  650. case MYSQL_TYPE_DATETIME:
  651. case MYSQL_TYPE_TIMESTAMP:
  652. param->set_param_func= set_param_datetime;
  653. param->item_type= Item::STRING_ITEM;
  654. param->item_result_type= STRING_RESULT;
  655. break;
  656. case MYSQL_TYPE_TINY_BLOB:
  657. case MYSQL_TYPE_MEDIUM_BLOB:
  658. case MYSQL_TYPE_LONG_BLOB:
  659. case MYSQL_TYPE_BLOB:
  660. param->set_param_func= set_param_str;
  661. param->value.cs_info.character_set_of_placeholder= &my_charset_bin;
  662. param->value.cs_info.character_set_client=
  663. thd->variables.character_set_client;
  664. DBUG_ASSERT(thd->variables.character_set_client);
  665. param->value.cs_info.final_character_set_of_str_value= &my_charset_bin;
  666. param->item_type= Item::STRING_ITEM;
  667. param->item_result_type= STRING_RESULT;
  668. break;
  669. default:
  670. /*
  671. The client library ensures that we won't get any other typecodes
  672. except typecodes above and typecodes for string types. Marking
  673. label as 'default' lets us to handle malformed packets as well.
  674. */
  675. {
  676. CHARSET_INFO *fromcs= thd->variables.character_set_client;
  677. CHARSET_INFO *tocs= thd->variables.collation_connection;
  678. uint32 dummy_offset;
  679. param->value.cs_info.character_set_of_placeholder= fromcs;
  680. param->value.cs_info.character_set_client= fromcs;
  681. /*
  682. Setup source and destination character sets so that they
  683. are different only if conversion is necessary: this will
  684. make later checks easier.
  685. */
  686. param->value.cs_info.final_character_set_of_str_value=
  687. String::needs_conversion(0, fromcs, tocs, &dummy_offset) ?
  688. tocs : fromcs;
  689. param->set_param_func= set_param_str;
  690. /*
  691. Exact value of max_length is not known unless data is converted to
  692. charset of connection, so we have to set it later.
  693. */
  694. param->item_type= Item::STRING_ITEM;
  695. param->item_result_type= STRING_RESULT;
  696. }
  697. }
  698. param->param_type= (enum enum_field_types) param_type;
  699. }
  700. #ifndef EMBEDDED_LIBRARY
  701. /**
  702. Check whether this parameter data type is compatible with long data.
  703. Used to detect whether a long data stream has been supplied to a
  704. incompatible data type.
  705. */
  706. inline bool is_param_long_data_type(Item_param *param)
  707. {
  708. return ((param->param_type >= MYSQL_TYPE_TINY_BLOB) &&
  709. (param->param_type <= MYSQL_TYPE_STRING));
  710. }
  711. /**
  712. Routines to assign parameters from data supplied by the client.
  713. Update the parameter markers by reading data from the packet and
  714. and generate a valid query for logging.
  715. @note
  716. This function, along with other _with_log functions is called when one of
  717. binary, slow or general logs is open. Logging of prepared statements in
  718. all cases is performed by means of conventional queries: if parameter
  719. data was supplied from C API, each placeholder in the query is
  720. replaced with its actual value; if we're logging a [Dynamic] SQL
  721. prepared statement, parameter markers are replaced with variable names.
  722. Example:
  723. @verbatim
  724. mysqld_stmt_prepare("UPDATE t1 SET a=a*1.25 WHERE a=?")
  725. --> general logs gets [Prepare] UPDATE t1 SET a*1.25 WHERE a=?"
  726. mysqld_stmt_execute(stmt);
  727. --> general and binary logs get
  728. [Execute] UPDATE t1 SET a*1.25 WHERE a=1"
  729. @endverbatim
  730. If a statement has been prepared using SQL syntax:
  731. @verbatim
  732. PREPARE stmt FROM "UPDATE t1 SET a=a*1.25 WHERE a=?"
  733. --> general log gets
  734. [Query] PREPARE stmt FROM "UPDATE ..."
  735. EXECUTE stmt USING @a
  736. --> general log gets
  737. [Query] EXECUTE stmt USING @a;
  738. @endverbatim
  739. @retval
  740. 0 if success
  741. @retval
  742. 1 otherwise
  743. */
  744. static bool insert_params_with_log(Prepared_statement *stmt, uchar *null_array,
  745. uchar *read_pos, uchar *data_end,
  746. String *query)
  747. {
  748. THD *thd= stmt->thd;
  749. Item_param **begin= stmt->param_array;
  750. Item_param **end= begin + stmt->param_count;
  751. uint32 length= 0;
  752. String str;
  753. const String *res;
  754. DBUG_ENTER("insert_params_with_log");
  755. if (query->copy(stmt->query(), stmt->query_length(), default_charset_info))
  756. DBUG_RETURN(1);
  757. for (Item_param **it= begin; it < end; ++it)
  758. {
  759. Item_param *param= *it;
  760. if (param->state != Item_param::LONG_DATA_VALUE)
  761. {
  762. if (is_param_null(null_array, (uint) (it - begin)))
  763. param->set_null();
  764. else
  765. {
  766. if (read_pos >= data_end)
  767. DBUG_RETURN(1);
  768. param->set_param_func(param, &read_pos, (uint) (data_end - read_pos));
  769. if (param->state == Item_param::NO_VALUE)
  770. DBUG_RETURN(1);
  771. }
  772. }
  773. /*
  774. A long data stream was supplied for this parameter marker.
  775. This was done after prepare, prior to providing a placeholder
  776. type (the types are supplied at execute). Check that the
  777. supplied type of placeholder can accept a data stream.
  778. */
  779. else if (! is_param_long_data_type(param))
  780. DBUG_RETURN(1);
  781. res= param->query_val_str(&str);
  782. if (param->convert_str_value(thd))
  783. DBUG_RETURN(1); /* out of memory */
  784. if (query->replace(param->pos_in_query+length, 1, *res))
  785. DBUG_RETURN(1);
  786. length+= res->length()-1;
  787. }
  788. DBUG_RETURN(0);
  789. }
  790. static bool insert_params(Prepared_statement *stmt, uchar *null_array,
  791. uchar *read_pos, uchar *data_end,
  792. String *expanded_query)
  793. {
  794. Item_param **begin= stmt->param_array;
  795. Item_param **end= begin + stmt->param_count;
  796. DBUG_ENTER("insert_params");
  797. for (Item_param **it= begin; it < end; ++it)
  798. {
  799. Item_param *param= *it;
  800. if (param->state != Item_param::LONG_DATA_VALUE)
  801. {
  802. if (is_param_null(null_array, (uint) (it - begin)))
  803. param->set_null();
  804. else
  805. {
  806. if (read_pos >= data_end)
  807. DBUG_RETURN(1);
  808. param->set_param_func(param, &read_pos, (uint) (data_end - read_pos));
  809. if (param->state == Item_param::NO_VALUE)
  810. DBUG_RETURN(1);
  811. }
  812. }
  813. /*
  814. A long data stream was supplied for this parameter marker.
  815. This was done after prepare, prior to providing a placeholder
  816. type (the types are supplied at execute). Check that the
  817. supplied type of placeholder can accept a data stream.
  818. */
  819. else if (! is_param_long_data_type(param))
  820. DBUG_RETURN(1);
  821. if (param->convert_str_value(stmt->thd))
  822. DBUG_RETURN(1); /* out of memory */
  823. }
  824. DBUG_RETURN(0);
  825. }
  826. static bool setup_conversion_functions(Prepared_statement *stmt,
  827. uchar **data, uchar *data_end)
  828. {
  829. /* skip null bits */
  830. uchar *read_pos= *data + (stmt->param_count+7) / 8;
  831. DBUG_ENTER("setup_conversion_functions");
  832. if (*read_pos++) //types supplied / first execute
  833. {
  834. /*
  835. First execute or types altered by the client, setup the
  836. conversion routines for all parameters (one time)
  837. */
  838. Item_param **it= stmt->param_array;
  839. Item_param **end= it + stmt->param_count;
  840. THD *thd= stmt->thd;
  841. for (; it < end; ++it)
  842. {
  843. ushort typecode;
  844. const uint signed_bit= 1 << 15;
  845. if (read_pos >= data_end)
  846. DBUG_RETURN(1);
  847. typecode= sint2korr(read_pos);
  848. read_pos+= 2;
  849. (**it).unsigned_flag= test(typecode & signed_bit);
  850. setup_one_conversion_function(thd, *it, (uchar) (typecode & ~signed_bit));
  851. }
  852. }
  853. *data= read_pos;
  854. DBUG_RETURN(0);
  855. }
  856. #else
  857. /**
  858. Embedded counterparts of parameter assignment routines.
  859. The main difference between the embedded library and the server is
  860. that in embedded case we don't serialize/deserialize parameters data.
  861. Additionally, for unknown reason, the client-side flag raised for
  862. changed types of placeholders is ignored and we simply setup conversion
  863. functions at each execute (TODO: fix).
  864. */
  865. static bool emb_insert_params(Prepared_statement *stmt, String *expanded_query)
  866. {
  867. THD *thd= stmt->thd;
  868. Item_param **it= stmt->param_array;
  869. Item_param **end= it + stmt->param_count;
  870. MYSQL_BIND *client_param= stmt->thd->client_params;
  871. DBUG_ENTER("emb_insert_params");
  872. for (; it < end; ++it, ++client_param)
  873. {
  874. Item_param *param= *it;
  875. setup_one_conversion_function(thd, param, client_param->buffer_type);
  876. if (param->state != Item_param::LONG_DATA_VALUE)
  877. {
  878. if (*client_param->is_null)
  879. param->set_null();
  880. else
  881. {
  882. uchar *buff= (uchar*) client_param->buffer;
  883. param->unsigned_flag= client_param->is_unsigned;
  884. param->set_param_func(param, &buff,
  885. client_param->length ?
  886. *client_param->length :
  887. client_param->buffer_length);
  888. if (param->state == Item_param::NO_VALUE)
  889. DBUG_RETURN(1);
  890. }
  891. }
  892. if (param->convert_str_value(thd))
  893. DBUG_RETURN(1); /* out of memory */
  894. }
  895. DBUG_RETURN(0);
  896. }
  897. static bool emb_insert_params_with_log(Prepared_statement *stmt,
  898. String *query)
  899. {
  900. THD *thd= stmt->thd;
  901. Item_param **it= stmt->param_array;
  902. Item_param **end= it + stmt->param_count;
  903. MYSQL_BIND *client_param= thd->client_params;
  904. String str;
  905. const String *res;
  906. uint32 length= 0;
  907. DBUG_ENTER("emb_insert_params_with_log");
  908. if (query->copy(stmt->query(), stmt->query_length(), default_charset_info))
  909. DBUG_RETURN(1);
  910. for (; it < end; ++it, ++client_param)
  911. {
  912. Item_param *param= *it;
  913. setup_one_conversion_function(thd, param, client_param->buffer_type);
  914. if (param->state != Item_param::LONG_DATA_VALUE)
  915. {
  916. if (*client_param->is_null)
  917. param->set_null();
  918. else
  919. {
  920. uchar *buff= (uchar*)client_param->buffer;
  921. param->unsigned_flag= client_param->is_unsigned;
  922. param->set_param_func(param, &buff,
  923. client_param->length ?
  924. *client_param->length :
  925. client_param->buffer_length);
  926. if (param->state == Item_param::NO_VALUE)
  927. DBUG_RETURN(1);
  928. }
  929. }
  930. res= param->query_val_str(&str);
  931. if (param->convert_str_value(thd))
  932. DBUG_RETURN(1); /* out of memory */
  933. if (query->replace(param->pos_in_query+length, 1, *res))
  934. DBUG_RETURN(1);
  935. length+= res->length()-1;
  936. }
  937. DBUG_RETURN(0);
  938. }
  939. #endif /*!EMBEDDED_LIBRARY*/
  940. /**
  941. Setup data conversion routines using an array of parameter
  942. markers from the original prepared statement.
  943. Swap the parameter data of the original prepared
  944. statement to the new one.
  945. Used only when we re-prepare a prepared statement.
  946. There are two reasons for this function to exist:
  947. 1) In the binary client/server protocol, parameter metadata
  948. is sent only at first execute. Consequently, if we need to
  949. reprepare a prepared statement at a subsequent execution,
  950. we may not have metadata information in the packet.
  951. In that case we use the parameter array of the original
  952. prepared statement to setup parameter types of the new
  953. prepared statement.
  954. 2) In the binary client/server protocol, we may supply
  955. long data in pieces. When the last piece is supplied,
  956. we assemble the pieces and convert them from client
  957. character set to the connection character set. After
  958. that the parameter value is only available inside
  959. the parameter, the original pieces are lost, and thus
  960. we can only assign the corresponding parameter of the
  961. reprepared statement from the original value.
  962. @param[out] param_array_dst parameter markers of the new statement
  963. @param[in] param_array_src parameter markers of the original
  964. statement
  965. @param[in] param_count total number of parameters. Is the
  966. same in src and dst arrays, since
  967. the statement query is the same
  968. @return this function never fails
  969. */
  970. static void
  971. swap_parameter_array(Item_param **param_array_dst,
  972. Item_param **param_array_src,
  973. uint param_count)
  974. {
  975. Item_param **dst= param_array_dst;
  976. Item_param **src= param_array_src;
  977. Item_param **end= param_array_dst + param_count;
  978. for (; dst < end; ++src, ++dst)
  979. (*dst)->set_param_type_and_swap_value(*src);
  980. }
  981. /**
  982. Assign prepared statement parameters from user variables.
  983. @param stmt Statement
  984. @param varnames List of variables. Caller must ensure that number
  985. of variables in the list is equal to number of statement
  986. parameters
  987. @param query Ignored
  988. */
  989. static bool insert_params_from_vars(Prepared_statement *stmt,
  990. List<LEX_STRING>& varnames,
  991. String *query __attribute__((unused)))
  992. {
  993. Item_param **begin= stmt->param_array;
  994. Item_param **end= begin + stmt->param_count;
  995. user_var_entry *entry;
  996. LEX_STRING *varname;
  997. List_iterator<LEX_STRING> var_it(varnames);
  998. DBUG_ENTER("insert_params_from_vars");
  999. for (Item_param **it= begin; it < end; ++it)
  1000. {
  1001. Item_param *param= *it;
  1002. varname= var_it++;
  1003. entry= (user_var_entry*)my_hash_search(&stmt->thd->user_vars,
  1004. (uchar*) varname->str,
  1005. varname->length);
  1006. if (param->set_from_user_var(stmt->thd, entry) ||
  1007. param->convert_str_value(stmt->thd))
  1008. DBUG_RETURN(1);
  1009. }
  1010. DBUG_RETURN(0);
  1011. }
  1012. /**
  1013. Do the same as insert_params_from_vars but also construct query text for
  1014. binary log.
  1015. @param stmt Prepared statement
  1016. @param varnames List of variables. Caller must ensure that number of
  1017. variables in the list is equal to number of statement
  1018. parameters
  1019. @param query The query with parameter markers replaced with corresponding
  1020. user variables that were used to execute the query.
  1021. */
  1022. static bool insert_params_from_vars_with_log(Prepared_statement *stmt,
  1023. List<LEX_STRING>& varnames,
  1024. String *query)
  1025. {
  1026. Item_param **begin= stmt->param_array;
  1027. Item_param **end= begin + stmt->param_count;
  1028. user_var_entry *entry;
  1029. LEX_STRING *varname;
  1030. List_iterator<LEX_STRING> var_it(varnames);
  1031. String buf;
  1032. const String *val;
  1033. uint32 length= 0;
  1034. THD *thd= stmt->thd;
  1035. DBUG_ENTER("insert_params_from_vars");
  1036. if (query->copy(stmt->query(), stmt->query_length(), default_charset_info))
  1037. DBUG_RETURN(1);
  1038. for (Item_param **it= begin; it < end; ++it)
  1039. {
  1040. Item_param *param= *it;
  1041. varname= var_it++;
  1042. entry= (user_var_entry *) my_hash_search(&thd->user_vars, (uchar*)
  1043. varname->str, varname->length);
  1044. /*
  1045. We have to call the setup_one_conversion_function() here to set
  1046. the parameter's members that might be needed further
  1047. (e.g. value.cs_info.character_set_client is used in the query_val_str()).
  1048. */
  1049. setup_one_conversion_function(thd, param, param->param_type);
  1050. if (param->set_from_user_var(thd, entry))
  1051. DBUG_RETURN(1);
  1052. val= param->query_val_str(&buf);
  1053. if (param->convert_str_value(thd))
  1054. DBUG_RETURN(1); /* out of memory */
  1055. if (query->replace(param->pos_in_query+length, 1, *val))
  1056. DBUG_RETURN(1);
  1057. length+= val->length()-1;
  1058. }
  1059. DBUG_RETURN(0);
  1060. }
  1061. /**
  1062. Validate INSERT statement.
  1063. @param stmt prepared statement
  1064. @param tables global/local table list
  1065. @retval
  1066. FALSE success
  1067. @retval
  1068. TRUE error, error message is set in THD
  1069. */
  1070. static bool mysql_test_insert(Prepared_statement *stmt,
  1071. TABLE_LIST *table_list,
  1072. List<Item> &fields,
  1073. List<List_item> &values_list,
  1074. List<Item> &update_fields,
  1075. List<Item> &update_values,
  1076. enum_duplicates duplic)
  1077. {
  1078. THD *thd= stmt->thd;
  1079. List_iterator_fast<List_item> its(values_list);
  1080. List_item *values;
  1081. DBUG_ENTER("mysql_test_insert");
  1082. if (insert_precheck(thd, table_list))
  1083. goto error;
  1084. /*
  1085. open temporary memory pool for temporary data allocated by derived
  1086. tables & preparation procedure
  1087. Note that this is done without locks (should not be needed as we will not
  1088. access any data here)
  1089. If we would use locks, then we have to ensure we are not using
  1090. TL_WRITE_DELAYED as having two such locks can cause table corruption.
  1091. */
  1092. if (open_normal_and_derived_tables(thd, table_list,
  1093. MYSQL_OPEN_FORCE_SHARED_MDL))
  1094. goto error;
  1095. if ((values= its++))
  1096. {
  1097. uint value_count;
  1098. ulong counter= 0;
  1099. Item *unused_conds= 0;
  1100. if (table_list->table)
  1101. {
  1102. // don't allocate insert_values
  1103. table_list->table->insert_values=(uchar *)1;
  1104. }
  1105. if (mysql_prepare_insert(thd, table_list, table_list->table,
  1106. fields, values, update_fields, update_values,
  1107. duplic, &unused_conds, FALSE, FALSE, FALSE))
  1108. goto error;
  1109. value_count= values->elements;
  1110. its.rewind();
  1111. if (table_list->lock_type == TL_WRITE_DELAYED &&
  1112. !(table_list->table->file->ha_table_flags() & HA_CAN_INSERT_DELAYED))
  1113. {
  1114. my_error(ER_DELAYED_NOT_SUPPORTED, MYF(0), (table_list->view ?
  1115. table_list->view_name.str :
  1116. table_list->table_name));
  1117. goto error;
  1118. }
  1119. while ((values= its++))
  1120. {
  1121. counter++;
  1122. if (values->elements != value_count)
  1123. {
  1124. my_error(ER_WRONG_VALUE_COUNT_ON_ROW, MYF(0), counter);
  1125. goto error;
  1126. }
  1127. if (setup_fields(thd, 0, *values, MARK_COLUMNS_NONE, 0, 0))
  1128. goto error;
  1129. }
  1130. }
  1131. DBUG_RETURN(FALSE);
  1132. error:
  1133. /* insert_values is cleared in open_table */
  1134. DBUG_RETURN(TRUE);
  1135. }
  1136. /**
  1137. Validate UPDATE statement.
  1138. @param stmt prepared statement
  1139. @param tables list of tables used in this query
  1140. @todo
  1141. - here we should send types of placeholders to the client.
  1142. @retval
  1143. 0 success
  1144. @retval
  1145. 1 error, error message is set in THD
  1146. @retval
  1147. 2 convert to multi_update
  1148. */
  1149. static int mysql_test_update(Prepared_statement *stmt,
  1150. TABLE_LIST *table_list)
  1151. {
  1152. int res;
  1153. THD *thd= stmt->thd;
  1154. uint table_count= 0;
  1155. SELECT_LEX *select= &stmt->lex->select_lex;
  1156. #ifndef NO_EMBEDDED_ACCESS_CHECKS
  1157. uint want_privilege;
  1158. #endif
  1159. DBUG_ENTER("mysql_test_update");
  1160. if (update_precheck(thd, table_list) ||
  1161. open_tables(thd, &table_list, &table_count, MYSQL_OPEN_FORCE_SHARED_MDL))
  1162. goto error;
  1163. if (table_list->multitable_view)
  1164. {
  1165. DBUG_ASSERT(table_list->view != 0);
  1166. DBUG_PRINT("info", ("Switch to multi-update"));
  1167. /* pass counter value */
  1168. thd->lex->table_count= table_count;
  1169. /* convert to multiupdate */
  1170. DBUG_RETURN(2);
  1171. }
  1172. /*
  1173. thd->fill_derived_tables() is false here for sure (because it is
  1174. preparation of PS, so we even do not check it).
  1175. */
  1176. if (mysql_handle_derived(thd->lex, &mysql_derived_prepare))
  1177. goto error;
  1178. #ifndef NO_EMBEDDED_ACCESS_CHECKS
  1179. /* Force privilege re-checking for views after they have been opened. */
  1180. want_privilege= (table_list->view ? UPDATE_ACL :
  1181. table_list->grant.want_privilege);
  1182. #endif
  1183. if (mysql_prepare_update(thd, table_list, &select->where,
  1184. select->order_list.elements,
  1185. select->order_list.first))
  1186. goto error;
  1187. #ifndef NO_EMBEDDED_ACCESS_CHECKS
  1188. table_list->grant.want_privilege= want_privilege;
  1189. table_list->table->grant.want_privilege= want_privilege;
  1190. table_list->register_want_access(want_privilege);
  1191. #endif
  1192. thd->lex->select_lex.no_wrap_view_item= TRUE;
  1193. res= setup_fields(thd, 0, select->item_list, MARK_COLUMNS_READ, 0, 0);
  1194. thd->lex->select_lex.no_wrap_view_item= FALSE;
  1195. if (res)
  1196. goto error;
  1197. #ifndef NO_EMBEDDED_ACCESS_CHECKS
  1198. /* Check values */
  1199. table_list->grant.want_privilege=
  1200. table_list->table->grant.want_privilege=
  1201. (SELECT_ACL & ~table_list->table->grant.privilege);
  1202. table_list->register_want_access(SELECT_ACL);
  1203. #endif
  1204. if (setup_fields(thd, 0, stmt->lex->value_list, MARK_COLUMNS_NONE, 0, 0))
  1205. goto error;
  1206. /* TODO: here we should send types of placeholders to the client. */
  1207. DBUG_RETURN(0);
  1208. error:
  1209. DBUG_RETURN(1);
  1210. }
  1211. /**
  1212. Validate DELETE statement.
  1213. @param stmt prepared statement
  1214. @param tables list of tables used in this query
  1215. @retval
  1216. FALSE success
  1217. @retval
  1218. TRUE error, error message is set in THD
  1219. */
  1220. static bool mysql_test_delete(Prepared_statement *stmt,
  1221. TABLE_LIST *table_list)
  1222. {
  1223. THD *thd= stmt->thd;
  1224. LEX *lex= stmt->lex;
  1225. DBUG_ENTER("mysql_test_delete");
  1226. if (delete_precheck(thd, table_list) ||
  1227. open_normal_and_derived_tables(thd, table_list,
  1228. MYSQL_OPEN_FORCE_SHARED_MDL))
  1229. goto error;
  1230. if (!table_list->table)
  1231. {
  1232. my_error(ER_VIEW_DELETE_MERGE_VIEW, MYF(0),
  1233. table_list->view_db.str, table_list->view_name.str);
  1234. goto error;
  1235. }
  1236. DBUG_RETURN(mysql_prepare_delete(thd, table_list, &lex->select_lex.where));
  1237. error:
  1238. DBUG_RETURN(TRUE);
  1239. }
  1240. /**
  1241. Validate SELECT statement.
  1242. In case of success, if this query is not EXPLAIN, send column list info
  1243. back to the client.
  1244. @param stmt prepared statement
  1245. @param tables list of tables used in the query
  1246. @retval
  1247. 0 success
  1248. @retval
  1249. 1 error, error message is set in THD
  1250. @retval
  1251. 2 success, and statement metadata has been sent
  1252. */
  1253. static int mysql_test_select(Prepared_statement *stmt,
  1254. TABLE_LIST *tables)
  1255. {
  1256. THD *thd= stmt->thd;
  1257. LEX *lex= stmt->lex;
  1258. SELECT_LEX_UNIT *unit= &lex->unit;
  1259. DBUG_ENTER("mysql_test_select");
  1260. lex->select_lex.context.resolve_in_select_list= TRUE;
  1261. ulong privilege= lex->exchange ? SELECT_ACL | FILE_ACL : SELECT_ACL;
  1262. if (tables)
  1263. {
  1264. if (check_table_access(thd, privilege, tables, FALSE, UINT_MAX, FALSE))
  1265. goto error;
  1266. }
  1267. else if (check_access(thd, privilege, any_db, NULL, NULL, 0, 0))
  1268. goto error;
  1269. if (!lex->result && !(lex->result= new (stmt->mem_root) select_send))
  1270. {
  1271. my_error(ER_OUTOFMEMORY, MYF(0), sizeof(select_send));
  1272. goto error;
  1273. }
  1274. if (open_normal_and_derived_tables(thd, tables, MYSQL_OPEN_FORCE_SHARED_MDL))
  1275. goto error;
  1276. thd->used_tables= 0; // Updated by setup_fields
  1277. /*
  1278. JOIN::prepare calls
  1279. It is not SELECT COMMAND for sure, so setup_tables will be called as
  1280. usual, and we pass 0 as setup_tables_done_option
  1281. */
  1282. if (unit->prepare(thd, 0, 0))
  1283. goto error;
  1284. if (!lex->describe && !stmt->is_sql_prepare())
  1285. {
  1286. /* Make copy of item list, as change_columns may change it */
  1287. List<Item> fields(lex->select_lex.item_list);
  1288. /* Change columns if a procedure like analyse() */
  1289. if (unit->last_procedure && unit->last_procedure->change_columns(fields))
  1290. goto error;
  1291. /*
  1292. We can use lex->result as it should've been prepared in
  1293. unit->prepare call above.
  1294. */
  1295. if (send_prep_stmt(stmt, lex->result->field_count(fields)) ||
  1296. lex->result->send_result_set_metadata(fields, Protocol::SEND_EOF) ||
  1297. thd->protocol->flush())
  1298. goto error;
  1299. DBUG_RETURN(2);
  1300. }
  1301. DBUG_RETURN(0);
  1302. error:
  1303. DBUG_RETURN(1);
  1304. }
  1305. /**
  1306. Validate and prepare for execution DO statement expressions.
  1307. @param stmt prepared statement
  1308. @param tables list of tables used in this query
  1309. @param values list of expressions
  1310. @retval
  1311. FALSE success
  1312. @retval
  1313. TRUE error, error message is set in THD
  1314. */
  1315. static bool mysql_test_do_fields(Prepared_statement *stmt,
  1316. TABLE_LIST *tables,
  1317. List<Item> *values)
  1318. {
  1319. THD *thd= stmt->thd;
  1320. DBUG_ENTER("mysql_test_do_fields");
  1321. if (tables && check_table_access(thd, SELECT_ACL, tables, FALSE,
  1322. UINT_MAX, FALSE))
  1323. DBUG_RETURN(TRUE);
  1324. if (open_normal_and_derived_tables(thd, tables, MYSQL_OPEN_FORCE_SHARED_MDL))
  1325. DBUG_RETURN(TRUE);
  1326. DBUG_RETURN(setup_fields(thd, 0, *values, MARK_COLUMNS_NONE, 0, 0));
  1327. }
  1328. /**
  1329. Validate and prepare for execution SET statement expressions.
  1330. @param stmt prepared statement
  1331. @param tables list of tables used in this query
  1332. @param values list of expressions
  1333. @retval
  1334. FALSE success
  1335. @retval
  1336. TRUE error, error message is set in THD
  1337. */
  1338. static bool mysql_test_set_fields(Prepared_statement *stmt,
  1339. TABLE_LIST *tables,
  1340. List<set_var_base> *var_list)
  1341. {
  1342. DBUG_ENTER("mysql_test_set_fields");
  1343. List_iterator_fast<set_var_base> it(*var_list);
  1344. THD *thd= stmt->thd;
  1345. set_var_base *var;
  1346. if ((tables && check_table_access(thd, SELECT_ACL, tables, FALSE,
  1347. UINT_MAX, FALSE)) ||
  1348. open_normal_and_derived_tables(thd, tables, MYSQL_OPEN_FORCE_SHARED_MDL))
  1349. goto error;
  1350. while ((var= it++))
  1351. {
  1352. if (var->light_check(thd))
  1353. goto error;
  1354. }
  1355. DBUG_RETURN(FALSE);
  1356. error:
  1357. DBUG_RETURN(TRUE);
  1358. }
  1359. /**
  1360. Validate and prepare for execution CALL statement expressions.
  1361. @param stmt prepared statement
  1362. @param tables list of tables used in this query
  1363. @param value_list list of expressions
  1364. @retval FALSE success
  1365. @retval TRUE error, error message is set in THD
  1366. */
  1367. static bool mysql_test_call_fields(Prepared_statement *stmt,
  1368. TABLE_LIST *tables,
  1369. List<Item> *value_list)
  1370. {
  1371. DBUG_ENTER("mysql_test_call_fields");
  1372. List_iterator<Item> it(*value_list);
  1373. THD *thd= stmt->thd;
  1374. Item *item;
  1375. if ((tables && check_table_access(thd, SELECT_ACL, tables, FALSE,
  1376. UINT_MAX, FALSE)) ||
  1377. open_normal_and_derived_tables(thd, tables, MYSQL_OPEN_FORCE_SHARED_MDL))
  1378. goto err;
  1379. while ((item= it++))
  1380. {
  1381. if ((!item->fixed && item->fix_fields(thd, it.ref())) ||
  1382. item->check_cols(1))
  1383. goto err;
  1384. }
  1385. DBUG_RETURN(FALSE);
  1386. err:
  1387. DBUG_RETURN(TRUE);
  1388. }
  1389. /**
  1390. Check internal SELECT of the prepared command.
  1391. @param stmt prepared statement
  1392. @param specific_prepare function of command specific prepare
  1393. @param setup_tables_done_option options to be passed to LEX::unit.prepare()
  1394. @note
  1395. This function won't directly open tables used in select. They should
  1396. be opened either by calling function (and in this case you probably
  1397. should use select_like_stmt_test_with_open()) or by
  1398. "specific_prepare" call (like this happens in case of multi-update).
  1399. @retval
  1400. FALSE success
  1401. @retval
  1402. TRUE error, error message is set in THD
  1403. */
  1404. static bool select_like_stmt_test(Prepared_statement *stmt,
  1405. int (*specific_prepare)(THD *thd),
  1406. ulong setup_tables_done_option)
  1407. {
  1408. DBUG_ENTER("select_like_stmt_test");
  1409. THD *thd= stmt->thd;
  1410. LEX *lex= stmt->lex;
  1411. lex->select_lex.context.resolve_in_select_list= TRUE;
  1412. if (specific_prepare && (*specific_prepare)(thd))
  1413. DBUG_RETURN(TRUE);
  1414. thd->used_tables= 0; // Updated by setup_fields
  1415. /* Calls JOIN::prepare */
  1416. DBUG_RETURN(lex->unit.prepare(thd, 0, setup_tables_done_option));
  1417. }
  1418. /**
  1419. Check internal SELECT of the prepared command (with opening of used
  1420. tables).
  1421. @param stmt prepared statement
  1422. @param tables list of tables to be opened
  1423. before calling specific_prepare function
  1424. @param specific_prepare function of command specific prepare
  1425. @param setup_tables_done_option options to be passed to LEX::unit.prepare()
  1426. @retval
  1427. FALSE success
  1428. @retval
  1429. TRUE error
  1430. */
  1431. static bool
  1432. select_like_stmt_test_with_open(Prepared_statement *stmt,
  1433. TABLE_LIST *tables,
  1434. int (*specific_prepare)(THD *thd),
  1435. ulong setup_tables_done_option)
  1436. {
  1437. DBUG_ENTER("select_like_stmt_test_with_open");
  1438. /*
  1439. We should not call LEX::unit.cleanup() after this
  1440. open_normal_and_derived_tables() call because we don't allow
  1441. prepared EXPLAIN yet so derived tables will clean up after
  1442. themself.
  1443. */
  1444. if (open_normal_and_derived_tables(stmt->thd, tables,
  1445. MYSQL_OPEN_FORCE_SHARED_MDL))
  1446. DBUG_RETURN(TRUE);
  1447. DBUG_RETURN(select_like_stmt_test(stmt, specific_prepare,
  1448. setup_tables_done_option));
  1449. }
  1450. /**
  1451. Validate and prepare for execution CREATE TABLE statement.
  1452. @param stmt prepared statement
  1453. @param tables list of tables used in this query
  1454. @retval
  1455. FALSE success
  1456. @retval
  1457. TRUE error, error message is set in THD
  1458. */
  1459. static bool mysql_test_create_table(Prepared_statement *stmt)
  1460. {
  1461. DBUG_ENTER("mysql_test_create_table");
  1462. THD *thd= stmt->thd;
  1463. LEX *lex= stmt->lex;
  1464. SELECT_LEX *select_lex= &lex->select_lex;
  1465. bool res= FALSE;
  1466. bool link_to_local;
  1467. TABLE_LIST *create_table= lex->query_tables;
  1468. TABLE_LIST *tables= lex->create_last_non_select_table->next_global;
  1469. if (create_table_precheck(thd, tables, create_table))
  1470. DBUG_RETURN(TRUE);
  1471. if (select_lex->item_list.elements)
  1472. {
  1473. /* Base table and temporary table are not in the same name space. */
  1474. if (!(lex->create_info.options & HA_LEX_CREATE_TMP_TABLE))
  1475. create_table->open_type= OT_BASE_ONLY;
  1476. if (open_normal_and_derived_tables(stmt->thd, lex->query_tables,
  1477. MYSQL_OPEN_FORCE_SHARED_MDL))
  1478. DBUG_RETURN(TRUE);
  1479. select_lex->context.resolve_in_select_list= TRUE;
  1480. lex->unlink_first_table(&link_to_local);
  1481. res= select_like_stmt_test(stmt, 0, 0);
  1482. lex->link_first_table_back(create_table, link_to_local);
  1483. }
  1484. else
  1485. {
  1486. /*
  1487. Check that the source table exist, and also record
  1488. its metadata version. Even though not strictly necessary,
  1489. we validate metadata of all CREATE TABLE statements,
  1490. which keeps metadata validation code simple.
  1491. */
  1492. if (open_normal_and_derived_tables(stmt->thd, lex->query_tables,
  1493. MYSQL_OPEN_FORCE_SHARED_MDL))
  1494. DBUG_RETURN(TRUE);
  1495. }
  1496. DBUG_RETURN(res);
  1497. }
  1498. /**
  1499. @brief Validate and prepare for execution CREATE VIEW statement
  1500. @param stmt prepared statement
  1501. @note This function handles create view commands.
  1502. @retval FALSE Operation was a success.
  1503. @retval TRUE An error occured.
  1504. */
  1505. static bool mysql_test_create_view(Prepared_statement *stmt)
  1506. {
  1507. DBUG_ENTER("mysql_test_create_view");
  1508. THD *thd= stmt->thd;
  1509. LEX *lex= stmt->lex;
  1510. bool res= TRUE;
  1511. /* Skip first table, which is the view we are creating */
  1512. bool link_to_local;
  1513. TABLE_LIST *view= lex->unlink_first_table(&link_to_local);
  1514. TABLE_LIST *tables= lex->query_tables;
  1515. if (create_view_precheck(thd, tables, view, lex->create_view_mode))
  1516. goto err;
  1517. if (open_normal_and_derived_tables(thd, tables, MYSQL_OPEN_FORCE_SHARED_MDL))
  1518. goto err;
  1519. lex->view_prepare_mode= 1;
  1520. res= select_like_stmt_test(stmt, 0, 0);
  1521. err:
  1522. /* put view back for PS rexecuting */
  1523. lex->link_first_table_back(view, link_to_local);
  1524. DBUG_RETURN(res);
  1525. }
  1526. /*
  1527. Validate and prepare for execution a multi update statement.
  1528. @param stmt prepared statement
  1529. @param tables list of tables used in this query
  1530. @param converted converted to multi-update from usual update
  1531. @retval
  1532. FALSE success
  1533. @retval
  1534. TRUE error, error message is set in THD
  1535. */
  1536. static bool mysql_test_multiupdate(Prepared_statement *stmt,
  1537. TABLE_LIST *tables,
  1538. bool converted)
  1539. {
  1540. /* if we switched from normal update, rights are checked */
  1541. if (!converted && multi_update_precheck(stmt->thd, tables))
  1542. return TRUE;
  1543. return select_like_stmt_test(stmt, &mysql_multi_update_prepare,
  1544. OPTION_SETUP_TABLES_DONE);
  1545. }
  1546. /**
  1547. Validate and prepare for execution a multi delete statement.
  1548. @param stmt prepared statement
  1549. @param tables list of tables used in this query
  1550. @retval
  1551. FALSE success
  1552. @retval
  1553. TRUE error, error message in THD is set.
  1554. */
  1555. static bool mysql_test_multidelete(Prepared_statement *stmt,
  1556. TABLE_LIST *tables)
  1557. {
  1558. stmt->thd->lex->current_select= &stmt->thd->lex->select_lex;
  1559. if (add_item_to_list(stmt->thd, new Item_null()))
  1560. {
  1561. my_error(ER_OUTOFMEMORY, MYF(0), 0);
  1562. goto error;
  1563. }
  1564. if (multi_delete_precheck(stmt->thd, tables) ||
  1565. select_like_stmt_test_with_open(stmt, tables,
  1566. &mysql_multi_delete_prepare,
  1567. OPTION_SETUP_TABLES_DONE))
  1568. goto error;
  1569. if (!tables->table)
  1570. {
  1571. my_error(ER_VIEW_DELETE_MERGE_VIEW, MYF(0),
  1572. tables->view_db.str, tables->view_name.str);
  1573. goto error;
  1574. }
  1575. return FALSE;
  1576. error:
  1577. return TRUE;
  1578. }
  1579. /**
  1580. Wrapper for mysql_insert_select_prepare, to make change of local tables
  1581. after open_normal_and_derived_tables() call.
  1582. @param thd thread handle
  1583. @note
  1584. We need to remove the first local table after
  1585. open_normal_and_derived_tables(), because mysql_handle_derived
  1586. uses local tables lists.
  1587. */
  1588. static int mysql_insert_select_prepare_tester(THD *thd)
  1589. {
  1590. SELECT_LEX *first_select= &thd->lex->select_lex;
  1591. TABLE_LIST *second_table= first_select->table_list.first->next_local;
  1592. /* Skip first table, which is the table we are inserting in */
  1593. first_select->table_list.first= second_table;
  1594. thd->lex->select_lex.context.table_list=
  1595. thd->lex->select_lex.context.first_name_resolution_table= second_table;
  1596. return mysql_insert_select_prepare(thd);
  1597. }
  1598. /**
  1599. Validate and prepare for execution INSERT ... SELECT statement.
  1600. @param stmt prepared statement
  1601. @param tables list of tables used in this query
  1602. @retval
  1603. FALSE success
  1604. @retval
  1605. TRUE error, error message is set in THD
  1606. */
  1607. static bool mysql_test_insert_select(Prepared_statement *stmt,
  1608. TABLE_LIST *tables)
  1609. {
  1610. int res;
  1611. LEX *lex= stmt->lex;
  1612. TABLE_LIST *first_local_table;
  1613. if (tables->table)
  1614. {
  1615. // don't allocate insert_values
  1616. tables->table->insert_values=(uchar *)1;
  1617. }
  1618. if (insert_precheck(stmt->thd, tables))
  1619. return 1;
  1620. /* store it, because mysql_insert_select_prepare_tester change it */
  1621. first_local_table= lex->select_lex.table_list.first;
  1622. DBUG_ASSERT(first_local_table != 0);
  1623. res=
  1624. select_like_stmt_test_with_open(stmt, tables,
  1625. &mysql_insert_select_prepare_tester,
  1626. OPTION_SETUP_TABLES_DONE);
  1627. /* revert changes made by mysql_insert_select_prepare_tester */
  1628. lex->select_lex.table_list.first= first_local_table;
  1629. return res;
  1630. }
  1631. /**
  1632. Perform semantic analysis of the parsed tree and send a response packet
  1633. to the client.
  1634. This function
  1635. - opens all tables and checks access rights
  1636. - validates semantics of statement columns and SQL functions
  1637. by calling fix_fields.
  1638. @param stmt prepared statement
  1639. @retval
  1640. FALSE success, statement metadata is sent to client
  1641. @retval
  1642. TRUE error, error message is set in THD (but not sent)
  1643. */
  1644. static bool check_prepared_statement(Prepared_statement *stmt)
  1645. {
  1646. THD *thd= stmt->thd;
  1647. LEX *lex= stmt->lex;
  1648. SELECT_LEX *select_lex= &lex->select_lex;
  1649. TABLE_LIST *tables;
  1650. enum enum_sql_command sql_command= lex->sql_command;
  1651. int res= 0;
  1652. DBUG_ENTER("check_prepared_statement");
  1653. DBUG_PRINT("enter",("command: %d param_count: %u",
  1654. sql_command, stmt->param_count));
  1655. lex->first_lists_tables_same();
  1656. tables= lex->query_tables;
  1657. /* set context for commands which do not use setup_tables */
  1658. lex->select_lex.context.resolve_in_table_list_only(select_lex->
  1659. get_table_list());
  1660. /* Reset warning count for each query that uses tables */
  1661. if (tables)
  1662. thd->warning_info->opt_clear_warning_info(thd->query_id);
  1663. switch (sql_command) {
  1664. case SQLCOM_REPLACE:
  1665. case SQLCOM_INSERT:
  1666. res= mysql_test_insert(stmt, tables, lex->field_list,
  1667. lex->many_values,
  1668. lex->update_list, lex->value_list,
  1669. lex->duplicates);
  1670. break;
  1671. case SQLCOM_UPDATE:
  1672. res= mysql_test_update(stmt, tables);
  1673. /* mysql_test_update returns 2 if we need to switch to multi-update */
  1674. if (res != 2)
  1675. break;
  1676. case SQLCOM_UPDATE_MULTI:
  1677. res= mysql_test_multiupdate(stmt, tables, res == 2);
  1678. break;
  1679. case SQLCOM_DELETE:
  1680. res= mysql_test_delete(stmt, tables);
  1681. break;
  1682. /* The following allow WHERE clause, so they must be tested like SELECT */
  1683. case SQLCOM_SHOW_DATABASES:
  1684. case SQLCOM_SHOW_TABLES:
  1685. case SQLCOM_SHOW_TRIGGERS:
  1686. case SQLCOM_SHOW_EVENTS:
  1687. case SQLCOM_SHOW_OPEN_TABLES:
  1688. case SQLCOM_SHOW_FIELDS:
  1689. case SQLCOM_SHOW_KEYS:
  1690. case SQLCOM_SHOW_COLLATIONS:
  1691. case SQLCOM_SHOW_CHARSETS:
  1692. case SQLCOM_SHOW_VARIABLES:
  1693. case SQLCOM_SHOW_STATUS:
  1694. case SQLCOM_SHOW_TABLE_STATUS:
  1695. case SQLCOM_SHOW_STATUS_PROC:
  1696. case SQLCOM_SHOW_STATUS_FUNC:
  1697. case SQLCOM_SELECT:
  1698. res= mysql_test_select(stmt, tables);
  1699. if (res == 2)
  1700. {
  1701. /* Statement and field info has already been sent */
  1702. DBUG_RETURN(FALSE);
  1703. }
  1704. break;
  1705. case SQLCOM_CREATE_TABLE:
  1706. res= mysql_test_create_table(stmt);
  1707. break;
  1708. case SQLCOM_CREATE_VIEW:
  1709. if (lex->create_view_mode == VIEW_ALTER)
  1710. {
  1711. my_message(ER_UNSUPPORTED_PS, ER(ER_UNSUPPORTED_PS), MYF(0));
  1712. goto error;
  1713. }
  1714. res= mysql_test_create_view(stmt);
  1715. break;
  1716. case SQLCOM_DO:
  1717. res= mysql_test_do_fields(stmt, tables, lex->insert_list);
  1718. break;
  1719. case SQLCOM_CALL:
  1720. res= mysql_test_call_fields(stmt, tables, &lex->value_list);
  1721. break;
  1722. case SQLCOM_SET_OPTION:
  1723. res= mysql_test_set_fields(stmt, tables, &lex->var_list);
  1724. break;
  1725. case SQLCOM_DELETE_MULTI:
  1726. res= mysql_test_multidelete(stmt, tables);
  1727. break;
  1728. case SQLCOM_INSERT_SELECT:
  1729. case SQLCOM_REPLACE_SELECT:
  1730. res= mysql_test_insert_select(stmt, tables);
  1731. break;
  1732. /*
  1733. Note that we don't need to have cases in this list if they are
  1734. marked with CF_STATUS_COMMAND in sql_command_flags
  1735. */
  1736. case SQLCOM_DROP_TABLE:
  1737. case SQLCOM_RENAME_TABLE:
  1738. case SQLCOM_ALTER_TABLE:
  1739. case SQLCOM_COMMIT:
  1740. case SQLCOM_CREATE_INDEX:
  1741. case SQLCOM_DROP_INDEX:
  1742. case SQLCOM_ROLLBACK:
  1743. case SQLCOM_TRUNCATE:
  1744. case SQLCOM_DROP_VIEW:
  1745. case SQLCOM_REPAIR:
  1746. case SQLCOM_ANALYZE:
  1747. case SQLCOM_OPTIMIZE:
  1748. case SQLCOM_CHANGE_MASTER:
  1749. case SQLCOM_RESET:
  1750. case SQLCOM_FLUSH:
  1751. case SQLCOM_SLAVE_START:
  1752. case SQLCOM_SLAVE_STOP:
  1753. case SQLCOM_INSTALL_PLUGIN:
  1754. case SQLCOM_UNINSTALL_PLUGIN:
  1755. case SQLCOM_CREATE_DB:
  1756. case SQLCOM_DROP_DB:
  1757. case SQLCOM_ALTER_DB_UPGRADE:
  1758. case SQLCOM_CHECKSUM:
  1759. case SQLCOM_CREATE_USER:
  1760. case SQLCOM_RENAME_USER:
  1761. case SQLCOM_DROP_USER:
  1762. case SQLCOM_ASSIGN_TO_KEYCACHE:
  1763. case SQLCOM_PRELOAD_KEYS:
  1764. case SQLCOM_GRANT:
  1765. case SQLCOM_REVOKE:
  1766. case SQLCOM_KILL:
  1767. break;
  1768. case SQLCOM_PREPARE:
  1769. case SQLCOM_EXECUTE:
  1770. case SQLCOM_DEALLOCATE_PREPARE:
  1771. default:
  1772. /*
  1773. Trivial check of all status commands. This is easier than having
  1774. things in the above case list, as it's less chance for mistakes.
  1775. */
  1776. if (!(sql_command_flags[sql_command] & CF_STATUS_COMMAND))
  1777. {
  1778. /* All other statements are not supported yet. */
  1779. my_message(ER_UNSUPPORTED_PS, ER(ER_UNSUPPORTED_PS), MYF(0));
  1780. goto error;
  1781. }
  1782. break;
  1783. }
  1784. if (res == 0)
  1785. DBUG_RETURN(stmt->is_sql_prepare() ?
  1786. FALSE : (send_prep_stmt(stmt, 0) || thd->protocol->flush()));
  1787. error:
  1788. DBUG_RETURN(TRUE);
  1789. }
  1790. /**
  1791. Initialize array of parameters in statement from LEX.
  1792. (We need to have quick access to items by number in mysql_stmt_get_longdata).
  1793. This is to avoid using malloc/realloc in the parser.
  1794. */
  1795. static bool init_param_array(Prepared_statement *stmt)
  1796. {
  1797. LEX *lex= stmt->lex;
  1798. if ((stmt->param_count= lex->param_list.elements))
  1799. {
  1800. if (stmt->param_count > (uint) UINT_MAX16)
  1801. {
  1802. /* Error code to be defined in 5.0 */
  1803. my_message(ER_PS_MANY_PARAM, ER(ER_PS_MANY_PARAM), MYF(0));
  1804. return TRUE;
  1805. }
  1806. Item_param **to;
  1807. List_iterator<Item_param> param_iterator(lex->param_list);
  1808. /* Use thd->mem_root as it points at statement mem_root */
  1809. stmt->param_array= (Item_param **)
  1810. alloc_root(stmt->thd->mem_root,
  1811. sizeof(Item_param*) * stmt->param_count);
  1812. if (!stmt->param_array)
  1813. return TRUE;
  1814. for (to= stmt->param_array;
  1815. to < stmt->param_array + stmt->param_count;
  1816. ++to)
  1817. {
  1818. *to= param_iterator++;
  1819. }
  1820. }
  1821. return FALSE;
  1822. }
  1823. /**
  1824. COM_STMT_PREPARE handler.
  1825. Given a query string with parameter markers, create a prepared
  1826. statement from it and send PS info back to the client.
  1827. If parameter markers are found in the query, then store the information
  1828. using Item_param along with maintaining a list in lex->param_array, so
  1829. that a fast and direct retrieval can be made without going through all
  1830. field items.
  1831. @param packet query to be prepared
  1832. @param packet_length query string length, including ignored
  1833. trailing NULL or quote char.
  1834. @note
  1835. This function parses the query and sends the total number of parameters
  1836. and resultset metadata information back to client (if any), without
  1837. executing the query i.e. without any log/disk writes. This allows the
  1838. queries to be re-executed without re-parsing during execute.
  1839. @return
  1840. none: in case of success a new statement id and metadata is sent
  1841. to the client, otherwise an error message is set in THD.
  1842. */
  1843. void mysqld_stmt_prepare(THD *thd, const char *packet, uint packet_length)
  1844. {
  1845. Protocol *save_protocol= thd->protocol;
  1846. Prepared_statement *stmt;
  1847. DBUG_ENTER("mysqld_stmt_prepare");
  1848. DBUG_PRINT("prep_query", ("%s", packet));
  1849. /* First of all clear possible warnings from the previous command */
  1850. mysql_reset_thd_for_next_command(thd);
  1851. if (! (stmt= new Prepared_statement(thd)))
  1852. DBUG_VOID_RETURN; /* out of memory: error is set in Sql_alloc */
  1853. if (thd->stmt_map.insert(thd, stmt))
  1854. {
  1855. /*
  1856. The error is set in the insert. The statement itself
  1857. will be also deleted there (this is how the hash works).
  1858. */
  1859. DBUG_VOID_RETURN;
  1860. }
  1861. thd->protocol= &thd->protocol_binary;
  1862. if (stmt->prepare(packet, packet_length))
  1863. {
  1864. /* Statement map deletes statement on erase */
  1865. thd->stmt_map.erase(stmt);
  1866. }
  1867. thd->protocol= save_protocol;
  1868. /* check_prepared_statemnt sends the metadata packet in case of success */
  1869. DBUG_VOID_RETURN;
  1870. }
  1871. /**
  1872. Get an SQL statement text from a user variable or from plain text.
  1873. If the statement is plain text, just assign the
  1874. pointers, otherwise allocate memory in thd->mem_root and copy
  1875. the contents of the variable, possibly with character
  1876. set conversion.
  1877. @param[in] lex main lex
  1878. @param[out] query_len length of the SQL statement (is set only
  1879. in case of success)
  1880. @retval
  1881. non-zero success
  1882. @retval
  1883. 0 in case of error (out of memory)
  1884. */
  1885. static const char *get_dynamic_sql_string(LEX *lex, uint *query_len)
  1886. {
  1887. THD *thd= lex->thd;
  1888. char *query_str= 0;
  1889. if (lex->prepared_stmt_code_is_varref)
  1890. {
  1891. /* This is PREPARE stmt FROM or EXECUTE IMMEDIATE @var. */
  1892. String str;
  1893. CHARSET_INFO *to_cs= thd->variables.collation_connection;
  1894. bool needs_conversion;
  1895. user_var_entry *entry;
  1896. String *var_value= &str;
  1897. uint32 unused, len;
  1898. /*
  1899. Convert @var contents to string in connection character set. Although
  1900. it is known that int/real/NULL value cannot be a valid query we still
  1901. convert it for error messages to be uniform.
  1902. */
  1903. if ((entry=
  1904. (user_var_entry*)my_hash_search(&thd->user_vars,
  1905. (uchar*)lex->prepared_stmt_code.str,
  1906. lex->prepared_stmt_code.length))
  1907. && entry->value)
  1908. {
  1909. my_bool is_var_null;
  1910. var_value= entry->val_str(&is_var_null, &str, NOT_FIXED_DEC);
  1911. /*
  1912. NULL value of variable checked early as entry->value so here
  1913. we can't get NULL in normal conditions
  1914. */
  1915. DBUG_ASSERT(!is_var_null);
  1916. if (!var_value)
  1917. goto end;
  1918. }
  1919. else
  1920. {
  1921. /*
  1922. variable absent or equal to NULL, so we need to set variable to
  1923. something reasonable to get a readable error message during parsing
  1924. */
  1925. str.set(STRING_WITH_LEN("NULL"), &my_charset_latin1);
  1926. }
  1927. needs_conversion= String::needs_conversion(var_value->length(),
  1928. var_value->charset(), to_cs,
  1929. &unused);
  1930. len= (needs_conversion ? var_value->length() * to_cs->mbmaxlen :
  1931. var_value->length());
  1932. if (!(query_str= (char*) alloc_root(thd->mem_root, len+1)))
  1933. goto end;
  1934. if (needs_conversion)
  1935. {
  1936. uint dummy_errors;
  1937. len= copy_and_convert(query_str, len, to_cs, var_value->ptr(),
  1938. var_value->length(), var_value->charset(),
  1939. &dummy_errors);
  1940. }
  1941. else
  1942. memcpy(query_str, var_value->ptr(), var_value->length());
  1943. query_str[len]= '\0'; // Safety (mostly for debug)
  1944. *query_len= len;
  1945. }
  1946. else
  1947. {
  1948. query_str= lex->prepared_stmt_code.str;
  1949. *query_len= lex->prepared_stmt_code.length;
  1950. }
  1951. end:
  1952. return query_str;
  1953. }
  1954. /** Init PS/SP specific parse tree members. */
  1955. static void init_stmt_after_parse(LEX *lex)
  1956. {
  1957. SELECT_LEX *sl= lex->all_selects_list;
  1958. /*
  1959. Switch off a temporary flag that prevents evaluation of
  1960. subqueries in statement prepare.
  1961. */
  1962. for (; sl; sl= sl->next_select_in_list())
  1963. sl->uncacheable&= ~UNCACHEABLE_PREPARE;
  1964. }
  1965. /**
  1966. SQLCOM_PREPARE implementation.
  1967. Prepare an SQL prepared statement. This is called from
  1968. mysql_execute_command and should therefore behave like an
  1969. ordinary query (e.g. should not reset any global THD data).
  1970. @param thd thread handle
  1971. @return
  1972. none: in case of success, OK packet is sent to the client,
  1973. otherwise an error message is set in THD
  1974. */
  1975. void mysql_sql_stmt_prepare(THD *thd)
  1976. {
  1977. LEX *lex= thd->lex;
  1978. LEX_STRING *name= &lex->prepared_stmt_name;
  1979. Prepared_statement *stmt;
  1980. const char *query;
  1981. uint query_len= 0;
  1982. DBUG_ENTER("mysql_sql_stmt_prepare");
  1983. if ((stmt= (Prepared_statement*) thd->stmt_map.find_by_name(name)))
  1984. {
  1985. /*
  1986. If there is a statement with the same name, remove it. It is ok to
  1987. remove old and fail to insert a new one at the same time.
  1988. */
  1989. if (stmt->is_in_use())
  1990. {
  1991. my_error(ER_PS_NO_RECURSION, MYF(0));
  1992. DBUG_VOID_RETURN;
  1993. }
  1994. stmt->deallocate();
  1995. }
  1996. if (! (query= get_dynamic_sql_string(lex, &query_len)) ||
  1997. ! (stmt= new Prepared_statement(thd)))
  1998. {
  1999. DBUG_VOID_RETURN; /* out of memory */
  2000. }
  2001. stmt->set_sql_prepare();
  2002. /* Set the name first, insert should know that this statement has a name */
  2003. if (stmt->set_name(name))
  2004. {
  2005. delete stmt;
  2006. DBUG_VOID_RETURN;
  2007. }
  2008. if (thd->stmt_map.insert(thd, stmt))
  2009. {
  2010. /* The statement is deleted and an error is set if insert fails */
  2011. DBUG_VOID_RETURN;
  2012. }
  2013. if (stmt->prepare(query, query_len))
  2014. {
  2015. /* Statement map deletes the statement on erase */
  2016. thd->stmt_map.erase(stmt);
  2017. }
  2018. else
  2019. my_ok(thd, 0L, 0L, "Statement prepared");
  2020. DBUG_VOID_RETURN;
  2021. }
  2022. /**
  2023. Reinit prepared statement/stored procedure before execution.
  2024. @todo
  2025. When the new table structure is ready, then have a status bit
  2026. to indicate the table is altered, and re-do the setup_*
  2027. and open the tables back.
  2028. */
  2029. void reinit_stmt_before_use(THD *thd, LEX *lex)
  2030. {
  2031. SELECT_LEX *sl= lex->all_selects_list;
  2032. DBUG_ENTER("reinit_stmt_before_use");
  2033. /*
  2034. We have to update "thd" pointer in LEX, all its units and in LEX::result,
  2035. since statements which belong to trigger body are associated with TABLE
  2036. object and because of this can be used in different threads.
  2037. */
  2038. lex->thd= thd;
  2039. if (lex->empty_field_list_on_rset)
  2040. {
  2041. lex->empty_field_list_on_rset= 0;
  2042. lex->field_list.empty();
  2043. }
  2044. for (; sl; sl= sl->next_select_in_list())
  2045. {
  2046. if (!sl->first_execution)
  2047. {
  2048. /* remove option which was put by mysql_explain_union() */
  2049. sl->options&= ~SELECT_DESCRIBE;
  2050. /* see unique_table() */
  2051. sl->exclude_from_table_unique_test= FALSE;
  2052. /*
  2053. Copy WHERE, HAVING clause pointers to avoid damaging them
  2054. by optimisation
  2055. */
  2056. if (sl->prep_where)
  2057. {
  2058. sl->where= sl->prep_where->copy_andor_structure(thd);
  2059. sl->where->cleanup();
  2060. }
  2061. else
  2062. sl->where= NULL;
  2063. if (sl->prep_having)
  2064. {
  2065. sl->having= sl->prep_having->copy_andor_structure(thd);
  2066. sl->having->cleanup();
  2067. }
  2068. else
  2069. sl->having= NULL;
  2070. DBUG_ASSERT(sl->join == 0);
  2071. ORDER *order;
  2072. /* Fix GROUP list */
  2073. for (order= sl->group_list.first; order; order= order->next)
  2074. order->item= &order->item_ptr;
  2075. /* Fix ORDER list */
  2076. for (order= sl->order_list.first; order; order= order->next)
  2077. order->item= &order->item_ptr;
  2078. /* clear the no_error flag for INSERT/UPDATE IGNORE */
  2079. sl->no_error= FALSE;
  2080. }
  2081. {
  2082. SELECT_LEX_UNIT *unit= sl->master_unit();
  2083. unit->unclean();
  2084. unit->types.empty();
  2085. /* for derived tables & PS (which can't be reset by Item_subquery) */
  2086. unit->reinit_exec_mechanism();
  2087. unit->set_thd(thd);
  2088. }
  2089. }
  2090. /*
  2091. TODO: When the new table structure is ready, then have a status bit
  2092. to indicate the table is altered, and re-do the setup_*
  2093. and open the tables back.
  2094. */
  2095. /*
  2096. NOTE: We should reset whole table list here including all tables added
  2097. by prelocking algorithm (it is not a problem for substatements since
  2098. they have their own table list).
  2099. */
  2100. for (TABLE_LIST *tables= lex->query_tables;
  2101. tables;
  2102. tables= tables->next_global)
  2103. {
  2104. tables->reinit_before_use(thd);
  2105. }
  2106. /* Reset MDL tickets for procedures/functions */
  2107. for (Sroutine_hash_entry *rt=
  2108. (Sroutine_hash_entry*)thd->lex->sroutines_list.first;
  2109. rt; rt= rt->next)
  2110. rt->mdl_request.ticket= NULL;
  2111. /*
  2112. Cleanup of the special case of DELETE t1, t2 FROM t1, t2, t3 ...
  2113. (multi-delete). We do a full clean up, although at the moment all we
  2114. need to clean in the tables of MULTI-DELETE list is 'table' member.
  2115. */
  2116. for (TABLE_LIST *tables= lex->auxiliary_table_list.first;
  2117. tables;
  2118. tables= tables->next_global)
  2119. {
  2120. tables->reinit_before_use(thd);
  2121. }
  2122. lex->current_select= &lex->select_lex;
  2123. /* restore original list used in INSERT ... SELECT */
  2124. if (lex->leaf_tables_insert)
  2125. lex->select_lex.leaf_tables= lex->leaf_tables_insert;
  2126. if (lex->result)
  2127. {
  2128. lex->result->cleanup();
  2129. lex->result->set_thd(thd);
  2130. }
  2131. lex->allow_sum_func= 0;
  2132. lex->in_sum_func= NULL;
  2133. DBUG_VOID_RETURN;
  2134. }
  2135. /**
  2136. Clears parameters from data left from previous execution or long data.
  2137. @param stmt prepared statement for which parameters should
  2138. be reset
  2139. */
  2140. static void reset_stmt_params(Prepared_statement *stmt)
  2141. {
  2142. Item_param **item= stmt->param_array;
  2143. Item_param **end= item + stmt->param_count;
  2144. for (;item < end ; ++item)
  2145. (**item).reset();
  2146. }
  2147. /**
  2148. COM_STMT_EXECUTE handler: execute a previously prepared statement.
  2149. If there are any parameters, then replace parameter markers with the
  2150. data supplied from the client, and then execute the statement.
  2151. This function uses binary protocol to send a possible result set
  2152. to the client.
  2153. @param thd current thread
  2154. @param packet_arg parameter types and data, if any
  2155. @param packet_length packet length, including the terminator character.
  2156. @return
  2157. none: in case of success OK packet or a result set is sent to the
  2158. client, otherwise an error message is set in THD.
  2159. */
  2160. void mysqld_stmt_execute(THD *thd, char *packet_arg, uint packet_length)
  2161. {
  2162. uchar *packet= (uchar*)packet_arg; // GCC 4.0.1 workaround
  2163. ulong stmt_id= uint4korr(packet);
  2164. ulong flags= (ulong) packet[4];
  2165. /* Query text for binary, general or slow log, if any of them is open */
  2166. String expanded_query;
  2167. uchar *packet_end= packet + packet_length;
  2168. Prepared_statement *stmt;
  2169. Protocol *save_protocol= thd->protocol;
  2170. bool open_cursor;
  2171. DBUG_ENTER("mysqld_stmt_execute");
  2172. packet+= 9; /* stmt_id + 5 bytes of flags */
  2173. /* First of all clear possible warnings from the previous command */
  2174. mysql_reset_thd_for_next_command(thd);
  2175. if (!(stmt= find_prepared_statement(thd, stmt_id)))
  2176. {
  2177. char llbuf[22];
  2178. my_error(ER_UNKNOWN_STMT_HANDLER, MYF(0), sizeof(llbuf),
  2179. llstr(stmt_id, llbuf), "mysqld_stmt_execute");
  2180. DBUG_VOID_RETURN;
  2181. }
  2182. #if defined(ENABLED_PROFILING)
  2183. thd->profiling.set_query_source(stmt->query(), stmt->query_length());
  2184. #endif
  2185. DBUG_PRINT("exec_query", ("%s", stmt->query()));
  2186. DBUG_PRINT("info",("stmt: 0x%lx", (long) stmt));
  2187. open_cursor= test(flags & (ulong) CURSOR_TYPE_READ_ONLY);
  2188. thd->protocol= &thd->protocol_binary;
  2189. stmt->execute_loop(&expanded_query, open_cursor, packet, packet_end);
  2190. thd->protocol= save_protocol;
  2191. /* Close connection socket; for use with client testing (Bug#43560). */
  2192. DBUG_EXECUTE_IF("close_conn_after_stmt_execute", vio_close(thd->net.vio););
  2193. DBUG_VOID_RETURN;
  2194. }
  2195. /**
  2196. SQLCOM_EXECUTE implementation.
  2197. Execute prepared statement using parameter values from
  2198. lex->prepared_stmt_params and send result to the client using
  2199. text protocol. This is called from mysql_execute_command and
  2200. therefore should behave like an ordinary query (e.g. not change
  2201. global THD data, such as warning count, server status, etc).
  2202. This function uses text protocol to send a possible result set.
  2203. @param thd thread handle
  2204. @return
  2205. none: in case of success, OK (or result set) packet is sent to the
  2206. client, otherwise an error is set in THD
  2207. */
  2208. void mysql_sql_stmt_execute(THD *thd)
  2209. {
  2210. LEX *lex= thd->lex;
  2211. Prepared_statement *stmt;
  2212. LEX_STRING *name= &lex->prepared_stmt_name;
  2213. /* Query text for binary, general or slow log, if any of them is open */
  2214. String expanded_query;
  2215. DBUG_ENTER("mysql_sql_stmt_execute");
  2216. DBUG_PRINT("info", ("EXECUTE: %.*s\n", (int) name->length, name->str));
  2217. if (!(stmt= (Prepared_statement*) thd->stmt_map.find_by_name(name)))
  2218. {
  2219. my_error(ER_UNKNOWN_STMT_HANDLER, MYF(0),
  2220. name->length, name->str, "EXECUTE");
  2221. DBUG_VOID_RETURN;
  2222. }
  2223. if (stmt->param_count != lex->prepared_stmt_params.elements)
  2224. {
  2225. my_error(ER_WRONG_ARGUMENTS, MYF(0), "EXECUTE");
  2226. DBUG_VOID_RETURN;
  2227. }
  2228. DBUG_PRINT("info",("stmt: 0x%lx", (long) stmt));
  2229. (void) stmt->execute_loop(&expanded_query, FALSE, NULL, NULL);
  2230. DBUG_VOID_RETURN;
  2231. }
  2232. /**
  2233. COM_STMT_FETCH handler: fetches requested amount of rows from cursor.
  2234. @param thd Thread handle
  2235. @param packet Packet from client (with stmt_id & num_rows)
  2236. @param packet_length Length of packet
  2237. */
  2238. void mysqld_stmt_fetch(THD *thd, char *packet, uint packet_length)
  2239. {
  2240. /* assume there is always place for 8-16 bytes */
  2241. ulong stmt_id= uint4korr(packet);
  2242. ulong num_rows= uint4korr(packet+4);
  2243. Prepared_statement *stmt;
  2244. Statement stmt_backup;
  2245. Server_side_cursor *cursor;
  2246. DBUG_ENTER("mysqld_stmt_fetch");
  2247. /* First of all clear possible warnings from the previous command */
  2248. mysql_reset_thd_for_next_command(thd);
  2249. status_var_increment(thd->status_var.com_stmt_fetch);
  2250. if (!(stmt= find_prepared_statement(thd, stmt_id)))
  2251. {
  2252. char llbuf[22];
  2253. my_error(ER_UNKNOWN_STMT_HANDLER, MYF(0), sizeof(llbuf),
  2254. llstr(stmt_id, llbuf), "mysqld_stmt_fetch");
  2255. DBUG_VOID_RETURN;
  2256. }
  2257. cursor= stmt->cursor;
  2258. if (!cursor)
  2259. {
  2260. my_error(ER_STMT_HAS_NO_OPEN_CURSOR, MYF(0), stmt_id);
  2261. DBUG_VOID_RETURN;
  2262. }
  2263. thd->stmt_arena= stmt;
  2264. thd->set_n_backup_statement(stmt, &stmt_backup);
  2265. cursor->fetch(num_rows);
  2266. if (!cursor->is_open())
  2267. {
  2268. stmt->close_cursor();
  2269. reset_stmt_params(stmt);
  2270. }
  2271. thd->restore_backup_statement(stmt, &stmt_backup);
  2272. thd->stmt_arena= thd;
  2273. DBUG_VOID_RETURN;
  2274. }
  2275. /**
  2276. Reset a prepared statement in case there was a recoverable error.
  2277. This function resets statement to the state it was right after prepare.
  2278. It can be used to:
  2279. - clear an error happened during mysqld_stmt_send_long_data
  2280. - cancel long data stream for all placeholders without
  2281. having to call mysqld_stmt_execute.
  2282. - close an open cursor
  2283. Sends 'OK' packet in case of success (statement was reset)
  2284. or 'ERROR' packet (unrecoverable error/statement not found/etc).
  2285. @param thd Thread handle
  2286. @param packet Packet with stmt id
  2287. */
  2288. void mysqld_stmt_reset(THD *thd, char *packet)
  2289. {
  2290. /* There is always space for 4 bytes in buffer */
  2291. ulong stmt_id= uint4korr(packet);
  2292. Prepared_statement *stmt;
  2293. DBUG_ENTER("mysqld_stmt_reset");
  2294. /* First of all clear possible warnings from the previous command */
  2295. mysql_reset_thd_for_next_command(thd);
  2296. status_var_increment(thd->status_var.com_stmt_reset);
  2297. if (!(stmt= find_prepared_statement(thd, stmt_id)))
  2298. {
  2299. char llbuf[22];
  2300. my_error(ER_UNKNOWN_STMT_HANDLER, MYF(0), sizeof(llbuf),
  2301. llstr(stmt_id, llbuf), "mysqld_stmt_reset");
  2302. DBUG_VOID_RETURN;
  2303. }
  2304. stmt->close_cursor();
  2305. /*
  2306. Clear parameters from data which could be set by
  2307. mysqld_stmt_send_long_data() call.
  2308. */
  2309. reset_stmt_params(stmt);
  2310. stmt->state= Query_arena::PREPARED;
  2311. general_log_print(thd, thd->command, NullS);
  2312. my_ok(thd);
  2313. DBUG_VOID_RETURN;
  2314. }
  2315. /**
  2316. Delete a prepared statement from memory.
  2317. @note
  2318. we don't send any reply to this command.
  2319. */
  2320. void mysqld_stmt_close(THD *thd, char *packet)
  2321. {
  2322. /* There is always space for 4 bytes in packet buffer */
  2323. ulong stmt_id= uint4korr(packet);
  2324. Prepared_statement *stmt;
  2325. DBUG_ENTER("mysqld_stmt_close");
  2326. thd->stmt_da->disable_status();
  2327. if (!(stmt= find_prepared_statement(thd, stmt_id)))
  2328. DBUG_VOID_RETURN;
  2329. /*
  2330. The only way currently a statement can be deallocated when it's
  2331. in use is from within Dynamic SQL.
  2332. */
  2333. DBUG_ASSERT(! stmt->is_in_use());
  2334. stmt->deallocate();
  2335. general_log_print(thd, thd->command, NullS);
  2336. DBUG_VOID_RETURN;
  2337. }
  2338. /**
  2339. SQLCOM_DEALLOCATE implementation.
  2340. Close an SQL prepared statement. As this can be called from Dynamic
  2341. SQL, we should be careful to not close a statement that is currently
  2342. being executed.
  2343. @return
  2344. none: OK packet is sent in case of success, otherwise an error
  2345. message is set in THD
  2346. */
  2347. void mysql_sql_stmt_close(THD *thd)
  2348. {
  2349. Prepared_statement* stmt;
  2350. LEX_STRING *name= &thd->lex->prepared_stmt_name;
  2351. DBUG_PRINT("info", ("DEALLOCATE PREPARE: %.*s\n", (int) name->length,
  2352. name->str));
  2353. if (! (stmt= (Prepared_statement*) thd->stmt_map.find_by_name(name)))
  2354. my_error(ER_UNKNOWN_STMT_HANDLER, MYF(0),
  2355. name->length, name->str, "DEALLOCATE PREPARE");
  2356. else if (stmt->is_in_use())
  2357. my_error(ER_PS_NO_RECURSION, MYF(0));
  2358. else
  2359. {
  2360. stmt->deallocate();
  2361. my_ok(thd);
  2362. }
  2363. }
  2364. /**
  2365. Handle long data in pieces from client.
  2366. Get a part of a long data. To make the protocol efficient, we are
  2367. not sending any return packets here. If something goes wrong, then
  2368. we will send the error on 'execute' We assume that the client takes
  2369. care of checking that all parts are sent to the server. (No checking
  2370. that we get a 'end of column' in the server is performed).
  2371. @param thd Thread handle
  2372. @param packet String to append
  2373. @param packet_length Length of string (including end \\0)
  2374. */
  2375. void mysql_stmt_get_longdata(THD *thd, char *packet, ulong packet_length)
  2376. {
  2377. ulong stmt_id;
  2378. uint param_number;
  2379. Prepared_statement *stmt;
  2380. Item_param *param;
  2381. #ifndef EMBEDDED_LIBRARY
  2382. char *packet_end= packet + packet_length;
  2383. #endif
  2384. DBUG_ENTER("mysql_stmt_get_longdata");
  2385. status_var_increment(thd->status_var.com_stmt_send_long_data);
  2386. thd->stmt_da->disable_status();
  2387. #ifndef EMBEDDED_LIBRARY
  2388. /* Minimal size of long data packet is 6 bytes */
  2389. if (packet_length < MYSQL_LONG_DATA_HEADER)
  2390. DBUG_VOID_RETURN;
  2391. #endif
  2392. stmt_id= uint4korr(packet);
  2393. packet+= 4;
  2394. if (!(stmt=find_prepared_statement(thd, stmt_id)))
  2395. DBUG_VOID_RETURN;
  2396. param_number= uint2korr(packet);
  2397. packet+= 2;
  2398. #ifndef EMBEDDED_LIBRARY
  2399. if (param_number >= stmt->param_count)
  2400. {
  2401. /* Error will be sent in execute call */
  2402. stmt->state= Query_arena::ERROR;
  2403. stmt->last_errno= ER_WRONG_ARGUMENTS;
  2404. sprintf(stmt->last_error, ER(ER_WRONG_ARGUMENTS),
  2405. "mysqld_stmt_send_long_data");
  2406. DBUG_VOID_RETURN;
  2407. }
  2408. #endif
  2409. param= stmt->param_array[param_number];
  2410. #ifndef EMBEDDED_LIBRARY
  2411. if (param->set_longdata(packet, (ulong) (packet_end - packet)))
  2412. #else
  2413. if (param->set_longdata(thd->extra_data, thd->extra_length))
  2414. #endif
  2415. {
  2416. stmt->state= Query_arena::ERROR;
  2417. stmt->last_errno= ER_OUTOFMEMORY;
  2418. sprintf(stmt->last_error, ER(ER_OUTOFMEMORY), 0);
  2419. }
  2420. general_log_print(thd, thd->command, NullS);
  2421. DBUG_VOID_RETURN;
  2422. }
  2423. /***************************************************************************
  2424. Select_fetch_protocol_binary
  2425. ****************************************************************************/
  2426. Select_fetch_protocol_binary::Select_fetch_protocol_binary(THD *thd_arg)
  2427. :protocol(thd_arg)
  2428. {}
  2429. bool Select_fetch_protocol_binary::send_result_set_metadata(List<Item> &list, uint flags)
  2430. {
  2431. bool rc;
  2432. Protocol *save_protocol= thd->protocol;
  2433. /*
  2434. Protocol::send_result_set_metadata caches the information about column types:
  2435. this information is later used to send data. Therefore, the same
  2436. dedicated Protocol object must be used for all operations with
  2437. a cursor.
  2438. */
  2439. thd->protocol= &protocol;
  2440. rc= select_send::send_result_set_metadata(list, flags);
  2441. thd->protocol= save_protocol;
  2442. return rc;
  2443. }
  2444. bool Select_fetch_protocol_binary::send_eof()
  2445. {
  2446. ::my_eof(thd);
  2447. return FALSE;
  2448. }
  2449. bool
  2450. Select_fetch_protocol_binary::send_data(List<Item> &fields)
  2451. {
  2452. Protocol *save_protocol= thd->protocol;
  2453. bool rc;
  2454. thd->protocol= &protocol;
  2455. rc= select_send::send_data(fields);
  2456. thd->protocol= save_protocol;
  2457. return rc;
  2458. }
  2459. /*******************************************************************
  2460. * Reprepare_observer
  2461. *******************************************************************/
  2462. /** Push an error to the error stack and return TRUE for now. */
  2463. bool
  2464. Reprepare_observer::report_error(THD *thd)
  2465. {
  2466. /*
  2467. This 'error' is purely internal to the server:
  2468. - No exception handler is invoked,
  2469. - No condition is added in the condition area (warn_list).
  2470. The diagnostics area is set to an error status to enforce
  2471. that this thread execution stops and returns to the caller,
  2472. backtracking all the way to Prepared_statement::execute_loop().
  2473. */
  2474. thd->stmt_da->set_error_status(thd, ER_NEED_REPREPARE,
  2475. ER(ER_NEED_REPREPARE), "HY000");
  2476. m_invalidated= TRUE;
  2477. return TRUE;
  2478. }
  2479. /*******************************************************************
  2480. * Server_runnable
  2481. *******************************************************************/
  2482. Server_runnable::~Server_runnable()
  2483. {
  2484. }
  2485. ///////////////////////////////////////////////////////////////////////////
  2486. Execute_sql_statement::
  2487. Execute_sql_statement(LEX_STRING sql_text)
  2488. :m_sql_text(sql_text)
  2489. {}
  2490. /**
  2491. Parse and execute a statement. Does not prepare the query.
  2492. Allows to execute a statement from within another statement.
  2493. The main property of the implementation is that it does not
  2494. affect the environment -- i.e. you can run many
  2495. executions without having to cleanup/reset THD in between.
  2496. */
  2497. bool
  2498. Execute_sql_statement::execute_server_code(THD *thd)
  2499. {
  2500. bool error;
  2501. if (alloc_query(thd, m_sql_text.str, m_sql_text.length))
  2502. return TRUE;
  2503. Parser_state parser_state;
  2504. if (parser_state.init(thd, thd->query(), thd->query_length()))
  2505. return TRUE;
  2506. parser_state.m_lip.multi_statements= FALSE;
  2507. lex_start(thd);
  2508. error= parse_sql(thd, &parser_state, NULL) || thd->is_error();
  2509. if (error)
  2510. goto end;
  2511. thd->lex->set_trg_event_type_for_tables();
  2512. error= mysql_execute_command(thd);
  2513. /* report error issued during command execution */
  2514. if (error == 0 && thd->spcont == NULL)
  2515. general_log_write(thd, COM_STMT_EXECUTE,
  2516. thd->query(), thd->query_length());
  2517. end:
  2518. lex_end(thd->lex);
  2519. return error;
  2520. }
  2521. /***************************************************************************
  2522. Prepared_statement
  2523. ****************************************************************************/
  2524. Prepared_statement::Prepared_statement(THD *thd_arg)
  2525. :Statement(NULL, &main_mem_root,
  2526. INITIALIZED, ++thd_arg->statement_id_counter),
  2527. thd(thd_arg),
  2528. result(thd_arg),
  2529. param_array(0),
  2530. cursor(0),
  2531. param_count(0),
  2532. last_errno(0),
  2533. flags((uint) IS_IN_USE)
  2534. {
  2535. init_sql_alloc(&main_mem_root, thd_arg->variables.query_alloc_block_size,
  2536. thd_arg->variables.query_prealloc_size);
  2537. *last_error= '\0';
  2538. }
  2539. void Prepared_statement::setup_set_params()
  2540. {
  2541. /*
  2542. Note: BUG#25843 applies here too (query cache lookup uses thd->db, not
  2543. db from "prepare" time).
  2544. */
  2545. if (query_cache_maybe_disabled(thd)) // we won't expand the query
  2546. lex->safe_to_cache_query= FALSE; // so don't cache it at Execution
  2547. /*
  2548. Decide if we have to expand the query (because we must write it to logs or
  2549. because we want to look it up in the query cache) or not.
  2550. */
  2551. if ((mysql_bin_log.is_open() && is_update_query(lex->sql_command)) ||
  2552. opt_log || opt_slow_log ||
  2553. query_cache_is_cacheable_query(lex))
  2554. {
  2555. set_params_from_vars= insert_params_from_vars_with_log;
  2556. #ifndef EMBEDDED_LIBRARY
  2557. set_params= insert_params_with_log;
  2558. #else
  2559. set_params_data= emb_insert_params_with_log;
  2560. #endif
  2561. }
  2562. else
  2563. {
  2564. set_params_from_vars= insert_params_from_vars;
  2565. #ifndef EMBEDDED_LIBRARY
  2566. set_params= insert_params;
  2567. #else
  2568. set_params_data= emb_insert_params;
  2569. #endif
  2570. }
  2571. }
  2572. /**
  2573. Destroy this prepared statement, cleaning up all used memory
  2574. and resources.
  2575. This is called from ::deallocate() to handle COM_STMT_CLOSE and
  2576. DEALLOCATE PREPARE or when THD ends and all prepared statements are freed.
  2577. */
  2578. Prepared_statement::~Prepared_statement()
  2579. {
  2580. DBUG_ENTER("Prepared_statement::~Prepared_statement");
  2581. DBUG_PRINT("enter",("stmt: 0x%lx cursor: 0x%lx",
  2582. (long) this, (long) cursor));
  2583. delete cursor;
  2584. /*
  2585. We have to call free on the items even if cleanup is called as some items,
  2586. like Item_param, don't free everything until free_items()
  2587. */
  2588. free_items();
  2589. if (lex)
  2590. {
  2591. delete lex->result;
  2592. delete (st_lex_local *) lex;
  2593. }
  2594. free_root(&main_mem_root, MYF(0));
  2595. DBUG_VOID_RETURN;
  2596. }
  2597. Query_arena::Type Prepared_statement::type() const
  2598. {
  2599. return PREPARED_STATEMENT;
  2600. }
  2601. void Prepared_statement::cleanup_stmt()
  2602. {
  2603. DBUG_ENTER("Prepared_statement::cleanup_stmt");
  2604. DBUG_PRINT("enter",("stmt: 0x%lx", (long) this));
  2605. cleanup_items(free_list);
  2606. thd->cleanup_after_query();
  2607. thd->rollback_item_tree_changes();
  2608. DBUG_VOID_RETURN;
  2609. }
  2610. bool Prepared_statement::set_name(LEX_STRING *name_arg)
  2611. {
  2612. name.length= name_arg->length;
  2613. name.str= (char*) memdup_root(mem_root, name_arg->str, name_arg->length);
  2614. return name.str == 0;
  2615. }
  2616. /**
  2617. Remember the current database.
  2618. We must reset/restore the current database during execution of
  2619. a prepared statement since it affects execution environment:
  2620. privileges, @@character_set_database, and other.
  2621. @return Returns an error if out of memory.
  2622. */
  2623. bool
  2624. Prepared_statement::set_db(const char *db_arg, uint db_length_arg)
  2625. {
  2626. /* Remember the current database. */
  2627. if (db_arg && db_length_arg)
  2628. {
  2629. db= this->strmake(db_arg, db_length_arg);
  2630. db_length= db_length_arg;
  2631. }
  2632. else
  2633. {
  2634. db= NULL;
  2635. db_length= 0;
  2636. }
  2637. return db_arg != NULL && db == NULL;
  2638. }
  2639. /**************************************************************************
  2640. Common parts of mysql_[sql]_stmt_prepare, mysql_[sql]_stmt_execute.
  2641. Essentially, these functions do all the magic of preparing/executing
  2642. a statement, leaving network communication, input data handling and
  2643. global THD state management to the caller.
  2644. ***************************************************************************/
  2645. /**
  2646. Parse statement text, validate the statement, and prepare it for execution.
  2647. You should not change global THD state in this function, if at all
  2648. possible: it may be called from any context, e.g. when executing
  2649. a COM_* command, and SQLCOM_* command, or a stored procedure.
  2650. @param packet statement text
  2651. @param packet_len
  2652. @note
  2653. Precondition:
  2654. The caller must ensure that thd->change_list and thd->free_list
  2655. is empty: this function will not back them up but will free
  2656. in the end of its execution.
  2657. @note
  2658. Postcondition:
  2659. thd->mem_root contains unused memory allocated during validation.
  2660. */
  2661. bool Prepared_statement::prepare(const char *packet, uint packet_len)
  2662. {
  2663. bool error;
  2664. Statement stmt_backup;
  2665. Query_arena *old_stmt_arena;
  2666. MDL_ticket *mdl_savepoint= NULL;
  2667. DBUG_ENTER("Prepared_statement::prepare");
  2668. /*
  2669. If this is an SQLCOM_PREPARE, we also increase Com_prepare_sql.
  2670. However, it seems handy if com_stmt_prepare is increased always,
  2671. no matter what kind of prepare is processed.
  2672. */
  2673. status_var_increment(thd->status_var.com_stmt_prepare);
  2674. if (! (lex= new (mem_root) st_lex_local))
  2675. DBUG_RETURN(TRUE);
  2676. if (set_db(thd->db, thd->db_length))
  2677. DBUG_RETURN(TRUE);
  2678. /*
  2679. alloc_query() uses thd->memroot && thd->query, so we should call
  2680. both of backup_statement() and backup_query_arena() here.
  2681. */
  2682. thd->set_n_backup_statement(this, &stmt_backup);
  2683. thd->set_n_backup_active_arena(this, &stmt_backup);
  2684. if (alloc_query(thd, packet, packet_len))
  2685. {
  2686. thd->restore_backup_statement(this, &stmt_backup);
  2687. thd->restore_active_arena(this, &stmt_backup);
  2688. DBUG_RETURN(TRUE);
  2689. }
  2690. old_stmt_arena= thd->stmt_arena;
  2691. thd->stmt_arena= this;
  2692. Parser_state parser_state;
  2693. if (parser_state.init(thd, thd->query(), thd->query_length()))
  2694. {
  2695. thd->restore_backup_statement(this, &stmt_backup);
  2696. thd->restore_active_arena(this, &stmt_backup);
  2697. thd->stmt_arena= old_stmt_arena;
  2698. DBUG_RETURN(TRUE);
  2699. }
  2700. parser_state.m_lip.stmt_prepare_mode= TRUE;
  2701. parser_state.m_lip.multi_statements= FALSE;
  2702. lex_start(thd);
  2703. error= parse_sql(thd, & parser_state, NULL) ||
  2704. thd->is_error() ||
  2705. init_param_array(this);
  2706. lex->set_trg_event_type_for_tables();
  2707. /*
  2708. While doing context analysis of the query (in check_prepared_statement)
  2709. we allocate a lot of additional memory: for open tables, JOINs, derived
  2710. tables, etc. Let's save a snapshot of current parse tree to the
  2711. statement and restore original THD. In cases when some tree
  2712. transformation can be reused on execute, we set again thd->mem_root from
  2713. stmt->mem_root (see setup_wild for one place where we do that).
  2714. */
  2715. thd->restore_active_arena(this, &stmt_backup);
  2716. /*
  2717. If called from a stored procedure, ensure that we won't rollback
  2718. external changes when cleaning up after validation.
  2719. */
  2720. DBUG_ASSERT(thd->change_list.is_empty());
  2721. /*
  2722. Marker used to release metadata locks acquired while the prepared
  2723. statement is being checked.
  2724. */
  2725. mdl_savepoint= thd->mdl_context.mdl_savepoint();
  2726. /*
  2727. The only case where we should have items in the thd->free_list is
  2728. after stmt->set_params_from_vars(), which may in some cases create
  2729. Item_null objects.
  2730. */
  2731. if (error == 0)
  2732. error= check_prepared_statement(this);
  2733. /*
  2734. Currently CREATE PROCEDURE/TRIGGER/EVENT are prohibited in prepared
  2735. statements: ensure we have no memory leak here if by someone tries
  2736. to PREPARE stmt FROM "CREATE PROCEDURE ..."
  2737. */
  2738. DBUG_ASSERT(lex->sphead == NULL || error != 0);
  2739. /* The order is important */
  2740. lex->unit.cleanup();
  2741. /* No need to commit statement transaction, it's not started. */
  2742. DBUG_ASSERT(thd->transaction.stmt.is_empty());
  2743. close_thread_tables(thd);
  2744. thd->mdl_context.rollback_to_savepoint(mdl_savepoint);
  2745. lex_end(lex);
  2746. cleanup_stmt();
  2747. thd->restore_backup_statement(this, &stmt_backup);
  2748. thd->stmt_arena= old_stmt_arena;
  2749. if (error == 0)
  2750. {
  2751. setup_set_params();
  2752. init_stmt_after_parse(lex);
  2753. state= Query_arena::PREPARED;
  2754. flags&= ~ (uint) IS_IN_USE;
  2755. /*
  2756. Log COM_EXECUTE to the general log. Note, that in case of SQL
  2757. prepared statements this causes two records to be output:
  2758. Query PREPARE stmt from @user_variable
  2759. Prepare <statement SQL text>
  2760. This is considered user-friendly, since in the
  2761. second log entry we output the actual statement text.
  2762. Do not print anything if this is an SQL prepared statement and
  2763. we're inside a stored procedure (also called Dynamic SQL) --
  2764. sub-statements inside stored procedures are not logged into
  2765. the general log.
  2766. */
  2767. if (thd->spcont == NULL)
  2768. general_log_write(thd, COM_STMT_PREPARE, query(), query_length());
  2769. }
  2770. DBUG_RETURN(error);
  2771. }
  2772. /**
  2773. Assign parameter values either from variables, in case of SQL PS
  2774. or from the execute packet.
  2775. @param expanded_query a container with the original SQL statement.
  2776. '?' placeholders will be replaced with
  2777. their values in case of success.
  2778. The result is used for logging and replication
  2779. @param packet pointer to execute packet.
  2780. NULL in case of SQL PS
  2781. @param packet_end end of the packet. NULL in case of SQL PS
  2782. @todo Use a paremeter source class family instead of 'if's, and
  2783. support stored procedure variables.
  2784. @retval TRUE an error occurred when assigning a parameter (likely
  2785. a conversion error or out of memory, or malformed packet)
  2786. @retval FALSE success
  2787. */
  2788. bool
  2789. Prepared_statement::set_parameters(String *expanded_query,
  2790. uchar *packet, uchar *packet_end)
  2791. {
  2792. bool is_sql_ps= packet == NULL;
  2793. bool res= FALSE;
  2794. if (is_sql_ps)
  2795. {
  2796. /* SQL prepared statement */
  2797. res= set_params_from_vars(this, thd->lex->prepared_stmt_params,
  2798. expanded_query);
  2799. }
  2800. else if (param_count)
  2801. {
  2802. #ifndef EMBEDDED_LIBRARY
  2803. uchar *null_array= packet;
  2804. res= (setup_conversion_functions(this, &packet, packet_end) ||
  2805. set_params(this, null_array, packet, packet_end, expanded_query));
  2806. #else
  2807. /*
  2808. In embedded library we re-install conversion routines each time
  2809. we set parameters, and also we don't need to parse packet.
  2810. So we do it in one function.
  2811. */
  2812. res= set_params_data(this, expanded_query);
  2813. #endif
  2814. }
  2815. if (res)
  2816. {
  2817. my_error(ER_WRONG_ARGUMENTS, MYF(0),
  2818. is_sql_ps ? "EXECUTE" : "mysqld_stmt_execute");
  2819. reset_stmt_params(this);
  2820. }
  2821. return res;
  2822. }
  2823. /**
  2824. Execute a prepared statement. Re-prepare it a limited number
  2825. of times if necessary.
  2826. Try to execute a prepared statement. If there is a metadata
  2827. validation error, prepare a new copy of the prepared statement,
  2828. swap the old and the new statements, and try again.
  2829. If there is a validation error again, repeat the above, but
  2830. perform no more than MAX_REPREPARE_ATTEMPTS.
  2831. @note We have to try several times in a loop since we
  2832. release metadata locks on tables after prepared statement
  2833. prepare. Therefore, a DDL statement may sneak in between prepare
  2834. and execute of a new statement. If this happens repeatedly
  2835. more than MAX_REPREPARE_ATTEMPTS times, we give up.
  2836. @return TRUE if an error, FALSE if success
  2837. @retval TRUE either MAX_REPREPARE_ATTEMPTS has been reached,
  2838. or some general error
  2839. @retval FALSE successfully executed the statement, perhaps
  2840. after having reprepared it a few times.
  2841. */
  2842. bool
  2843. Prepared_statement::execute_loop(String *expanded_query,
  2844. bool open_cursor,
  2845. uchar *packet,
  2846. uchar *packet_end)
  2847. {
  2848. const int MAX_REPREPARE_ATTEMPTS= 3;
  2849. Reprepare_observer reprepare_observer;
  2850. bool error;
  2851. int reprepare_attempt= 0;
  2852. if (set_parameters(expanded_query, packet, packet_end))
  2853. return TRUE;
  2854. reexecute:
  2855. reprepare_observer.reset_reprepare_observer();
  2856. /*
  2857. If the free_list is not empty, we'll wrongly free some externally
  2858. allocated items when cleaning up after validation of the prepared
  2859. statement.
  2860. */
  2861. DBUG_ASSERT(thd->free_list == NULL);
  2862. /*
  2863. Install the metadata observer. If some metadata version is
  2864. different from prepare time and an observer is installed,
  2865. the observer method will be invoked to push an error into
  2866. the error stack.
  2867. */
  2868. if (sql_command_flags[lex->sql_command] &
  2869. CF_REEXECUTION_FRAGILE)
  2870. {
  2871. DBUG_ASSERT(thd->m_reprepare_observer == NULL);
  2872. thd->m_reprepare_observer = &reprepare_observer;
  2873. }
  2874. error= execute(expanded_query, open_cursor) || thd->is_error();
  2875. thd->m_reprepare_observer= NULL;
  2876. if (error && !thd->is_fatal_error && !thd->killed &&
  2877. reprepare_observer.is_invalidated() &&
  2878. reprepare_attempt++ < MAX_REPREPARE_ATTEMPTS)
  2879. {
  2880. DBUG_ASSERT(thd->stmt_da->sql_errno() == ER_NEED_REPREPARE);
  2881. thd->clear_error();
  2882. error= reprepare();
  2883. if (! error) /* Success */
  2884. goto reexecute;
  2885. }
  2886. reset_stmt_params(this);
  2887. return error;
  2888. }
  2889. bool
  2890. Prepared_statement::execute_server_runnable(Server_runnable *server_runnable)
  2891. {
  2892. Statement stmt_backup;
  2893. bool error;
  2894. Query_arena *save_stmt_arena= thd->stmt_arena;
  2895. Item_change_list save_change_list;
  2896. thd->change_list.move_elements_to(&save_change_list);
  2897. state= CONVENTIONAL_EXECUTION;
  2898. if (!(lex= new (mem_root) st_lex_local))
  2899. return TRUE;
  2900. thd->set_n_backup_statement(this, &stmt_backup);
  2901. thd->set_n_backup_active_arena(this, &stmt_backup);
  2902. thd->stmt_arena= this;
  2903. error= server_runnable->execute_server_code(thd);
  2904. thd->cleanup_after_query();
  2905. thd->restore_active_arena(this, &stmt_backup);
  2906. thd->restore_backup_statement(this, &stmt_backup);
  2907. thd->stmt_arena= save_stmt_arena;
  2908. save_change_list.move_elements_to(&thd->change_list);
  2909. /* Items and memory will freed in destructor */
  2910. return error;
  2911. }
  2912. /**
  2913. Reprepare this prepared statement.
  2914. Currently this is implemented by creating a new prepared
  2915. statement, preparing it with the original query and then
  2916. swapping the new statement and the original one.
  2917. @retval TRUE an error occurred. Possible errors include
  2918. incompatibility of new and old result set
  2919. metadata
  2920. @retval FALSE success, the statement has been reprepared
  2921. */
  2922. bool
  2923. Prepared_statement::reprepare()
  2924. {
  2925. char saved_cur_db_name_buf[NAME_LEN+1];
  2926. LEX_STRING saved_cur_db_name=
  2927. { saved_cur_db_name_buf, sizeof(saved_cur_db_name_buf) };
  2928. LEX_STRING stmt_db_name= { db, db_length };
  2929. bool cur_db_changed;
  2930. bool error;
  2931. Prepared_statement copy(thd);
  2932. copy.set_sql_prepare(); /* To suppress sending metadata to the client. */
  2933. status_var_increment(thd->status_var.com_stmt_reprepare);
  2934. if (mysql_opt_change_db(thd, &stmt_db_name, &saved_cur_db_name, TRUE,
  2935. &cur_db_changed))
  2936. return TRUE;
  2937. error= ((name.str && copy.set_name(&name)) ||
  2938. copy.prepare(query(), query_length()) ||
  2939. validate_metadata(&copy));
  2940. if (cur_db_changed)
  2941. mysql_change_db(thd, &saved_cur_db_name, TRUE);
  2942. if (! error)
  2943. {
  2944. swap_prepared_statement(&copy);
  2945. swap_parameter_array(param_array, copy.param_array, param_count);
  2946. #ifndef DBUG_OFF
  2947. is_reprepared= TRUE;
  2948. #endif
  2949. /*
  2950. Clear possible warnings during reprepare, it has to be completely
  2951. transparent to the user. We use clear_warning_info() since
  2952. there were no separate query id issued for re-prepare.
  2953. Sic: we can't simply silence warnings during reprepare, because if
  2954. it's failed, we need to return all the warnings to the user.
  2955. */
  2956. thd->warning_info->clear_warning_info(thd->query_id);
  2957. }
  2958. return error;
  2959. }
  2960. /**
  2961. Validate statement result set metadata (if the statement returns
  2962. a result set).
  2963. Currently we only check that the number of columns of the result
  2964. set did not change.
  2965. This is a helper method used during re-prepare.
  2966. @param[in] copy the re-prepared prepared statement to verify
  2967. the metadata of
  2968. @retval TRUE error, ER_PS_REBIND is reported
  2969. @retval FALSE statement return no or compatible metadata
  2970. */
  2971. bool Prepared_statement::validate_metadata(Prepared_statement *copy)
  2972. {
  2973. /**
  2974. If this is an SQL prepared statement or EXPLAIN,
  2975. return FALSE -- the metadata of the original SELECT,
  2976. if any, has not been sent to the client.
  2977. */
  2978. if (is_sql_prepare() || lex->describe)
  2979. return FALSE;
  2980. if (lex->select_lex.item_list.elements !=
  2981. copy->lex->select_lex.item_list.elements)
  2982. {
  2983. /** Column counts mismatch, update the client */
  2984. thd->server_status|= SERVER_STATUS_METADATA_CHANGED;
  2985. }
  2986. return FALSE;
  2987. }
  2988. /**
  2989. Replace the original prepared statement with a prepared copy.
  2990. This is a private helper that is used as part of statement
  2991. reprepare
  2992. @return This function does not return any errors.
  2993. */
  2994. void
  2995. Prepared_statement::swap_prepared_statement(Prepared_statement *copy)
  2996. {
  2997. Statement tmp_stmt;
  2998. /* Swap memory roots. */
  2999. swap_variables(MEM_ROOT, main_mem_root, copy->main_mem_root);
  3000. /* Swap the arenas */
  3001. tmp_stmt.set_query_arena(this);
  3002. set_query_arena(copy);
  3003. copy->set_query_arena(&tmp_stmt);
  3004. /* Swap the statement parent classes */
  3005. tmp_stmt.set_statement(this);
  3006. set_statement(copy);
  3007. copy->set_statement(&tmp_stmt);
  3008. /* Swap ids back, we need the original id */
  3009. swap_variables(ulong, id, copy->id);
  3010. /* Swap mem_roots back, they must continue pointing at the main_mem_roots */
  3011. swap_variables(MEM_ROOT *, mem_root, copy->mem_root);
  3012. /*
  3013. Swap the old and the new parameters array. The old array
  3014. is allocated in the old arena.
  3015. */
  3016. swap_variables(Item_param **, param_array, copy->param_array);
  3017. /* Don't swap flags: the copy has IS_SQL_PREPARE always set. */
  3018. /* swap_variables(uint, flags, copy->flags); */
  3019. /* Swap names, the old name is allocated in the wrong memory root */
  3020. swap_variables(LEX_STRING, name, copy->name);
  3021. /* Ditto */
  3022. swap_variables(char *, db, copy->db);
  3023. DBUG_ASSERT(db_length == copy->db_length);
  3024. DBUG_ASSERT(param_count == copy->param_count);
  3025. DBUG_ASSERT(thd == copy->thd);
  3026. last_error[0]= '\0';
  3027. last_errno= 0;
  3028. }
  3029. /**
  3030. Execute a prepared statement.
  3031. You should not change global THD state in this function, if at all
  3032. possible: it may be called from any context, e.g. when executing
  3033. a COM_* command, and SQLCOM_* command, or a stored procedure.
  3034. @param expanded_query A query for binlogging which has all parameter
  3035. markers ('?') replaced with their actual values.
  3036. @param open_cursor True if an attempt to open a cursor should be made.
  3037. Currenlty used only in the binary protocol.
  3038. @note
  3039. Preconditions, postconditions.
  3040. - See the comment for Prepared_statement::prepare().
  3041. @retval
  3042. FALSE ok
  3043. @retval
  3044. TRUE Error
  3045. */
  3046. bool Prepared_statement::execute(String *expanded_query, bool open_cursor)
  3047. {
  3048. Statement stmt_backup;
  3049. Query_arena *old_stmt_arena;
  3050. bool error= TRUE;
  3051. char saved_cur_db_name_buf[NAME_LEN+1];
  3052. LEX_STRING saved_cur_db_name=
  3053. { saved_cur_db_name_buf, sizeof(saved_cur_db_name_buf) };
  3054. bool cur_db_changed;
  3055. LEX_STRING stmt_db_name= { db, db_length };
  3056. status_var_increment(thd->status_var.com_stmt_execute);
  3057. /* Check if we got an error when sending long data */
  3058. if (state == Query_arena::ERROR)
  3059. {
  3060. my_message(last_errno, last_error, MYF(0));
  3061. return TRUE;
  3062. }
  3063. if (flags & (uint) IS_IN_USE)
  3064. {
  3065. my_error(ER_PS_NO_RECURSION, MYF(0));
  3066. return TRUE;
  3067. }
  3068. /*
  3069. For SHOW VARIABLES lex->result is NULL, as it's a non-SELECT
  3070. command. For such queries we don't return an error and don't
  3071. open a cursor -- the client library will recognize this case and
  3072. materialize the result set.
  3073. For SELECT statements lex->result is created in
  3074. check_prepared_statement. lex->result->simple_select() is FALSE
  3075. in INSERT ... SELECT and similar commands.
  3076. */
  3077. if (open_cursor && lex->result && lex->result->check_simple_select())
  3078. {
  3079. DBUG_PRINT("info",("Cursor asked for not SELECT stmt"));
  3080. return TRUE;
  3081. }
  3082. /* In case the command has a call to SP which re-uses this statement name */
  3083. flags|= IS_IN_USE;
  3084. close_cursor();
  3085. /*
  3086. If the free_list is not empty, we'll wrongly free some externally
  3087. allocated items when cleaning up after execution of this statement.
  3088. */
  3089. DBUG_ASSERT(thd->change_list.is_empty());
  3090. /*
  3091. The only case where we should have items in the thd->free_list is
  3092. after stmt->set_params_from_vars(), which may in some cases create
  3093. Item_null objects.
  3094. */
  3095. thd->set_n_backup_statement(this, &stmt_backup);
  3096. /*
  3097. Change the current database (if needed).
  3098. Force switching, because the database of the prepared statement may be
  3099. NULL (prepared statements can be created while no current database
  3100. selected).
  3101. */
  3102. if (mysql_opt_change_db(thd, &stmt_db_name, &saved_cur_db_name, TRUE,
  3103. &cur_db_changed))
  3104. goto error;
  3105. /* Allocate query. */
  3106. if (expanded_query->length() &&
  3107. alloc_query(thd, (char*) expanded_query->ptr(),
  3108. expanded_query->length()))
  3109. {
  3110. my_error(ER_OUTOFMEMORY, 0, expanded_query->length());
  3111. goto error;
  3112. }
  3113. /*
  3114. Expanded query is needed for slow logging, so we want thd->query
  3115. to point at it even after we restore from backup. This is ok, as
  3116. expanded query was allocated in thd->mem_root.
  3117. */
  3118. stmt_backup.set_query_inner(thd->query(), thd->query_length());
  3119. /*
  3120. At first execution of prepared statement we may perform logical
  3121. transformations of the query tree. Such changes should be performed
  3122. on the parse tree of current prepared statement and new items should
  3123. be allocated in its memory root. Set the appropriate pointer in THD
  3124. to the arena of the statement.
  3125. */
  3126. old_stmt_arena= thd->stmt_arena;
  3127. thd->stmt_arena= this;
  3128. reinit_stmt_before_use(thd, lex);
  3129. /* Go! */
  3130. if (open_cursor)
  3131. error= mysql_open_cursor(thd, &result, &cursor);
  3132. else
  3133. {
  3134. /*
  3135. Try to find it in the query cache, if not, execute it.
  3136. Note that multi-statements cannot exist here (they are not supported in
  3137. prepared statements).
  3138. */
  3139. if (query_cache_send_result_to_client(thd, thd->query(),
  3140. thd->query_length()) <= 0)
  3141. {
  3142. MYSQL_QUERY_EXEC_START(thd->query(),
  3143. thd->thread_id,
  3144. (char *) (thd->db ? thd->db : ""),
  3145. thd->security_ctx->priv_user,
  3146. (char *) thd->security_ctx->host_or_ip,
  3147. 1);
  3148. error= mysql_execute_command(thd);
  3149. MYSQL_QUERY_EXEC_DONE(error);
  3150. }
  3151. }
  3152. /*
  3153. Restore the current database (if changed).
  3154. Force switching back to the saved current database (if changed),
  3155. because it may be NULL. In this case, mysql_change_db() would generate
  3156. an error.
  3157. */
  3158. if (cur_db_changed)
  3159. mysql_change_db(thd, &saved_cur_db_name, TRUE);
  3160. /* Assert that if an error, no cursor is open */
  3161. DBUG_ASSERT(! (error && cursor));
  3162. if (! cursor)
  3163. cleanup_stmt();
  3164. thd->set_statement(&stmt_backup);
  3165. thd->stmt_arena= old_stmt_arena;
  3166. if (state == Query_arena::PREPARED)
  3167. state= Query_arena::EXECUTED;
  3168. if (error == 0 && this->lex->sql_command == SQLCOM_CALL)
  3169. {
  3170. if (is_sql_prepare())
  3171. thd->protocol_text.send_out_parameters(&this->lex->param_list);
  3172. else
  3173. thd->protocol->send_out_parameters(&this->lex->param_list);
  3174. }
  3175. /*
  3176. Log COM_EXECUTE to the general log. Note, that in case of SQL
  3177. prepared statements this causes two records to be output:
  3178. Query EXECUTE <statement name>
  3179. Execute <statement SQL text>
  3180. This is considered user-friendly, since in the
  3181. second log entry we output values of parameter markers.
  3182. Do not print anything if this is an SQL prepared statement and
  3183. we're inside a stored procedure (also called Dynamic SQL) --
  3184. sub-statements inside stored procedures are not logged into
  3185. the general log.
  3186. */
  3187. if (error == 0 && thd->spcont == NULL)
  3188. general_log_write(thd, COM_STMT_EXECUTE, thd->query(), thd->query_length());
  3189. error:
  3190. flags&= ~ (uint) IS_IN_USE;
  3191. return error;
  3192. }
  3193. /** Common part of DEALLOCATE PREPARE and mysqld_stmt_close. */
  3194. void Prepared_statement::deallocate()
  3195. {
  3196. /* We account deallocate in the same manner as mysqld_stmt_close */
  3197. status_var_increment(thd->status_var.com_stmt_close);
  3198. /* Statement map calls delete stmt on erase */
  3199. thd->stmt_map.erase(this);
  3200. }
  3201. /***************************************************************************
  3202. * Ed_result_set
  3203. ***************************************************************************/
  3204. /**
  3205. Use operator delete to free memory of Ed_result_set.
  3206. Accessing members of a class after the class has been destroyed
  3207. is a violation of the C++ standard but is commonly used in the
  3208. server code.
  3209. */
  3210. void Ed_result_set::operator delete(void *ptr, size_t size) throw ()
  3211. {
  3212. if (ptr)
  3213. {
  3214. /*
  3215. Make a stack copy, otherwise free_root() will attempt to
  3216. write to freed memory.
  3217. */
  3218. MEM_ROOT own_root= ((Ed_result_set*) ptr)->m_mem_root;
  3219. free_root(&own_root, MYF(0));
  3220. }
  3221. }
  3222. /**
  3223. Initialize an instance of Ed_result_set.
  3224. Instances of the class, as well as all result set rows, are
  3225. always allocated in the memory root passed over as the second
  3226. argument. In the constructor, we take over ownership of the
  3227. memory root. It will be freed when the class is destroyed.
  3228. sic: Ed_result_est is not designed to be allocated on stack.
  3229. */
  3230. Ed_result_set::Ed_result_set(List<Ed_row> *rows_arg,
  3231. size_t column_count_arg,
  3232. MEM_ROOT *mem_root_arg)
  3233. :m_mem_root(*mem_root_arg),
  3234. m_column_count(column_count_arg),
  3235. m_rows(rows_arg),
  3236. m_next_rset(NULL)
  3237. {
  3238. /* Take over responsibility for the memory */
  3239. clear_alloc_root(mem_root_arg);
  3240. }
  3241. /***************************************************************************
  3242. * Ed_result_set
  3243. ***************************************************************************/
  3244. /**
  3245. Create a new "execute direct" connection.
  3246. */
  3247. Ed_connection::Ed_connection(THD *thd)
  3248. :m_warning_info(thd->query_id),
  3249. m_thd(thd),
  3250. m_rsets(0),
  3251. m_current_rset(0)
  3252. {
  3253. }
  3254. /**
  3255. Free all result sets of the previous statement, if any,
  3256. and reset warnings and errors.
  3257. Called before execution of the next query.
  3258. */
  3259. void
  3260. Ed_connection::free_old_result()
  3261. {
  3262. while (m_rsets)
  3263. {
  3264. Ed_result_set *rset= m_rsets->m_next_rset;
  3265. delete m_rsets;
  3266. m_rsets= rset;
  3267. }
  3268. m_current_rset= m_rsets;
  3269. m_diagnostics_area.reset_diagnostics_area();
  3270. m_warning_info.clear_warning_info(m_thd->query_id);
  3271. }
  3272. /**
  3273. A simple wrapper that uses a helper class to execute SQL statements.
  3274. */
  3275. bool
  3276. Ed_connection::execute_direct(LEX_STRING sql_text)
  3277. {
  3278. Execute_sql_statement execute_sql_statement(sql_text);
  3279. DBUG_PRINT("ed_query", ("%s", sql_text.str));
  3280. return execute_direct(&execute_sql_statement);
  3281. }
  3282. /**
  3283. Execute a fragment of server functionality without an effect on
  3284. thd, and store results in memory.
  3285. Conventions:
  3286. - the code fragment must finish with OK, EOF or ERROR.
  3287. - the code fragment doesn't have to close thread tables,
  3288. free memory, commit statement transaction or do any other
  3289. cleanup that is normally done in the end of dispatch_command().
  3290. @param server_runnable A code fragment to execute.
  3291. */
  3292. bool Ed_connection::execute_direct(Server_runnable *server_runnable)
  3293. {
  3294. bool rc= FALSE;
  3295. Protocol_local protocol_local(m_thd, this);
  3296. Prepared_statement stmt(m_thd);
  3297. Protocol *save_protocol= m_thd->protocol;
  3298. Diagnostics_area *save_diagnostics_area= m_thd->stmt_da;
  3299. Warning_info *save_warning_info= m_thd->warning_info;
  3300. DBUG_ENTER("Ed_connection::execute_direct");
  3301. free_old_result(); /* Delete all data from previous execution, if any */
  3302. m_thd->protocol= &protocol_local;
  3303. m_thd->stmt_da= &m_diagnostics_area;
  3304. m_thd->warning_info= &m_warning_info;
  3305. rc= stmt.execute_server_runnable(server_runnable);
  3306. m_thd->protocol->end_statement();
  3307. m_thd->protocol= save_protocol;
  3308. m_thd->stmt_da= save_diagnostics_area;
  3309. m_thd->warning_info= save_warning_info;
  3310. /*
  3311. Protocol_local makes use of m_current_rset to keep
  3312. track of the last result set, while adding result sets to the end.
  3313. Reset it to point to the first result set instead.
  3314. */
  3315. m_current_rset= m_rsets;
  3316. DBUG_RETURN(rc);
  3317. }
  3318. /**
  3319. A helper method that is called only during execution.
  3320. Although Ed_connection doesn't support multi-statements,
  3321. a statement may generate many result sets. All subsequent
  3322. result sets are appended to the end.
  3323. @pre This is called only by Protocol_local.
  3324. */
  3325. void
  3326. Ed_connection::add_result_set(Ed_result_set *ed_result_set)
  3327. {
  3328. if (m_rsets)
  3329. {
  3330. m_current_rset->m_next_rset= ed_result_set;
  3331. /* While appending, use m_current_rset as a pointer to the tail. */
  3332. m_current_rset= ed_result_set;
  3333. }
  3334. else
  3335. m_current_rset= m_rsets= ed_result_set;
  3336. }
  3337. /**
  3338. Release ownership of the current result set to the client.
  3339. Since we use a simple linked list for result sets,
  3340. this method uses a linear search of the previous result
  3341. set to exclude the released instance from the list.
  3342. @todo Use double-linked list, when this is really used.
  3343. XXX: This has never been tested with more than one result set!
  3344. @pre There must be a result set.
  3345. */
  3346. Ed_result_set *
  3347. Ed_connection::store_result_set()
  3348. {
  3349. Ed_result_set *ed_result_set;
  3350. DBUG_ASSERT(m_current_rset);
  3351. if (m_current_rset == m_rsets)
  3352. {
  3353. /* Assign the return value */
  3354. ed_result_set= m_current_rset;
  3355. /* Exclude the return value from the list. */
  3356. m_current_rset= m_rsets= m_rsets->m_next_rset;
  3357. }
  3358. else
  3359. {
  3360. Ed_result_set *prev_rset= m_rsets;
  3361. /* Assign the return value. */
  3362. ed_result_set= m_current_rset;
  3363. /* Exclude the return value from the list */
  3364. while (prev_rset->m_next_rset != m_current_rset)
  3365. prev_rset= ed_result_set->m_next_rset;
  3366. m_current_rset= prev_rset->m_next_rset= m_current_rset->m_next_rset;
  3367. }
  3368. ed_result_set->m_next_rset= NULL; /* safety */
  3369. return ed_result_set;
  3370. }
  3371. /*************************************************************************
  3372. * Protocol_local
  3373. **************************************************************************/
  3374. Protocol_local::Protocol_local(THD *thd, Ed_connection *ed_connection)
  3375. :Protocol(thd),
  3376. m_connection(ed_connection),
  3377. m_rset(NULL),
  3378. m_column_count(0),
  3379. m_current_row(NULL),
  3380. m_current_column(NULL)
  3381. {
  3382. clear_alloc_root(&m_rset_root);
  3383. }
  3384. /**
  3385. Called between two result set rows.
  3386. Prepare structures to fill result set rows.
  3387. Unfortunately, we can't return an error here. If memory allocation
  3388. fails, we'll have to return an error later. And so is done
  3389. in methods such as @sa store_column().
  3390. */
  3391. void Protocol_local::prepare_for_resend()
  3392. {
  3393. DBUG_ASSERT(alloc_root_inited(&m_rset_root));
  3394. opt_add_row_to_rset();
  3395. /* Start a new row. */
  3396. m_current_row= (Ed_column *) alloc_root(&m_rset_root,
  3397. sizeof(Ed_column) * m_column_count);
  3398. m_current_column= m_current_row;
  3399. }
  3400. /**
  3401. In "real" protocols this is called to finish a result set row.
  3402. Unused in the local implementation.
  3403. */
  3404. bool Protocol_local::write()
  3405. {
  3406. return FALSE;
  3407. }
  3408. /**
  3409. A helper function to add the current row to the current result
  3410. set. Called in @sa prepare_for_resend(), when a new row is started,
  3411. and in send_eof(), when the result set is finished.
  3412. */
  3413. void Protocol_local::opt_add_row_to_rset()
  3414. {
  3415. if (m_current_row)
  3416. {
  3417. /* Add the old row to the result set */
  3418. Ed_row *ed_row= new (&m_rset_root) Ed_row(m_current_row, m_column_count);
  3419. if (ed_row)
  3420. m_rset->push_back(ed_row, &m_rset_root);
  3421. }
  3422. }
  3423. /**
  3424. Add a NULL column to the current row.
  3425. */
  3426. bool Protocol_local::store_null()
  3427. {
  3428. if (m_current_column == NULL)
  3429. return TRUE; /* prepare_for_resend() failed to allocate memory. */
  3430. bzero(m_current_column, sizeof(*m_current_column));
  3431. ++m_current_column;
  3432. return FALSE;
  3433. }
  3434. /**
  3435. A helper method to add any column to the current row
  3436. in its binary form.
  3437. Allocates memory for the data in the result set memory root.
  3438. */
  3439. bool Protocol_local::store_column(const void *data, size_t length)
  3440. {
  3441. if (m_current_column == NULL)
  3442. return TRUE; /* prepare_for_resend() failed to allocate memory. */
  3443. /*
  3444. alloc_root() automatically aligns memory, so we don't need to
  3445. do any extra alignment if we're pointing to, say, an integer.
  3446. */
  3447. m_current_column->str= (char*) memdup_root(&m_rset_root,
  3448. data,
  3449. length + 1 /* Safety */);
  3450. if (! m_current_column->str)
  3451. return TRUE;
  3452. m_current_column->str[length]= '\0'; /* Safety */
  3453. m_current_column->length= length;
  3454. ++m_current_column;
  3455. return FALSE;
  3456. }
  3457. /**
  3458. Store a string value in a result set column, optionally
  3459. having converted it to character_set_results.
  3460. */
  3461. bool
  3462. Protocol_local::store_string(const char *str, size_t length,
  3463. CHARSET_INFO *src_cs, CHARSET_INFO *dst_cs)
  3464. {
  3465. /* Store with conversion */
  3466. uint error_unused;
  3467. if (dst_cs && !my_charset_same(src_cs, dst_cs) &&
  3468. src_cs != &my_charset_bin &&
  3469. dst_cs != &my_charset_bin)
  3470. {
  3471. if (convert->copy(str, length, src_cs, dst_cs, &error_unused))
  3472. return TRUE;
  3473. str= convert->ptr();
  3474. length= convert->length();
  3475. }
  3476. return store_column(str, length);
  3477. }
  3478. /** Store a tiny int as is (1 byte) in a result set column. */
  3479. bool Protocol_local::store_tiny(longlong value)
  3480. {
  3481. char v= (char) value;
  3482. return store_column(&v, 1);
  3483. }
  3484. /** Store a short as is (2 bytes, host order) in a result set column. */
  3485. bool Protocol_local::store_short(longlong value)
  3486. {
  3487. int16 v= (int16) value;
  3488. return store_column(&v, 2);
  3489. }
  3490. /** Store a "long" as is (4 bytes, host order) in a result set column. */
  3491. bool Protocol_local::store_long(longlong value)
  3492. {
  3493. int32 v= (int32) value;
  3494. return store_column(&v, 4);
  3495. }
  3496. /** Store a "longlong" as is (8 bytes, host order) in a result set column. */
  3497. bool Protocol_local::store_longlong(longlong value, bool unsigned_flag)
  3498. {
  3499. int64 v= (int64) value;
  3500. return store_column(&v, 8);
  3501. }
  3502. /** Store a decimal in string format in a result set column */
  3503. bool Protocol_local::store_decimal(const my_decimal *value)
  3504. {
  3505. char buf[DECIMAL_MAX_STR_LENGTH];
  3506. String str(buf, sizeof (buf), &my_charset_bin);
  3507. int rc;
  3508. rc= my_decimal2string(E_DEC_FATAL_ERROR, value, 0, 0, 0, &str);
  3509. if (rc)
  3510. return TRUE;
  3511. return store_column(str.ptr(), str.length());
  3512. }
  3513. /** Convert to cs_results and store a string. */
  3514. bool Protocol_local::store(const char *str, size_t length,
  3515. CHARSET_INFO *src_cs)
  3516. {
  3517. CHARSET_INFO *dst_cs;
  3518. dst_cs= m_connection->m_thd->variables.character_set_results;
  3519. return store_string(str, length, src_cs, dst_cs);
  3520. }
  3521. /** Store a string. */
  3522. bool Protocol_local::store(const char *str, size_t length,
  3523. CHARSET_INFO *src_cs, CHARSET_INFO *dst_cs)
  3524. {
  3525. return store_string(str, length, src_cs, dst_cs);
  3526. }
  3527. /* Store MYSQL_TIME (in binary format) */
  3528. bool Protocol_local::store(MYSQL_TIME *time)
  3529. {
  3530. return store_column(time, sizeof(MYSQL_TIME));
  3531. }
  3532. /** Store MYSQL_TIME (in binary format) */
  3533. bool Protocol_local::store_date(MYSQL_TIME *time)
  3534. {
  3535. return store_column(time, sizeof(MYSQL_TIME));
  3536. }
  3537. /** Store MYSQL_TIME (in binary format) */
  3538. bool Protocol_local::store_time(MYSQL_TIME *time)
  3539. {
  3540. return store_column(time, sizeof(MYSQL_TIME));
  3541. }
  3542. /* Store a floating point number, as is. */
  3543. bool Protocol_local::store(float value, uint32 decimals, String *buffer)
  3544. {
  3545. return store_column(&value, sizeof(float));
  3546. }
  3547. /* Store a double precision number, as is. */
  3548. bool Protocol_local::store(double value, uint32 decimals, String *buffer)
  3549. {
  3550. return store_column(&value, sizeof (double));
  3551. }
  3552. /* Store a Field. */
  3553. bool Protocol_local::store(Field *field)
  3554. {
  3555. if (field->is_null())
  3556. return store_null();
  3557. return field->send_binary(this);
  3558. }
  3559. /** Called to start a new result set. */
  3560. bool Protocol_local::send_result_set_metadata(List<Item> *columns, uint)
  3561. {
  3562. DBUG_ASSERT(m_rset == 0 && !alloc_root_inited(&m_rset_root));
  3563. init_sql_alloc(&m_rset_root, MEM_ROOT_BLOCK_SIZE, 0);
  3564. if (! (m_rset= new (&m_rset_root) List<Ed_row>))
  3565. return TRUE;
  3566. m_column_count= columns->elements;
  3567. return FALSE;
  3568. }
  3569. /**
  3570. Normally this is a separate result set with OUT parameters
  3571. of stored procedures. Currently unsupported for the local
  3572. version.
  3573. */
  3574. bool Protocol_local::send_out_parameters(List<Item_param> *sp_params)
  3575. {
  3576. return FALSE;
  3577. }
  3578. /** Called for statements that don't have a result set, at statement end. */
  3579. bool
  3580. Protocol_local::send_ok(uint server_status, uint statement_warn_count,
  3581. ulonglong affected_rows, ulonglong last_insert_id,
  3582. const char *message)
  3583. {
  3584. /*
  3585. Just make sure nothing is sent to the client, we have grabbed
  3586. the status information in the connection diagnostics area.
  3587. */
  3588. return FALSE;
  3589. }
  3590. /**
  3591. Called at the end of a result set. Append a complete
  3592. result set to the list in Ed_connection.
  3593. Don't send anything to the client, but instead finish
  3594. building of the result set at hand.
  3595. */
  3596. bool Protocol_local::send_eof(uint server_status, uint statement_warn_count)
  3597. {
  3598. Ed_result_set *ed_result_set;
  3599. DBUG_ASSERT(m_rset);
  3600. opt_add_row_to_rset();
  3601. m_current_row= 0;
  3602. ed_result_set= new (&m_rset_root) Ed_result_set(m_rset, m_column_count,
  3603. &m_rset_root);
  3604. m_rset= NULL;
  3605. if (! ed_result_set)
  3606. return TRUE;
  3607. /* In case of successful allocation memory ownership was transferred. */
  3608. DBUG_ASSERT(!alloc_root_inited(&m_rset_root));
  3609. /*
  3610. Link the created Ed_result_set instance into the list of connection
  3611. result sets. Never fails.
  3612. */
  3613. m_connection->add_result_set(ed_result_set);
  3614. return FALSE;
  3615. }
  3616. /** Called to send an error to the client at the end of a statement. */
  3617. bool
  3618. Protocol_local::send_error(uint sql_errno, const char *err_msg, const char*)
  3619. {
  3620. /*
  3621. Just make sure that nothing is sent to the client (default
  3622. implementation).
  3623. */
  3624. return FALSE;
  3625. }
  3626. #ifdef EMBEDDED_LIBRARY
  3627. void Protocol_local::remove_last_row()
  3628. { }
  3629. #endif