mariadb

Tree: 783b774837

Commit Graph

Author	SHA1	Message	Date
unknown	dca006dfd2	Fix for BUG#735 "Prepared Statements: there is no support for Query Cache". WL#1569 "Prepared Statements: implement support of Query Cache". Prepared SELECTs did not look up in the query cache, and their results were not stored in the query cache. This made them slower than non-prepared SELECTs in some cases. The fix is to re-use the expanded query (the prepared query where "?" placeholders are replaced by their values, at execution time) for searching/storing in the query cache. It works fine for statements prepared via mysql_stmt_prepare(), which are the most commonly used and were the scope of this bugfix and WL. It works less fine for statements prepared via the SQL command PREPARE...FROM, which are still not using the query cache if they have at least one parameter (because then the expanded query contains names of user variables, and user variables don't work with the query cache, even in non-prepared queries). Note that results from prepared SELECTs, which are in the binary protocol, and results from normal SELECTs, which are in the text protocol, ignore each other in the query cache, because a result in the binary protocol should never be served to a SELECT expecting the text protocol and vice-versa. Note, after this patch, bug 25843 starts applying to query cache ("changing default database between PREPARE and EXECUTE of statement breaks binlog"), we need to fix it. mysql-test/include/have_query_cache.inc: Now prepared statements work with the query cache, so don't disable prep stmts by default when doing a query cache test. All tests which include this file will now be really tested against prepared statements (in particular, query_cache.test). mysql-test/r/query_cache.result: result update mysql-test/t/grant_cache.test: Cannot enable this test in ps-protocol, because in normal protocol, a SELECT failing due to insufficient privileges increments Qcache_not_cached, while in ps-protocol, no. In detail: in normal protocol, the "access denied" errors on SELECT are issued at (stack trace): mysql_parse/mysql_execute_command/execute_sqlcom_select/handle_select/ mysql_select/JOIN::prepare/setup_wild/insert_fields/ check_grant_all_columns/my_error/my_message_sql, which then calls push_warning/query_cache_abort: at this moment, query_cache_store_query() has been called, so query exists in cache, so thd->net.query_cache_query!=NULL, so query_cache_abort() removes the query from cache, which causes a query_cache.refused++ (thus, a Qcache_not_cached++). While in ps-protocol, the error is issued at prepare time; for this mysql_test_select() is called, not execute_sqlcom_select() (and that also leads to JOIN::prepare/etc). Thus, as query_cache_store_query() has not been called, thd->net.query_cache_query==NULL, so query_cache_abort() does nothing: Qcache_not_cached is not incremented. As this test prints Qcache_not_cached after SELECT failures, we cannot enable this test in ps-protocol. mysql-test/t/ndb_cache_multi2.test: The principle of this test is: two mysqlds connected to one cluster, both using their query cache. Queries are cached in server1 ("select a!=3 from t1", "select * from t1"), table t1 is modified in server2, we want to see that this invalidates the query cache of server1. Invalidation with NDB works like this: when a query is found in the query cache, NDB is asked if the tables have changed. In this test, ha_ndbcluster calls NDB every millisecond to collect change information about tables. Due to this millisecond delay, there is need for a loop ("while...") in this test, which waits until a query1 ("select a!=3 from t1") is invalidated (which is equivalent to it returning up-to-date results), and then expects query2 ("select * from t1") to have been invalidated (see up-to-date results). But when enabling --ps-protocol in this test, the logic breaks, because query1 is still done via mysql_real_query() (see mysqltest.c: eval_expr() always uses mysql_real_query()). So, query1 returning up-to-date results is not a sign of it being invalidated in the cache, because it was NOT in the cache ("select a!=3 from t1" on line 39 was done with prep stmts, while `select a!=3 from t1` is not, thus the second does not see the first in the cache). Thus, we may run query2 when cache still has not been invalidated. The solution is to make the initial "select a!=3 from t1" run as a normal query, this repairs the broken logic. But note, "select * from t1" is still using prepared statements which was the goal of this test with --ps-protocol. mysql-test/t/query_cache.test: now that prepared statements work with the query cache, we check that results in binary protocol (prepared statements) and in text protocol (normal queries) don't mix in the query cache even though the text of the statement/query are identical. sql/mysql_priv.h: In class Query_cache_flags, we add a bit to say if the result is in binary or text format (because, a result in binary format should never be served to a query expecting text format, and vice- versa). A macro to emphasize that we read the size of the query cache without mutex ("maybe" word). A macro which gives a first indication of if a query is cache-able (first indication - it does not consider the query cache's state). sql/protocol.cc: indentation. sql/protocol.h: Children classes of Protocol report their type (currently, text or binary). Query cache needs to know that. sql/sql_cache.cc: When we store a result in the query cache, we need to remember if it's in binary or text format. And when we search for a result in the query cache, we need to select only those results which are in the format which the current statement expects (binary or text). sql/sql_prepare.cc: Enabling use of the query cache by prepared statements. 1) Prep stmts are of two types: a) prepared via the mysql_stmt_prepare() API call b) prepared via the SQL PREPARE...FROM statement. 2) We already, when executing a prepared statement, sometimes built an "expanded" statement. For a), "?" placeholders were replaced by their values. For b), by names of the user variables containing the values. We did that only when we needed to write the query to logs. We now use this expanded query also for storing/searching in the query cache. Assume a query "SELECT * FROM T WHERE c=?", and the parameter is 10. For a), the expanded query is "SELECT * FROM T WHERE c=10", we look for "SELECT * FROM T WHERE c=10" in the query cache, and store that query's result in the query cache. For b), the expanded query is "SELECT * FROM T WHERE c=@somevar", and user variables don't work with the query cache (even inside non- prepared queries), so we don't enable query caching for SQL PREPARE'd statements if they have at least one parameter (see "if (stmt->param_count > 0)" in the patch). 3) If query cache is enabled and this is a SELECT, we build the expanded query (as an optimisation, we don't want to build this expanded query if the query cache is disabled or this is not a SELECT). As the decision of building the expanded query or not is taken at prepare time (setup_set_params()), if query cache is disabled at prepare time, we won't build the expanded query at all next executions, thus shouldn't use this query for query cacheing. To ensure that we don't, we set safe_to_cache_query to FALSE. Note that we read the size of the query cache without mutex, which is ok: if we see it 0 but that cache has just been enlarged, no big deal, just our statement will not use the query cache; if we see it >0 but that cache has just been made destroyed, we'll build the expanded query at all executions, but query_cache_store_query() and query_cache_send_result_to_client() will read the size with a mutex and so properly decide to cache or not cache. 4) Some functions in this file were named "withlog", others "with_log", now using "with_log" for all. tests/mysql_client_test.c: Testing of how prepared statements enter and hit the query cache. test_ps_query_cache() is inspired from test_ps_conj_select(). It creates data, a prepared SELECT statement, executes it once, then a second time with the same parameter value, to see that cache is hit, then a 3rd time with another parameter value to see that cache is not hit. Then, same from another connection, expecting hits. Then, tests border cases (enables query cache at prepare and disables at execute and vice-versa). It checks all results of SELECTs, cache hits and misses. mysql-test/r/query_cache_sql_prepare.result: result of new test: we see hits when there is no parameter, no hit when there is a parameter. mysql-test/t/query_cache_sql_prepare.test: new test to see if SQL PREPARE'd statements enter/hit the query cache: - if having at least one parameter, they should not - if having zero parameters, they should.	19 years ago

Author

SHA1

Message

Date

unknown

dca006dfd2

Fix for BUG#735 "Prepared Statements: there is no support for Query

Cache".
WL#1569 "Prepared Statements: implement support of Query Cache".
Prepared SELECTs did not look up in the query cache, and their results
were not stored in the query cache. This made them slower than
non-prepared SELECTs in some cases.
The fix is to re-use the expanded query (the prepared query where
"?" placeholders are replaced by their values, at execution time)
for searching/storing in the query cache.
It works fine for statements prepared via mysql_stmt_prepare(), which
are the most commonly used and were the scope of this bugfix and WL.
It works less fine for statements prepared via the SQL command
PREPARE...FROM, which are still not using the query cache if they
have at least one parameter (because then the expanded query contains
names of user variables, and user variables don't work with the
query cache, even in non-prepared queries).
Note that results from prepared SELECTs, which are in the binary
protocol, and results from normal SELECTs, which are in the text
protocol, ignore each other in the query cache, because a result in the
binary protocol should never be served to a SELECT expecting the text
protocol and vice-versa.
Note, after this patch, bug 25843 starts applying to query cache
("changing default database between PREPARE and EXECUTE of statement
breaks binlog"), we need to fix it.


mysql-test/include/have_query_cache.inc:
  Now prepared statements work with the query cache, so don't disable
  prep stmts by default when doing a query cache test. All tests which
  include this file will now be really tested against prepared
  statements (in particular, query_cache.test).
mysql-test/r/query_cache.result:
  result update
mysql-test/t/grant_cache.test:
  Cannot enable this test in ps-protocol, because in normal protocol,
  a SELECT failing due to insufficient privileges increments
  Qcache_not_cached, while in ps-protocol, no.
  In detail: in normal protocol,
  the "access denied" errors on SELECT are issued at (stack trace):
  mysql_parse/mysql_execute_command/execute_sqlcom_select/handle_select/
  mysql_select/JOIN::prepare/setup_wild/insert_fields/
  check_grant_all_columns/my_error/my_message_sql, which then calls
  push_warning/query_cache_abort: at this moment,
  query_cache_store_query() has been called, so query exists in cache,
  so thd->net.query_cache_query!=NULL, so query_cache_abort() removes
  the query from cache, which causes a query_cache.refused++ (thus,
  a Qcache_not_cached++).
  While in ps-protocol, the error is issued at prepare time;
  for this mysql_test_select() is called, not execute_sqlcom_select()
  (and that also leads to JOIN::prepare/etc). Thus, as
  query_cache_store_query() has not been called,
  thd->net.query_cache_query==NULL, so query_cache_abort() does nothing:
  Qcache_not_cached is not incremented.
  As this test prints Qcache_not_cached after SELECT failures,
  we cannot enable this test in ps-protocol.
mysql-test/t/ndb_cache_multi2.test:
  The principle of this test is: two mysqlds connected to one cluster,
  both using their query cache. Queries are cached in server1
  ("select a!=3 from t1", "select * from t1"),
  table t1 is modified in server2, we want to see that this invalidates
  the query cache of server1. Invalidation with NDB works like this:
  when a query is found in the query cache, NDB is asked if the tables
  have changed. In this test, ha_ndbcluster calls NDB every millisecond
  to collect change information about tables.
  Due to this millisecond delay, there is need for a loop ("while...")
  in this test, which waits until a query1 ("select a!=3 from t1") is
  invalidated (which is equivalent to it returning
  up-to-date results), and then expects query2 ("select * from t1")
  to have been invalidated (see up-to-date results).
  But when enabling --ps-protocol in this test, the logic breaks,
  because query1 is still done via mysql_real_query() (see mysqltest.c:
  eval_expr() always uses mysql_real_query()). So, query1 returning
  up-to-date results is not a sign of it being invalidated in the cache,
  because it was NOT in the cache ("select a!=3 from t1" on line 39
  was done with prep stmts, while `select a!=3 from t1` is not,
  thus the second does not see the first in the cache). Thus, we may run
  query2 when cache still has not been invalidated.
  The solution is to make the initial "select a!=3 from t1" run
  as a normal query, this repairs the broken logic.
  But note, "select * from t1" is still using prepared statements
  which was the goal of this test with --ps-protocol.
mysql-test/t/query_cache.test:
  now that prepared statements work with the query cache, we check
  that results in binary protocol (prepared statements) and in text
  protocol (normal queries) don't mix in the query cache even though
  the text of the statement/query are identical.
sql/mysql_priv.h:
  In class Query_cache_flags, we add a bit to say if the result
  is in binary or text format (because, a result in binary format
  should never be served to a query expecting text format, and vice-
  versa).
  A macro to emphasize that we read the size of the query cache
  without mutex ("maybe" word).
  A macro which gives a first indication of if a query is cache-able
  (first indication - it does not consider the query cache's state).
sql/protocol.cc:
  indentation.
sql/protocol.h:
  Children classes of Protocol report their type (currently,
  text or binary). Query cache needs to know that.
sql/sql_cache.cc:
  When we store a result in the query cache, we need to remember if it's
  in binary or text format. And when we search for a result in the query
  cache, we need to select only those results which are in the format
  which the current statement expects (binary or text).
sql/sql_prepare.cc:
  Enabling use of the query cache by prepared statements.
  1) Prep stmts are of two types:
    a) prepared via the mysql_stmt_prepare() API call
    b) prepared via the SQL PREPARE...FROM statement.
  2) We already, when executing a prepared statement, sometimes built an
  "expanded" statement. For a), "?" placeholders were replaced by their
  values. For b), by names of the user variables containing the values.
  We did that only when we needed to write the query to logs.
  We now use this expanded query also for storing/searching
  in the query cache.
  Assume a query "SELECT * FROM T WHERE c=?", and the parameter is 10.
  For a), the expanded query is "SELECT * FROM T WHERE c=10", we look
  for "SELECT * FROM T WHERE c=10" in the query cache, and store that
  query's result in the query cache.
  For b), the expanded query is "SELECT * FROM T WHERE c=@somevar", and
  user variables don't work with the query cache (even inside non-
  prepared queries), so we don't enable query caching for SQL PREPARE'd
  statements if they have at least one parameter (see
  "if (stmt->param_count > 0)" in the patch).
  3) If query cache is enabled and this is a SELECT, we build the
  expanded query (as an optimisation, we don't want to build this
  expanded query if the query cache is disabled or this is not a SELECT).
  As the decision of building the expanded query or not is taken
  at prepare time (setup_set_params()), if query cache is disabled
  at prepare time, we won't build the expanded query at all next
  executions, thus shouldn't use this query for query cacheing.
  To ensure that we don't, we set safe_to_cache_query to FALSE.
  Note that we read the size of the query cache without mutex, which is
  ok: if we see it 0 but that cache has just been enlarged, no big deal,
  just our statement will not use the query cache; if we see it >0 but
  that cache has just been made destroyed, we'll build the expanded
  query at all executions, but query_cache_store_query() and
  query_cache_send_result_to_client() will read the size with a mutex
  and so properly decide to cache or not cache.
  4) Some functions in this file were named "withlog", others "with_log",
  now using "with_log" for all.
tests/mysql_client_test.c:
  Testing of how prepared statements enter and hit the query cache.
  test_ps_query_cache() is inspired from test_ps_conj_select().
  It creates data, a prepared SELECT statement, executes it once,
  then a second time with the same parameter value, to see that cache
  is hit, then a 3rd time with another parameter value to see that cache
  is not hit. Then, same from another connection, expecting hits.
  Then, tests border cases (enables query cache at prepare and disables
  at execute and vice-versa).
  It checks all results of SELECTs, cache hits and misses.
mysql-test/r/query_cache_sql_prepare.result:
  result of new test: we see hits when there is no parameter,
  no hit when there is a parameter.
mysql-test/t/query_cache_sql_prepare.test:
  new test to see if SQL PREPARE'd statements enter/hit the query cache:
  - if having at least one parameter, they should not
  - if having zero parameters, they should.

19 years ago

1 Commits (783b7748370880c8ba4ff80e76d27209efb369f6)