mariadb

Commit Graph

Author	SHA1	Message	Date
Igor Babaev	cfd2646c31	Fixed the results after the merge of 10.4 into bb-10.4-mdev16188.	7 years ago
Igor Babaev	658128af43	MDEV-16188 Use in-memory PK filters built from range index scans This patch contains a full implementation of the optimization that allows to use in-memory rowid / primary filters built for range conditions over indexes. In many cases usage of such filters reduce the number of disk seeks spent for fetching table rows. In this implementation the choice of what possible filter to be applied (if any) is made purely on cost-based considerations. This implementation re-achitectured the partial implementation of the feature pushed by Galina Shalygina in the commit `8d5a11122c`. Besides this patch contains a better implementation of the generic handler function handler::multi_range_read_info_const() that takes into account gaps between ranges when calculating the cost of range index scans. It also contains some corrections of the implementation of the handler function records_in_range() for MyISAM. This patch supports the feature for InnoDB and MyISAM.	7 years ago
Varun Gupta	93c360e3a5	MDEV-15253: Default optimizer setting changes for MariaDB 10.4 use_stat_tables= PREFERABLY optimizer_use_condition_selectivity= 4	8 years ago
Oleksandr Byelkin	de745ecf29	MDEV-11953: support of brackets in UNION/EXCEPT/INTERSECT operations	8 years ago
Galina Shalygina	d3ff133390	MDEV-12387 Push conditions into materialized subqueries The logic and the implementation scheme are similar with the MDEV-9197 Pushdown conditions into non-mergeable views/derived tables How the push down is made on the example: select * from t1 where a>3 and b>10 and (a,b) in (select x,max(y) from t2 group by x); --> select * from t1 where a>3 and b>10 and (a,b) in (select x,max(y) from t2 where x>3 group by x having max(y)>10); The implementation scheme: 1. Search for the condition cond that depends only on the fields from the left part of the IN subquery (left_part) 2. Find fields F_group in the select of the right part of the IN subquery (right_part) that are used in the GROUP BY 3. Extract from the cond condition cond_where that depends only on the fields from the left_part that stay at the same places in the left_part (have the same indexes) as the F_group fields in the projection of the right_part 4. Transform cond_where so it can be pushed into the WHERE clause of the right_part and delete cond_where from the cond 5. Transform cond so it can be pushed into the HAVING clause of the right_part The optimization is made in the Item_in_subselect::pushdown_cond_for_in_subquery() and is controlled by the variable condition_pushdown_for_subquery. New test file in_subq_cond_pushdown.test is created. There are also some changes made for setup_jtbm_semi_joins(). Now it is decomposed into the 2 procedures: setup_degenerate_jtbm_semi_joins() that is called before optimize_cond() for cond and setup_jtbm_semi_joins() that is called after optimize_cond(). New setup_jtbm_semi_joins() is made in the way so that the result of its work is the same as if it was called before optimize_cond(). The code that is common for pushdown into materialized derived and into materialized IN subqueries is factored out into pushdown_cond_for_derived(), Item_in_subselect::pushdown_cond_for_in_subquery() and st_select_lex::pushdown_cond_into_where_clause().	8 years ago
Michael Widenius	a7abddeffa	Create 'main' test directory and move 't' and 'r' there	8 years ago
Varun Gupta	926edd48e1	MDEV-15235: Assertion `length > 0' failed in create_ref_for_key The issue is that we are creating a materialised table with key of length 0 which is incorrect, we should disable materialisation for such a case.	8 years ago
Monty	6660708523	Changed "const row not found" to "Const row not found" Also changed Unique row not found to use uppercase first. This was done to make these messages consistent with the rest	8 years ago
Igor Babaev	c258ca2463	Fixed the bug mdev-12838. If the optimizer chose an execution plan where a semi-join nest were materialized and the result of materialization was scanned to access other tables by ref access it could build a key over columns of the tables from the nest that were actually inaccessible. The patch performs a proper check whether a key that uses columns of the tables from a materialized semi-join nest can be employed to access outer tables.	9 years ago
Igor Babaev	97fb1f2679	Fixed bug mdev-10053. The implementation of the walk method for the class Item_in_subselect was missing. As a result the method never traversed the left operand of any IN subquery predicate. Item_exists_subselect::exists2in_processor() that performs the Exist-To-In transformation calls the walk method to collect info on outer references. As the walk method did not traverse the left operands of the IN subqueries the outer references there were not taken into account and some subqueries that were actually correlated were marked as uncorrelated. It could lead to an attempt of the materialization of such a subquery. Also added a cleanup for some test cases merged from 5.5.	9 years ago
Igor Babaev	b0395d8701	Fixed the bug mdev-12429 and its duplicates mdev-12145 and mdev-9886. Also fixed a wrong result for a test case for mdev-7691 (the alternative one). The test cases for all these bug have materialized semi-joins used inside dependent sub-queries. The patch actually reverts the change inroduced by Monty in 2003. It looks like this change is not valid anymore after the implementation of semi-joins. Adjusted output from EXPLAIN for many other test cases.	9 years ago
Oleksandr Byelkin	05d3c3d3f7	MDEV-10141: Add support for INTERSECT (and common parts for EXCEPT) MDEV-10140: Add support for EXCEPT	9 years ago
Oleksandr Byelkin	7aa09a5ed2	MDEV-10141: Add support for INTERSECT (and common parts for EXCEPT) MDEV-10140: Add support for EXCEPT	9 years ago
Nirbhay Choubey	8b2e642aa2	MDEV-7635: Update tests to adapt to the new default sql_mode	9 years ago
Sergei Golubchik	180065ebb0	Item::print(): remove redundant parentheses by introducing new Item::precedence() method and using it to decide whether parentheses are required	9 years ago
Oleksandr Byelkin	a02642b66e	MDEV-10017: Get unexpected `Empty Set` for correlated subquery with aggregate functions (part 1) Make aggregate function dependency visible.	9 years ago
Monty	6f31dd093a	Added new status variables to make it easier to debug certain problems: - Handler_read_retry - Update_scan - Delete_scan	9 years ago
Sergei Petrunia	a63ceaeab1	MDEV-10389: Query returns different results on a debug vs non-debug build The problem was caused by a merged semi-join, which contained a non-merged semi-join, which used references to the top-level query in the left_expr. When moving non-merged semi-join from the subquery to its parent, do not forget to call fix_after_pullout for its Item_subselect. We need to do that specifically, because non-merged semi-joins do not have their IN-equality in the WHERE clause at this stage.	9 years ago
Oleksandr Byelkin	6c414fcf89	MDEV-5542: GROUP_CONCAT truncate output to 65.536 chars when using DISTINCT or ORDER BY port of mysql fix WL#6098	10 years ago
Oleksandr Byelkin	4b08b10b3a	MDEV-9489:Assertion `0' failed in Protocol::end_statement() on UNION ALL Restoring currect_select fixed.	10 years ago
Alexander Barkov	3cc6e5b7f2	MDEV-8852 Implicit or explicit CAST from MAX(string) to INT,DOUBLE,DECIMAL does not produce warnings	10 years ago
Sergei Golubchik	4d5772c578	MDEV-7810 Wrong result on execution of a query as a PS (both 1st and further executions) Alternative fix that doesn't cause view.test crash in --ps: Remember when Item_ref was fixed right in the constructor and did not have a full Item_ref::fix_fields() call. Later in PS/SP, after Item_ref::cleanup, we use this knowledge to avoid doing full fix_fields() for items that were never supposed to be fix_field'ed. Simplify the test case.	10 years ago
Sergei Petrunia	c4456b99b3	MDEV-7971: Assertion `name != __null' failed in ACL_internal_schema_registry::lookup... [Attempt #] Make the code that handles "Prepare" phase for multi-table UPDATE statements handle non-merged semijoins. It can encounter them when a prepared statement is executed for the second time.	10 years ago
Monty	8c815751c9	Problem was that for cases like: SELECT ... WHERE XX IN (SELECT YY) this was transformed to something like: SELECT ... WHERE IF_EXISTS(SELECT ... HAVING XX=YY) The bug was that for normal execution XX was fixed in the original outer SELECT context while in PS it was fixed in the sub query context and this confused the optimizer. Fixed by ensuring that XX is always fixed in the outer context.	10 years ago
Sergei Petrunia	775528ada3	MDEV-7220: Materialization strategy is not used for REPLACE ... SELECT Enable subquery materialization for non-SELECT queries with a SELECT part	11 years ago
Sergei Petrunia	c945233a19	MDEV-6657: Poor plan choice for ORDER BY key DESC optimization... The problem was caused by the following scenario: - range optimizer picks an index IDX1 which doesn't match the ORDER BY ... LIMIT clause. - test_if_skip_sort_order() decides to switch to index IDX2 which matches the ORDER BY ... LIMIT. - it runs SQL_SELECT::test_quick_select() for the second time to produce an quick select for IDX2. - However, test_quick_select() would figure that full index scan on IDX1 is still cheaper (its calculations ignore the LIMIT n). Fixed this by - passing force_quick_range=true to test_quick_select() - in test_quick_select, don't consider full index scans if the mentioned parameter is true. Numerous changes in .result files are caused by test_quick_select() being run after "early/late NULLs filtering" feature has injected NOT NULL condition.	11 years ago
Sergei Golubchik	1c616869d1	fix the test to pass on windows (lower_case_file_system)	11 years ago
Sergey Petrunya	6b353dd1de	MDEV-6289 : Unexpected results when querying information_schema - When traversing JOIN_TABs with first_linear_tab/next_linear_tab(), don't forget to enter the semi-join nest when it is the first table in the join order. Failure to do so could cause e.g. I_S tables not to be filled.	11 years ago
Sergey Petrunya	8c04dd33dd	MDEV-5811: Server crashes in best_access_path with materialization+semijoin and big_tables=ON - With big_tables=ON, materialized table will use Aria (or MyISAM) SE, which allows prefix key reads. However, the temp.table has rec_per_key=NULL which causes the optimizer to crash when attempting to read index statistics for a prefix index read. - Fixed by providing a rec_per_key array with zeros (i.e. "no statistics data")	12 years ago
Sergey Petrunya	519c7305ac	MDEV-5368: Server crashes in Item_in_subselect::optimize on ... - convert_subq_to_sj() must connect child select's tables into parent select's TABLE_LIST::next_local chain. - The problem was that it took child's leaf_tables.head() which is different. This could cause certain tables (in this bug's case, child select's non-merged semi-join) not to be present in TABLE_LIST::next_local chain. Which would cause non-merged semi-join not to be initialized in setup_tables(), which would lead to NULL pointer dereference.	12 years ago
Sergey Petrunya	f0fa66a26d	MDEV-5368: Server crashes in Item_in_subselect::optimize on ... - convert_subq_to_sj() must connect child select's tables into parent select's TABLE_LIST::next_local chain. - The problem was that it took child's leaf_tables.head() which is different. This could cause certain tables (in this bug's case, child select's non-merged semi-join) not to be present in TABLE_LIST::next_local chain. Which would cause non-merged semi-join not to be initialized in setup_tables(), which would lead to NULL pointer dereference.	12 years ago
Sergey Petrunya	229aa1d4bf	MDEV-5056: Wrong result (extra rows) with materialization+semijoin, IN subqueries Apply fix suggested by Igor: - When eliminate_item_equal() generates pair-wise equalities from a multi-equality, do generate a "bridge" equality between the first field inside SJM nest and the field that's first in the overall multi-equality.	12 years ago
timour@askmonty.org	afed809297	MDEV-5123 Remove duplicated conditions pushed both to join_tab->select_cond and join_tab->cache_select->cond for blocked joins. BNL and BNLH joins pre-filter the records from a joined table via JOIN_TAB::cache_select->cond. There is no need to re-evaluate the same conditions via JOIN_TAB::select_cond. This patch removes the duplicated conditions from the top-level conjuncts of each pushed condition. The added "Using where" in few EXPLAINs is due to taking into account tab->cache_select->cond in addition to tab->select_cond in JOIN::save_explain_data_intern.	12 years ago
Sergey Petrunya	7e4845beea	MDEV-5011: ERROR Plugin 'MEMORY' has ref_count=1 after shutdown for SJM queries - Provide a special execution path for cleanup of degenerate non-merged semi-join children of degenerate selects.	12 years ago
unknown	f8af4423b4	MDEV-4908: Assertion `((Item_cond ) cond)->functype() == ((Item_cond ) new_item)->functype()' fails on a query with IN and equal conditions, AND/OR, materialization+semijoin A new AND Item should be prepared (fix_field() call) before using.	12 years ago
Sergey Petrunya	6ed00c4d8a	MDEV-4465: Reproducible crash (mysqld got signal 11) in multi_delete::initialize_tables... - make multi_delete::initialize_tables() take into account that the JOIN structure may have semi-join nests (which are not fully initialized when this function is called, they have tab->table=NULL which caused the crash) - Also checked multi_update::initialize_tables(): it has a different logic and needed no fixing.	13 years ago
Igor Babaev	c9b63e6a49	Fixed bug mdev-3913. The wrong result set returned by the left join query from the bug test case happened due to several inconsistencies and bugs of the legacy mysql code. The bug test case uses an execution plan that employs a scan of a materialized IN subquery from the WHERE condition. When materializing such an IN- subquery the optimizer injects additional equalities into the WHERE clause. These equalities express the constraints imposed by the subquery predicate. The injected equality of the query in the test case happens to belong to the same equality class, and a new equality imposing a condition on the rows of the materialized subquery is inferred from this class. Simultaneously the multiple equality is added to the ON expression of the LEFT JOIN used in the main query. The inferred equality of the form f1=f2 is taken into account when optimizing the scan of the rows the temporary table that is the result of the subquery materialization: only the values of the field f1 are read from the table into the record buffer. Meanwhile the inferred equality is removed from the WHERE conditions altogether as a constraint on the fields of the temporary table that has been used when filling this table. This equality is supposed to be removed from the ON expression when the multiple equalities of the ON expression are converted into an optimal set of equality predicates. It supposed to be removed from the ON expression as an equality inferred from only equalities of the WHERE condition. Yet, it did not happened due to the following bug in the code. Erroneously the code tried to build multiple equality for ON expression twice: the first time, when it called optimize_cond() for the WHERE condition, the second time, when it called this function for the HAVING condition. When executing optimize_con() for the WHERE condition a reference to the multiple equality of the WHERE condition is set in the multiple equality of the ON expression. This reference would allow later to convert multiple equalities of the ON expression into equality predicates. However the the second call of build_equal_items() for the ON expression that happened when optimize_cond() was called for the HAVING condition reset this reference to NULL. This bug fix blocks calling build_equal_items() for ON expressions for the second time. In general, it will be beneficial for many queries as it removes from ON expressions any equalities that are to be checked for the WHERE condition. The patch also fixes two bugs in the list manipulation operations and a bug in the function substitute_for_best_equal_field() that resulted in passing wrong reference to the multiple equalities of where conditions when processing multiple equalities of ON expressions. The code of substitute_for_best_equal_field() and the code the helper function eliminate_item_equal() were also streamlined and cleaned up. Now the conversion of the multiple equalities into an optimal set of equality predicates first produces the sequence of the all equalities processing multiple equalities one by one, and, only after this, it inserts the equalities at the beginning of the other conditions. The multiple changes in the output of EXPLAIN EXTENDED are mainly the result of this streamlining, but in some cases is the result of the removal of unneeded equalities from ON expressions. In some test cases this removal were reflected in the output of EXPLAIN resulted in disappearance of “Using where” in some rows of the execution plans.	13 years ago
Sergey Petrunya	55597a4869	MDEV-410: EXPLAIN shows type=range, while SHOW EXPLAIN and userstat show full table scan is used - Make Item_subselect::fix_fields() ignore UNCACHEABLE_EXPLAIN flag when deciding whether the subquery item should be marked as constant.	13 years ago
unknown	f1ab00891a	Fixed bug lp:1000649 Analysis: When the method JOIN::choose_subquery_plan() decided to apply the IN-TO-EXISTS strategy, it set the unit and select_lex uncacheable flag to UNCACHEABLE_DEPENDENT_INJECTED unconditionally. As result, even if IN-TO-EXISTS injected non-correlated predicates, the subquery was still treated as correlated. Solution: Set the subquery as correlated only if the injected predicate(s) depend on the outer query.	14 years ago
unknown	941018f8d1	Patch for mdev-287: CHEAP SQ: A query with subquery in SELECT list, EXISTS, inner joins takes hundreds times longer Analysis: The fix for lp:944706 introduces early subquery optimization. While a subquery is being optimized some of its predicates may be removed. In the test case, the EXISTS subquery is constant, and is evaluated to TRUE. As a result the whole OR is TRUE, and thus the correlated condition "b = alias1.b" is optimized away. The subquery becomes non-correlated. The subquery cache is designed to work only for correlated subqueries. If constant subquery optimization is disallowed, then the constant subquery is not evaluated, the subquery remains correlated, and its execution is cached. As a result execution is fast. However, when the constant subquery was optimized away, it was neither cached by the subquery cache, nor it was cached by the internal subquery caching. The latter was due to the fact that the subquery still appeared as correlated to the subselect_XYZ_engine::exec methods, and they re-executed the subquery on each call to Item_subselect::exec. Solution: The solution is to update the correlated status of the subquery after it has been optimized. This status consists of: - st_select_lex::is_correlated - Item_subselect::is_correlated - SELECT_LEX::uncacheable - SELECT_LEX_UNIT::uncacheable The status is updated by st_select_lex::update_correlated_cache(), and its caller st_select_lex::optimize_unflattened_subqueries. The solution relies on the fact that the optimizer already called st_select_lex::update_used_tables() for each subquery. This allows to efficiently update the correlated status of each subquery without walking the whole subquery tree. Notice that his patch is an improvement over MySQL 5.6 and older, where subqueries are not pre-optimized, and the above analysis is not possible.	14 years ago
unknown	da5214831d	Fix for bug lp:944706, task MDEV-193 The patch enables back constant subquery execution during query optimization after it was disabled during the development of MWL#89 (cost-based choice of IN-TO-EXISTS vs MATERIALIZATION). The main idea is that constant subqueries are allowed to be executed during optimization if their execution is not expensive. The approach is as follows: - Constant subqueries are recursively optimized in the beginning of JOIN::optimize of the outer query. This is done by the new method JOIN::optimize_constant_subqueries(). This is done so that the cost of executing these queries can be estimated. - Optimization of the outer query proceeds normally. During this phase the optimizer may request execution of non-expensive constant subqueries. Each place where the optimizer may potentially execute an expensive expression is guarded with the predicate Item::is_expensive(). - The implementation of Item_subselect::is_expensive has been extended to use the number of examined rows (estimated by the optimizer) as a way to determine whether the subquery is expensive or not. - The new system variable "expensive_subquery_limit" controls how many examined rows are considered to be not expensive. The default is 100. In addition, multiple changes were needed to make this solution work in the light of the changes made by MWL#89. These changes were needed to fix various crashes and wrong results, and legacy bugs discovered during development.	14 years ago
Sergey Petrunya	d028d986a9	BUG#952297: Server crashes on 2nd execution of PS in Field::is_null with semijoin+materialization - The bug would show up - when using PS (so that we get re-execution) - the left_expr of the subquery is a reference to viewname.column_name, so that it crashes when one tries to use it without having called fix_fields for it. - when using SJ-Materialization, which makes use of sj_subq_pred->left_expr expression - The fix is to have setup_conds() fix sj_subq_pred->left_expr for semi-join nests it finds.	14 years ago
Igor Babaev	04ee30e75a	Fixed LP bug #946055 . The function create_hj_key_for_table() that builds the descriptor of the hash join key to access a table of a materialized subquery must ignore any equi-join predicate depending on the tables not belonging to the subquery.	14 years ago
Igor Babaev	000deedf3b	Fixed LP bug #943543 . This bug appeared after the patch for bug 939009 that in the function merge_key_fields forgot to reset a proper value for the val field in the result of the merge operation of the key field created for a regular key access and the key field created to look for a NULL key. Adjusted the results of the test case for bug 939009 that actually were incorrect.	14 years ago
Igor Babaev	841a74a4d6	Fixed LP bug #939009 . The result of materialization of the right part of an IN subquery predicate is placed into a temporary table. Each row of the materialized table is distinct. A unique key over all fields of the temporary table is defined and created. It allows to perform key look-ups into the table. The table created for a materialized subquery can be accessed by key as any other table. The function best_access-path search for the best access to join a table to a given partial join. With some where conditions this function considers a possibility of a ref_or_null access. If such access employs the unique key on the temporary table then when estimating the cost this access the function tries to use the array rec_per_key. Yet, such array is not built for this unique key. This causes a crash of the server. Rows returned by the subquery that contain nulls don't have to be placed into temporary table, as they cannot be match any row produced by the left part of the subquery predicate. So all fields of the temporary table can be defined as non-nullable. In this case any ref_or_null access to the temporary table does not make any sense and it does not make sense to estimate such an access. The fix makes sure that the temporary table for a materialized IN subquery is defined with columns that are all non-nullable. The also ensures that any row with nulls returned by the subquery is not placed into the temporary table.	14 years ago
Sergey Petrunya	6400df6d71	Update test results.	14 years ago
Sergey Petrunya	fecad7c945	BUG#933407: Valgrind warnings in mark_as_null_row with materialization+semijoin, STRAIGHT_JOIN, impossible WHERE - In return_zero_rows(), don't call mark_as_null_row() for semi-join materialized tables, because 1) they may have been already freed, and 2)there is no real need to call mark_as_null_row() for them.	14 years ago
Sergey Petrunya	c9355dc279	BUG#928048: Query containing IN subquery with OR in the where clause returns a wrong result - Make equality propagation work correctly when done inside the OR branches	14 years ago
Sergey Petrunya	53fde5bb6f	BUG#922254: Assertion `0' failed at item_cmpfunc.cc:5899: Item* Item_equal::get_first(JOIN_TAB, Item) Fixed Item* Item_equal::get_first(JOIN_TAB context, Item field_item) to work correctly in the case where: - context!= NO_PARTICULAR_TAB, it points to a table within SJ-Materialization nest - field_item points to an item_equal that has a constant Item_field but does not have any fields from tables that are within semi-join nests.	14 years ago
unknown	072073c09e	Backport of WL#5953 from MySQL 5.6 The patch differs from the original MySQL patch as follows: - All test case differences have been reviewed one by one, and care has been taken to restore the original plan so that each test case executes the code path it was designed for. - A bug was found and fixed in MariaDB 5.3 in Item_allany_subselect::cleanup(). - ORDER BY is not removed because we are unsure of all effects, and it would prevent enabling ORDER BY ... LIMIT subqueries. - ref_pointer_array.m_size is not adjusted because we don't do array bounds checking, and because it looks risky. Original comment by Jorgen Loland: ------------------------------------------------------------- WL#5953 - Optimize away useless subquery clauses For IN/ALL/ANY/SOME/EXISTS subqueries, the following clauses are meaningless: * ORDER BY (since we don't support LIMIT in these subqueries) * DISTINCT * GROUP BY if there is no HAVING clause and no aggregate functions This WL detects and optimizes away these useless parts of the query during JOIN::prepare()	14 years ago

13 Commits (cfd2646c3152bbefe11e381c00b701a49cf5c4ef)