mariadb

Tree: e538cb095f

Author	SHA1	Message	Date
Igor Babaev	480a06718d	MDEV-25128 Wrong result from join with materialized semi-join and splittable derived If one of joined tables of the processed query is a materialized derived table (or view or CTE) with GROUP BY clause then under some conditions it can be subject to split optimization. With this optimization new equalities are injected into the WHERE condition of the SELECT that specifies this derived table. The injected equalities are generated for all join orders with which the split optimization can employed. After the best join order has been chosen only certain of this equalities are really needed. The others can be safely removed. If it's not done and some of injected equalities involve expressions over semi-joins with look-up access then the query may return a wrong result set. This patch effectively removes equalities injected for split optimization that are needed only at the optimization stage and not needed for execution. Approved by serg@mariadb.com	5 years ago
Varun Gupta	1be707286e	Added the test case for MDEV-23804	5 years ago
Sergei Petrunia	d01d94d77b	MDEV-17568: LATERAL DERIVED is not clearly visible in EXPLAIN FORMAT=JSON Make LATERAL DERIVED tables visible in EXPLAIN FORMAT=JSON output.	6 years ago
Monty	27d9986c1b	Added more digits to JSON output of double sprintf() format of double changed from '%lg' to '%-.11lg' The change was to make it easier to read optimizer trace output with tables that has millions of records.	6 years ago
Monty	eb483c5181	Updated optimizer costs in multi_range_read_info_const() and sql_select.cc - multi_range_read_info_const now uses the new records_in_range interface - Added handler::avg_io_cost() - Don't calculate avg_io_cost() in get_sweep_read_cost if avg_io_cost is not 1.0. In this case we trust the avg_io_cost() from the handler. - Changed test_quick_select to use TIME_FOR_COMPARE instead of TIME_FOR_COMPARE_IDX to align this with the rest of the code. - Fixed bug when using test_if_cheaper_ordering where we didn't use keyread if index was changed - Fixed a bug where we didn't use index only read when using order-by-index - Added keyread_time() to HEAP. The default keyread_time() was optimized for blocks and not suitable for HEAP. The effect was the HEAP prefered table scans over ranges for btree indexes. - Fixed get_sweep_read_cost() for HEAP tables - Ensure that range and ref have same cost for simple ranges Added a small cost (MULTI_RANGE_READ_SETUP_COST) to ranges to ensure we favior ref for range for simple queries. - Fixed that matching_candidates_in_table() uses same number of records as the rest of the optimizer - Added avg_io_cost() to JT_EQ_REF cost. This helps calculate the cost for HEAP and temporary tables better. A few tests changed because of this. - heap::read_time() and heap::keyread_time() adjusted to not add +1. This was to ensure that handler::keyread_time() doesn't give higher cost for heap tables than for normal tables. One effect of this is that heap and derived tables stored in heap will prefer key access as this is now regarded as cheap. - Changed cost for index read in sql_select.cc to match multi_range_read_info_const(). All index cost calculation is now done trough one function. - 'ref' will now use quick_cost for keys if it exists. This is done so that for '=' ranges, 'ref' is prefered over 'range'. - scan_time() now takes avg_io_costs() into account - get_delayed_table_estimates() uses block_size and avg_io_cost() - Removed default argument to test_if_order_by_key(); simplifies code	6 years ago
Igor Babaev	caf110fa52	MDEV-21883 Server crashes when joining a subselect with 32 tables and GROUP BY This bug could cause a crash for any query that used a derived table/view/CTE whose specification was a SELECT with a GROUP BY clause and a FROM list containing 32 or more table references. The problem appeared only in the cases when the splitting optimization could be applied to such derived table/view/CTE.	6 years ago
Igor Babaev	8d7462ec49	MDEV-21614 Wrong query results with optimizer_switch="split_materialized=on" Do not materialize a semi-join nest if it contains a materialized derived table /view that potentially can be subject to the split optimization. Splitting of materialization of such nest would help, but currently there is no code to support this technique.	6 years ago
Igor Babaev	1c97cd339e	MDEV-21184 Assertion `used_tables_cache == 0' failed in Item_func::fix_fields with condition_pushdown_from_having This bug could manifest itself for queries with GROUP BY and HAVING clauses when the HAVING clause was a conjunctive condition that depended exclusively on grouping fields and at least one conjunct contained an equality of the form fld=sq where fld is a grouping field and sq is a constant subquery. In this case the optimizer tries to perform a pushdown of the HAVING condition into WHERE. To construct the pushable condition the optimizer first transforms all multiple equalities in HAVING into simple equalities. This has to be done for a proper processing of the pushed conditions in WHERE. The multiple equalities at all AND/OR levels must be converted to simple equalities because any multiple equality may refer to a multiple equality at the upper level. Before this patch the conversion was performed like this: multiple_equality(x,f1,...,fn) => x=f1 and ... and x=fn. When an equality item for x=fi was constructed both the items for x and fi were cloned. If x happened to be a constant subquery that could not be cloned the conversion failed. If the conversions of multiple equalities previously performed had succeeded then the whole condition became in an inconsistent state that could cause different failures. The solution provided by the patch is: 1. to use a different conversion rule if x is a constant multiple_equality(x,f1,...,fn) => f1=x and f2=f1 and ... and fn=f1 2. not to clone x if it's a constant. Such conversions cannot fail and besides the result of the conversion preserves the equivalence of f1,...,fn that can be used for other optimizations. This patch also made sure that expensive predicates are not pushed from HAVING to WHERE.	6 years ago
Alexander Barkov	462d689397	MDEV-19468 Hybrid type expressions return wrong format for FLOAT	7 years ago
Galina Shalygina	ae15f91f22	MDEV-18769 Assertion `fixed == 1' failed in Item_cond_or::val_int This bug is caused by pushdown from HAVING into WHERE. It appears because condition that is pushed wasn't fixed. It is also discovered that condition pushdown from HAVING into WHERE is done wrong. There is no need to build clones for some conditions that can be pushed. They can be simply moved from HAVING into WHERE without cloning. build_pushable_cond_for_having_pushdown(), remove_pushed_top_conjuncts_for_having() methods are changed. It is found that there is no transformation made for fields of pushed condition. field_transformer_for_having_pushdown transformer is added. New tests are added. Some comments are changed.	7 years ago
Igor Babaev	9bd47835d0	MDEV-18679 Server crashes in JOIN::optimize The bug manifested itself when executing a query with materialized view/derived/CTE whose specification was a SELECT query contained another materialized derived and impossible WHERE/HAVING condition was detected for this SELECT. As soon as such condition is detected the join structures of all derived tables used in the SELECT are destroyed. So optimization of the queries specifying these derived tables is impossible. Besides it's not needed. In 10.3 optimization of a materialized derived table is performed before detection of impossible WHERE/HAVING condition in the embedding SELECT.	7 years ago
Varun Gupta	d6db6df995	MDEV-17903: New optimizer defaults: change optimize_join_buffer_size to be ON optimize_join_buffer_size is switched ON.	7 years ago
Galina Shalygina	9741930490	MDEV-18636 The test case for bug mdev-16765 crashes the server in the tree bb-10.4-mdev7486 The crash was caused because of the similar problem as in mdev-16765: Item_cond::excl_dep_on_group_fields_for_having_pushdown() was missing.	7 years ago
Sergei Petrunia	15a77a1a2c	MDEV-18608: Defaults for 10.4: histogram size should be set Change the defaults: -histogram_size=0 +histogram_size=254 -histogram_type=SINGLE_PREC_HB +histogram_type=DOUBLE_PREC_HB Adjust the testcases: - Some have ignorable changes in EXPLAIN outputs and more counter increments due to EITS table reads. - Testcases that meaningfully depend on the old defaults are changed to use the old values.	7 years ago
Galina Shalygina	7a77b221f1	MDEV-7486: Condition pushdown from HAVING into WHERE Condition can be pushed from the HAVING clause into the WHERE clause if it depends only on the fields that are used in the GROUP BY list or depends on the fields that are equal to grouping fields. Aggregate functions can't be pushed down. How the pushdown is performed on the example: SELECT t1.a,MAX(t1.b) FROM t1 GROUP BY t1.a HAVING (t1.a>2) AND (MAX(c)>12); => SELECT t1.a,MAX(t1.b) FROM t1 WHERE (t1.a>2) GROUP BY t1.a HAVING (MAX(c)>12); The implementation scheme: 1. Extract the most restrictive condition cond from the HAVING clause of the select that depends only on the fields that are used in the GROUP BY list of the select (directly or indirectly through equalities) 2. Save cond as a condition that can be pushed into the WHERE clause of the select 3. Remove cond from the HAVING clause if it is possible The optimization is implemented in the function st_select_lex::pushdown_from_having_into_where(). New test file having_cond_pushdown.test is created.	7 years ago
Igor Babaev	658128af43	MDEV-16188 Use in-memory PK filters built from range index scans This patch contains a full implementation of the optimization that allows to use in-memory rowid / primary filters built for range conditions over indexes. In many cases usage of such filters reduce the number of disk seeks spent for fetching table rows. In this implementation the choice of what possible filter to be applied (if any) is made purely on cost-based considerations. This implementation re-achitectured the partial implementation of the feature pushed by Galina Shalygina in the commit `8d5a11122c`. Besides this patch contains a better implementation of the generic handler function handler::multi_range_read_info_const() that takes into account gaps between ranges when calculating the cost of range index scans. It also contains some corrections of the implementation of the handler function records_in_range() for MyISAM. This patch supports the feature for InnoDB and MyISAM.	7 years ago
Varun Gupta	93c360e3a5	MDEV-15253: Default optimizer setting changes for MariaDB 10.4 use_stat_tables= PREFERABLY optimizer_use_condition_selectivity= 4	7 years ago
Igor Babaev	c2c1550f57	MDEV-17419 Subquery with group by returns wrong results Added only test case because the bug was fixed by the patch for mdev-17382.	7 years ago
Igor Babaev	d453374fc4	MDEV-16801 seg_fault on a query The bug was in the in the code of JOIN::check_for_splittable_materialized() where the structures describing the fields of a materialized derived table that potentially could be used in split optimization were build. As a result of this bug some fields that were not usable for splitting were detected as usable. This could trigger crashes further in st_join_table::choose_best_splitting().	7 years ago
Oleksandr Byelkin	de745ecf29	MDEV-11953: support of brackets in UNION/EXCEPT/INTERSECT operations	7 years ago
Igor Babaev	fc0f5adb7f	MDEV-16104 Server crash in JOIN::fix_all_splittings_in_plan upon select with view and subqueries This bug occurred when a splittable materialized derived/view were used inside another splittable materialized derived/view. The bug happened because the function JOIN::fix_all_splittings_in_plan() was called at the very beginning of the optimization phase 2 at the moment when the plan structure of the embedding derived/view were not valid. The proper position for this call is the very end of the optimization phase 1.	8 years ago
Varun Gupta	724ab9a1cb	MDEV-16057: Using optimization Splitting with Group BY we see an unnecessary attached condition t1.pk IS NOT NULL where pk is a PRIMARY KEY For equalites in the WHERE clause we create a keyuse array that contains the set of all equalities. For each KEYUSE inside the keyuse array we have a field "null_rejecting" which tells that the equality will not hold if either the left or right hand side of the equality is NULL. If the equality is NULL rejecting then we accordingly add a NOT NULL condition for the field present in the item val(present in the KEYUSE struct) when we are doing ref access. For the optimization of splitting with GROUP BY we always set the null_rejecting to TRUE and we are doing ref access on the GROUP by field. This does create a problem when the equality is NOT NULL rejecting. This happens in this case as in the equality we have the right hand side as t1.pk where pk is a PRIMARY KEY , hence it is NOT NULLABLE. So we should have null rejecting set to FALSE for such a case.	8 years ago
Igor Babaev	615ad709a3	MDEV-15902 Assertion `n < m_size' failed, sql_array.h:64: Element_type& Bounds_checked_array<Element_type>::operator[] (size_t) [with Element_type = Item*; size_t = long unsigned int] In sql_yacc.yy the semantic actions for the MEDIAN window function lacked a call of st_select_lex::prepare_add_window_spec(). This function saves the head of the thd->lex->order_list into lex->save_order_list in order this head to be restored in st_select_lex::add_window_spec after the specification of the window function has been parsed. Without a call of prepare_add_window_spec() when add_window_spec() was called the head of an empty list was copied into thd->lex->order_list (instead of assumed saved head this list). This made the list thd->lex->order_list invalid and potentially could cause many different problems. Corrected the result set in the test case for MDEV-15899 that used the MEDIAN window function and could not be correct without this fix.	8 years ago
Igor Babaev	cff60be7fe	MDEV-15899 Server crashes in st_join_table::is_inner_table_of_outer_join The crash happened because JOIN::check_for_splittable_materialized() called by mistake the function JOIN_TAB::is_inner_table_of_outer_join() instead of the function TABLE_LIST::is_inner_table_of_outer_join(). The former cannot be called before the call of make_outerjoin_info().	8 years ago
Sergei Golubchik	c43a0e0a77	bugfix: Item_cache_temporal::convert_to_basic_const_item assumed DATETIME this is a 10.3 version of `1c6f6dc892`	8 years ago
Michael Widenius	a7abddeffa	Create 'main' test directory and move 't' and 'r' there	8 years ago
Igor Babaev	7a9611aee2	Fixed MDEV-14994 Assertion `join->best_read < double(1.79...15e+308L)' or server crash in JOIN::fix_all_splittings_in_plan Cost formulas must take into account the case when a splittable table has now rows.	8 years ago
Igor Babaev	775aa5542d	Fixed mdev-15017 Server crashes in in st_join_table::fix_splitting Do not apply splitting for constant tables.	8 years ago
Igor Babaev	c5ac1f953b	Fixed mdev-14880: Assertion `inj_cond_list.elements' failed in JOIN::inject_best_splitting_cond The value of SplM_opt_info::last_plan should be set to NULL before any search for a splitting plan for a splittable materialized table.	8 years ago
Igor Babaev	578345305e	Added a test case for mdev-13454: Improper error in ONLY_FULL_GROUP_BY sql_mode with condition_pushdown_for_derived=on This bug is a consequence of the bug mdev-14368 fixed in 5.5.59.	8 years ago
Igor Babaev	86cf60a615	Fixed mdev-14845 Server crashes in st_join_table::is_inner_table_of_outer_join Do not try to apply the splitting optimization to a materialized derived if it's specified by a select with impossible where or if all joined tables of this select are constant.	8 years ago
Igor Babaev	4f0299f8b3	This is a full cost-based implementation of the optimization that employs splitting technique for equi-joins of materialized derived tables/views/CTEs. (see mdev-13369 and mdev-13389).	8 years ago
Galina Shalygina	3791d0cfcf	Test result changed	8 years ago
Galina Shalygina	86308aa995	MDEV-14579: New tests for condition pushdown into materialized views/defined tables defined with INTERSECT/EXCEPT added	8 years ago
Igor Babaev	343bcb152f	Fixed mdev-14237 Server crash on query with regexp_substr It's better to prohibit pushdown of conditions that involve regexp_substr() and regexp_replace() into materialized derived tables / views until proper implementations of the get_copy() virtual method are provided for those functions.	8 years ago
Igor Babaev	3a9ee22ba9	Fixed the bug mdev-13734. If the method SELECT_LEX::mark_const_derived() is called for a select that is used in the specification of a materialized derived table / view D then method should not set the flag fill_me for D on when the flag JOIN::with_two_phase_optimization is set on for this select.	8 years ago
Igor Babaev	61074d0426	Fixed the bug mdev-13710. This patch corrects the code of the patch for mdev-13369 that introduced the splitting technique when using materialized derived tables / views with GROUP BY. The second actual parameters of the call of the method JOIN::reoptimize() in the function JOIN::push_splitting_cond_into_derived() was calculated incorrectly. This could cause different failures for queries using derived tables or views with GROUP BY when their FROM lists contained empty or single-row tables.	8 years ago
Igor Babaev	5e4aa1a2f8	Fixed the bug mdev-13709. Currently condition pushdown into materialized views / derived tables is not implemented yet (see mdev-12387) and grouping views are optimized early when subqueries are converted to semi-joins in convert_join_subqueries_to_semijoins(). If a subquery that is converted to a semi-join uses a grouping view this view is optimized in two phases. For such a view V only the first phase of optimization is done after the conversion of subqueries of the outer join into semi-joins. At the same time the reference of the view V appears in the join expression of the outer join. In fixed code there was an attempt to push conditions into this view and to optimize it after this. This triggered the second phase of the optimization of the view and it was done prematurely. The second phase of the optimization for the materialized view is supposed to be called after the splitting condition is pushed into the view in the call of JOIN::improve_chosen_plan for the outer join. The fix blocks the attempt to push conditions into splittable views if they have been already partly optimized and the following optimization for them. The test case of the patch shows that the code for mdev-13369 basically supported the splitting technique for materialized views / derived tables. The patch also replaces the name of the state JOIN::OPTIMIZATION_IN_STAGE_2 for JOIN::OPTIMIZATION_PHASE_1_DONE and fixes a bug in TABLE_LIST::fetch_number_of_rows()	8 years ago
Igor Babaev	a1e444e1cc	Corrected the function compare_order_elements() to make it platform independent.	8 years ago
Igor Babaev	d7b45e01b4	This patch complements the patch for mdev-10855. It allows to push conditions into derived with window functions not only in the cases when the window specifications of these window functions use the same partition, but also in the cases when the window functions use partitions that share only some fields. In these cases only the conditions over the common fields are pushed.	8 years ago
Igor Babaev	61bbabb202	Implemented condition pushdown into derived tables / views with window functions (mdev-10855). This patch just modified the function pushdown_cond_for_derived() to support this feature. Some test cases demonstrating this optimization were added to derived_cond_pushdown.test.	8 years ago
Igor Babaev	bf75dcac89	This is a modification of the first patch committed for mdev-13369 developed to cover the case of mdev-13389: "Optimization for equi-joins of derived tables with window functions".	8 years ago
Igor Babaev	b14e2b044b	This first patch prepared for the task MDEV-13369: "Optimization for equi-joins of derived tables with GROUP BY" should be considered rather as a 'proof of concept'. The task itself is targeted at an optimization that employs re-writing equi-joins with grouping derived tables / views into lateral derived tables. Here's an example of such transformation: select t1.a,t.max,t.min from t1 [left] join (select a, max(t2.b) max, min(t2.b) min from t2 group by t2.a) as t on t1.a=t.a; => select t1.a,tl.max,tl.min from t1 [left] join lateral (select a, max(t2.b) max, min(t2.b) min from t2 where t1.a=t2.a) as t on 1=1; The transformation pushes the equi-join condition t1.a=t.a into the derived table making it dependent on table t1. It means that for every row from t1 a new derived table must be filled out. However the size of any of these derived tables is just a fraction of the original derived table t. One could say that transformation 'splits' the rows used for the GROUP BY operation into separate groups performing aggregation for a group only in the case when there is a match for the current row of t1. Apparently the transformation may produce a query with a better performance only in the case when - the GROUP BY list refers only to fields returned by the derived table - there is an index I on one of the tables T used in FROM list of the specification of the derived table whose prefix covers the the fields from the proper beginning of the GROUP BY list or fields that are equal to those fields. Whether the result of the re-writing can be executed faster depends on many factors: - the size of the original derived table - the size of the table T - whether the index I is clustering for table T - whether the index I fully covers the GROUP BY list. This patch only tries to improve the chosen execution plan using this transformation. It tries to do it only when the chosen plan reaches the derived table by a key whose prefix covers all the fields of the derived table produced by the fields of the table T from the GROUP BY list. The code of the patch does not evaluates the cost of the improved plan. If certain conditions are met the transformation is applied.	8 years ago
Igor Babaev	9f0c1c0cf6	Fixed the bug mdev-13193. When an equality that can be pushed into a materialized derived table / view is extracted from multiple equalities and their operands are cloned then if they have some pointers to Item_equal objects those pointers must be set to NULL in the clones. Anyway they are not valid in the pushed predicates.	8 years ago
Igor Babaev	02655a91cf	Fixed the bug mdev-13166. This patch corrects the fix for mdev-12845.	8 years ago
Igor Babaev	9f3622191d	Fixed the bug mdev-12845. This patch fills in a serious flaw in the code that supports condition pushdown into materialized views / derived tables. If a predicate happened to contain a reference to a mergeable view / derived table and it does not depended directly on the target materialized view / derived table then the predicate was not considered as a subject to pusdown to this view / derived table.	8 years ago
Sergei Golubchik	5ccaabe962	MDEV-10355 Weird error message upon CREATE TABLE with DEFAULT post-fix for `0b52b28b91` * restore EXPLAIN output * remove redundant code	9 years ago
Igor Babaev	4beb7e5355	Fixed the bug mdev-12373. Condition pushdown into derived tables / views with side effects is not allowed.	9 years ago
Jacob Mathew	0b52b28b91	MDEV-10355 Weird error message upon CREATE TABLE with DEFAULT Fixed handling of default values with cached temporal functions so that the CREATE TABLE statement now succeeds. Fixed virtual column session cleanup. Fixed the error message. Added quoting of date/time values in cases when this was omitted. Added a test case in default.test. Updated test result files.	9 years ago
Oleksandr Byelkin	05d3c3d3f7	MDEV-10141: Add support for INTERSECT (and common parts for EXCEPT) MDEV-10140: Add support for EXCEPT	9 years ago

1 2

69 Commits (e538cb095f6290c40e8928e3813db6ac679740a2)