Handles materialization; the first call to Init() will scan the given iterator to the end, store the results in a temporary table (optionally with deduplication), and then Read() will allow you to read that table repeatedly without the cost of executing the given subquery many times (unless you ask for rematerialization).
More...
|
| MaterializeIterator (THD *thd, Mem_root_array< QueryBlock > query_blocks_to_materialize, TABLE *table, unique_ptr_destroy_only< RowIterator > table_iterator, Common_table_expr *cte, Query_expression *unit, JOIN *join, int ref_slice, bool rematerialize, ha_rows limit_rows, bool reject_multiple_rows) |
|
bool | Init () override |
| Initialize or reinitialize the iterator. More...
|
|
int | Read () override |
| Read a single row. More...
|
|
void | SetNullRowFlag (bool is_null_row) override |
| Mark the current row buffer as containing a NULL row or not, so that if you read from it and the flag is true, you'll get only NULLs no matter what is actually in the buffer (typically some old leftover row). More...
|
|
void | StartPSIBatchMode () override |
| Start performance schema batch mode, if supported (otherwise ignored). More...
|
|
void | EndPSIBatchModeIfStarted () override |
| Ends performance schema batch mode, if started. More...
|
|
void | UnlockRow () override |
|
void | AddInvalidator (const CacheInvalidatorIterator *invalidator) |
| Add a cache invalidator that must be checked on every Init(). More...
|
|
uint64_t | num_init_calls () const |
|
uint64_t | num_rows () const |
|
| TableRowIterator (THD *thd, TABLE *table) |
|
void | UnlockRow () override |
| The default implementation of unlock-row method of RowIterator, used in all access methods except EQRefIterator. More...
|
|
void | SetNullRowFlag (bool is_null_row) override |
| Mark the current row buffer as containing a NULL row or not, so that if you read from it and the flag is true, you'll get only NULLs no matter what is actually in the buffer (typically some old leftover row). More...
|
|
void | StartPSIBatchMode () override |
| Start performance schema batch mode, if supported (otherwise ignored). More...
|
|
void | EndPSIBatchModeIfStarted () override |
| Ends performance schema batch mode, if started. More...
|
|
| RowIterator (THD *thd) |
|
virtual | ~RowIterator ()=default |
|
| RowIterator (const RowIterator &)=delete |
|
| RowIterator (RowIterator &&)=default |
|
virtual std::string | TimingString () const |
|
virtual RowIterator * | real_iterator () |
| If this iterator is wrapping a different iterator (e.g. More...
|
|
virtual const RowIterator * | real_iterator () const |
|
Handles materialization; the first call to Init() will scan the given iterator to the end, store the results in a temporary table (optionally with deduplication), and then Read() will allow you to read that table repeatedly without the cost of executing the given subquery many times (unless you ask for rematerialization).
When materializing, MaterializeIterator takes care of evaluating any items that need so, and storing the results in the fields of the outgoing table – which items is governed by the temporary table parameters.
Conceptually (although not performance-wise!), the MaterializeIterator is a no-op if you don't ask for deduplication, and in some cases (e.g. when scanning a table only once), we elide it. However, it's not necessarily straightforward to do so by just not inserting the iterator, as the optimizer will have set up everything (e.g., read sets, or what table upstream items will read from) assuming the materialization will happen, so the realistic option is setting up everything as if materialization would happen but not actually write to the table; see StreamingIterator for details.
MaterializeIterator conceptually materializes iterators, not JOINs or Query_expressions. However, there are many details that leak out (e.g., setting performance schema batch mode, slices, reusing CTEs, etc.), so we need to send them in anyway.
bool MaterializeIterator::doing_hash_deduplication |
( |
| ) |
const |
|
inlineprivate |
Whether we are deduplicating using a hash field on the temporary table.
(This condition mirrors check_unique_constraint().) If so, we compute a hash value for every row, look up all rows with the same hash and manually compare them to the row we are trying to insert.
Note that this is not the common way of deduplicating as we go. The common method is to have a regular index on the table over the right columns, and in that case, ha_write_row() will fail with an ignorable error, so that the row is ignored even though check_unique_constraint() is not called. However, B-tree indexes have limitations, in particular on length, that sometimes require us to do this instead. See create_tmp_table() for details.
bool MaterializeIterator::MaterializeRecursive |
( |
| ) |
|
|
private |
Recursive materialization happens much like regular materialization, but some steps are repeated multiple times.
Our general strategy is:
- Materialize all non-recursive query blocks, once.
- Materialize all recursive query blocks in turn.
- Repeat #2 until no query block writes any more rows (ie., we have converged) – for UNION DISTINCT queries, rows removed by deduplication do not count. Each materialization sees only rows that were newly added since the previous iteration; see FollowTailIterator for more details on the implementation.
Note that the result table is written to while other iterators are still reading from it; again, see FollowTailIterator. This means that each run of #2 can potentially run many actual CTE iterations – possibly the entire query to completion if we have only one query block.
This is not how the SQL standard specifies recursive CTE execution (it assumes building up the new result set from scratch for each iteration, using the previous iteration's results), but it is equivalent, and more efficient for the class of queries we support, since we don't need to re-create the same rows over and over again.
void MaterializeIterator::SetNullRowFlag |
( |
bool |
is_null_row | ) |
|
|
inlineoverridevirtual |
Mark the current row buffer as containing a NULL row or not, so that if you read from it and the flag is true, you'll get only NULLs no matter what is actually in the buffer (typically some old leftover row).
This is used for outer joins, when an iterator hasn't produced any rows and we need to produce a NULL-complemented row. Init() or Read() won't necessarily reset this flag, so if you ever set is to true, make sure to also set it to false when needed.
Note that this can be called without Init() having been called first. For example, NestedLoopIterator can hit EOF immediately on the outer iterator, which means the inner iterator doesn't get an Init() call, but will still forward SetNullRowFlag to both inner and outer iterators.
TODO: We shouldn't need this. See the comments on AggregateIterator for a bit more discussion on abstracting out a row interface.
Implements RowIterator.
void MaterializeIterator::StartPSIBatchMode |
( |
| ) |
|
|
inlineoverridevirtual |
Start performance schema batch mode, if supported (otherwise ignored).
PFS batch mode is a mitigation to reduce the overhead of performance schema, typically applied at the innermost table of the entire join. If you start it before scanning the table and then end it afterwards, the entire set of handler calls will be timed only once, as a group, and the costs will be distributed evenly out. This reduces timer overhead.
If you start PFS batch mode, you must also take care to end it at the end of the scan, one way or the other. Do note that this is true even if the query ends abruptly (LIMIT is reached, or an error happens). The easiest workaround for this is to simply call EndPSIBatchModeIfStarted() on the root iterator at the end of the scan. See the PFSBatchMode class for a useful helper.
The rules for starting batch and ending mode are:
- If you are an iterator with exactly one child (FilterIterator etc.), forward any StartPSIBatchMode() calls to it.
- If you drive an iterator (read rows from it using a for loop or similar), use PFSBatchMode as described above.
- If you have multiple children, ignore the call and do your own handling of batch mode as appropriate. For materialization, #2 would typically apply. For joins, it depends on the join type (e.g., NestedLoopIterator applies batch mode only when scanning the innermost table).
The upshot of this is that when scanning a single table, batch mode will typically be activated for that table (since we call StartPSIBatchMode() on the root iterator, and it will trickle all the way down to the table iterator), but for a join, the call will be ignored and the join iterator will activate batch mode by itself as needed.
Reimplemented from RowIterator.