AggregateIterator Class Reference

Handles aggregation (typically used for GROUP BY) for the case where the rows are already properly grouped coming in, ie., all rows that are supposed to be part of the same group are adjacent in the input stream. More...

#include <composite_iterators.h>

Inheritance diagram for AggregateIterator:

Public Member Functions
	AggregateIterator (THD *thd, unique_ptr_destroy_only< RowIterator > source, JOIN *join, pack_rows::TableCollection tables, bool rollup)
bool	Init () override
	Initialize or reinitialize the iterator. More...
int	Read () override
	Read a single row. More...
void	SetNullRowFlag (bool is_null_row) override
	Mark the current row buffer as containing a NULL row or not, so that if you read from it and the flag is true, you'll get only NULLs no matter what is actually in the buffer (typically some old leftover row). More...
void	StartPSIBatchMode () override
	Start performance schema batch mode, if supported (otherwise ignored). More...
void	EndPSIBatchModeIfStarted () override
	Ends performance schema batch mode, if started. More...
void	UnlockRow () override
Public Member Functions inherited from RowIterator
	RowIterator (THD *thd)
virtual	~RowIterator ()=default
	RowIterator (const RowIterator &)=delete
	RowIterator (RowIterator &&)=default
virtual const IteratorProfiler *	GetProfiler () const
	Get profiling data for this iterator (for 'EXPLAIN ANALYZE'). More...
virtual void	SetOverrideProfiler ([[maybe_unused]] const IteratorProfiler *profiler)
virtual RowIterator *	real_iterator ()
	If this iterator is wrapping a different iterator (e.g. More...
virtual const RowIterator *	real_iterator () const

Private Types
enum	{ READING_FIRST_ROW , LAST_ROW_STARTED_NEW_GROUP , OUTPUTTING_ROLLUP_ROWS , DONE_OUTPUTTING_ROWS }

Private Member Functions
void	SetRollupLevel (int level)

Private Attributes
enum AggregateIterator:: { ... }	m_state
unique_ptr_destroy_only< RowIterator >	m_source
JOIN *	m_join = nullptr
	The join we are part of. More...
bool	m_seen_eof
	Whether we have seen the last input row. More...
table_map	m_save_nullinfo
	Used to save NULL information in the specific case where we have zero input rows. More...
const bool	m_rollup
	Whether this is a rollup query. More...
int	m_last_unchanged_group_item_idx
	For rollup: The index of the first group item that did not change when we last switched groups. More...
int	m_current_rollup_position
	If we are in state OUTPUTTING_ROLLUP_ROWS, where we are in the iteration. More...
pack_rows::TableCollection	m_tables
	The list of tables we are reading from; they are the ones for which we need to save and restore rows. More...
String	m_first_row_this_group
	Packed version of the first row in the group we are currently processing. More...
String	m_first_row_next_group
	If applicable, packed version of the first row in the next group. More...
int	m_output_slice = -1
	The slice we're setting when returning rows. More...

Additional Inherited Members
Protected Member Functions inherited from RowIterator
THD *	thd () const

Detailed Description

Handles aggregation (typically used for GROUP BY) for the case where the rows are already properly grouped coming in, ie., all rows that are supposed to be part of the same group are adjacent in the input stream.

(This could be because they were sorted earlier, because we are scanning an index that already gives us the rows in a group-compatible order, or because there is no grouping.)

AggregateIterator needs to be able to save and restore rows; it doesn't know when a group ends until it's seen the first row that is part of the next group. When that happens, it needs to tuck away that next row, and then restore the previous row so that the output row gets the correct grouped values. A simple example, doing SELECT a, SUM(b) FROM t1 GROUP BY a:

t1.a t1.b SUM(b) 1 1 <– first row, save it 1 1 2 3 1 3 6 2 1 <– group changed, save row [1 1] <– restore first row, output 6 reset aggregate --> 0 [2 1] <– restore new row, process it 1 2 10 11 <– EOF, output 11

To save and restore rows like this, it uses the infrastructure from pack_rows.h to pack and unpack all relevant rows into record[0] of every input table. (Currently, there can only be one input table, but this may very well change in the future.) It would be nice to have a more abstract concept of sending a row around and taking copies of it if needed, as opposed to it implicitly staying in the table's buffer. (This would also solve some issues in EQRefIterator and when synthesizing NULL rows for outer joins.) However, that's a large refactoring.

Member Enumeration Documentation

anonymous enum

anonymous enum

private

Enumerator
READING_FIRST_ROW
LAST_ROW_STARTED_NEW_GROUP
OUTPUTTING_ROLLUP_ROWS
DONE_OUTPUTTING_ROWS

Constructor & Destructor Documentation

AggregateIterator()

AggregateIterator::AggregateIterator	(	THD *	thd,
		unique_ptr_destroy_only< RowIterator >	source,
		JOIN *	join,
		pack_rows::TableCollection	tables,
		bool	rollup
	)

Member Function Documentation

EndPSIBatchModeIfStarted()

void AggregateIterator::EndPSIBatchModeIfStarted ( )

inlineoverridevirtual

Ends performance schema batch mode, if started.

It's always safe to call this.

Iterators that have children (composite iterators) must forward the EndPSIBatchModeIfStarted() call to every iterator they could conceivably have called StartPSIBatchMode() on. This ensures that after such a call to on the root iterator, all handlers are out of batch mode.

Reimplemented from RowIterator.

Init()

bool AggregateIterator::Init ( )

overridevirtual

Initialize or reinitialize the iterator.

You must always call Init() before trying a Read() (but Init() does not imply Read()).

You can call Init() multiple times; subsequent calls will rewind the iterator (or reposition it, depending on whether the iterator takes in e.g. a Index_lookup) and allow you to read the records anew.

Implements RowIterator.

Read()

int AggregateIterator::Read ( )

overridevirtual

Read a single row.

The row data is not actually returned from the function; it is put in the table's (or tables', in case of a join) record buffer, ie., table->records[0].

Return values

0	OK
-1	End of records
1	Error

Implements RowIterator.

SetNullRowFlag()

void AggregateIterator::SetNullRowFlag ( bool  is_null_row )

inlineoverridevirtual

Mark the current row buffer as containing a NULL row or not, so that if you read from it and the flag is true, you'll get only NULLs no matter what is actually in the buffer (typically some old leftover row).

This is used for outer joins, when an iterator hasn't produced any rows and we need to produce a NULL-complemented row. Init() or Read() won't necessarily reset this flag, so if you ever set is to true, make sure to also set it to false when needed.

Note that this can be called without Init() having been called first. For example, NestedLoopIterator can hit EOF immediately on the outer iterator, which means the inner iterator doesn't get an Init() call, but will still forward SetNullRowFlag to both inner and outer iterators.

TODO: We shouldn't need this. See the comments on AggregateIterator for a bit more discussion on abstracting out a row interface.

Implements RowIterator.

SetRollupLevel()

void AggregateIterator::SetRollupLevel ( int  level )

private

StartPSIBatchMode()

void AggregateIterator::StartPSIBatchMode ( )

inlineoverridevirtual

Start performance schema batch mode, if supported (otherwise ignored).

PFS batch mode is a mitigation to reduce the overhead of performance schema, typically applied at the innermost table of the entire join. If you start it before scanning the table and then end it afterwards, the entire set of handler calls will be timed only once, as a group, and the costs will be distributed evenly out. This reduces timer overhead.

If you start PFS batch mode, you must also take care to end it at the end of the scan, one way or the other. Do note that this is true even if the query ends abruptly (LIMIT is reached, or an error happens). The easiest workaround for this is to simply call EndPSIBatchModeIfStarted() on the root iterator at the end of the scan. See the PFSBatchMode class for a useful helper.

The rules for starting batch and ending mode are:

1. If you are an iterator with exactly one child (FilterIterator etc.), forward any StartPSIBatchMode() calls to it.
2. If you drive an iterator (read rows from it using a for loop or similar), use PFSBatchMode as described above.
3. If you have multiple children, ignore the call and do your own handling of batch mode as appropriate. For materialization, #2 would typically apply. For joins, it depends on the join type (e.g., NestedLoopIterator applies batch mode only when scanning the innermost table).

The upshot of this is that when scanning a single table, batch mode will typically be activated for that table (since we call StartPSIBatchMode() on the root iterator, and it will trickle all the way down to the table iterator), but for a join, the call will be ignored and the join iterator will activate batch mode by itself as needed.

Reimplemented from RowIterator.

UnlockRow()

void AggregateIterator::UnlockRow ( )

inlineoverridevirtual

Implements RowIterator.

Member Data Documentation

m_current_rollup_position

int AggregateIterator::m_current_rollup_position

private

If we are in state OUTPUTTING_ROLLUP_ROWS, where we are in the iteration.

This value will start at the index of the last group expression and then count backwards down to and including m_last_unchanged_group_item_idx. It is used to communicate to the rollup group items whether to turn themselves into NULLs, and the sum items which of their sums to output.

m_first_row_next_group

String AggregateIterator::m_first_row_next_group

private

If applicable, packed version of the first row in the next group.

This is used only in the LAST_ROW_STARTED_NEW_GROUP state; we just saw a row that didn't belong to the current group, so we saved it here and went to output a group. On the next Read() call, we need to process this deferred row first of all.

Even when not in use, this string contains a buffer that is large enough to pack a full row into, sans blobs. (If blobs are present, StoreFromTableBuffers() will automatically allocate more space if needed.)

m_first_row_this_group

String AggregateIterator::m_first_row_this_group

private

Packed version of the first row in the group we are currently processing.

m_join

JOIN* AggregateIterator::m_join = nullptr

private

The join we are part of.

It would be nicer not to rely on this, but we need a large number of members from there, like which aggregate functions we have, the THD, temporary table parameters and so on.

m_last_unchanged_group_item_idx

int AggregateIterator::m_last_unchanged_group_item_idx

private

For rollup: The index of the first group item that did not change when we last switched groups.

E.g., if we have group fields A,B,C,D and then switch to group A,B,E,D, this value will become 1 (which means that we need to output rollup rows for 2 – A,B,E,NULL – and then 1 – A,B,NULL,NULL). m_current_rollup_position will count down from the end until it becomes less than this value.

If we do not have rollup, this value is perennially zero.

m_output_slice

int AggregateIterator::m_output_slice = -1

private

The slice we're setting when returning rows.

See the comment in the constructor.

m_rollup

const bool AggregateIterator::m_rollup

private

Whether this is a rollup query.

m_save_nullinfo

table_map AggregateIterator::m_save_nullinfo

private

Used to save NULL information in the specific case where we have zero input rows.

m_seen_eof

bool AggregateIterator::m_seen_eof

private

Whether we have seen the last input row.

m_source

unique_ptr_destroy_only<RowIterator> AggregateIterator::m_source

private

enum { ... } AggregateIterator::m_state

m_tables

pack_rows::TableCollection AggregateIterator::m_tables

private

The list of tables we are reading from; they are the ones for which we need to save and restore rows.

---

The documentation for this class was generated from the following files:

• sql/iterators/composite_iterators.h
• sql/iterators/composite_iterators.cc