MySQL  8.0.20
Source Code Documentation
Parallel_reader_adapter Class Reference

Traverse an index in the leaf page block list order and send records to adapter. More...

#include <row0pread-adapter.h>

Classes

struct  MySQL_row
 MySQL row meta data. More...
 

Public Types

using Load_fn = handler::Load_cbk
 
using End_fn = handler::Load_end_cbk
 
using Init_fn = handler::Load_init_cbk
 

Public Member Functions

 Parallel_reader_adapter (size_t max_threads, ulint rowlen)
 Constructor. More...
 
 ~Parallel_reader_adapter ()
 Destructor. More...
 
dberr_t add_scan (trx_t *trx, const Parallel_reader::Config &config, Parallel_reader::F &&f)
 Add scan context. More...
 
dberr_t run (void **thread_contexts, Init_fn init_fn, Load_fn load_fn, End_fn end_fn)
 Run the parallel scan. More...
 
dberr_t process_rows (const Parallel_reader::Ctx *ctx)
 Convert the record in InnoDB format to MySQL format and send them. More...
 
void set (row_prebuilt_t *prebuilt)
 Set up the query processing state cache. More...
 

Private Types

using Shards = Counter::Shards< Parallel_reader::MAX_THREADS >
 
using Buffer = std::vector< byte >
 The row buffer. More...
 

Private Member Functions

dberr_t init (size_t thread_id)
 The callers init function. More...
 
dberr_t end (Parallel_reader::Ctx *ctx, size_t thread_id)
 For pushing any left over rows to the caller. More...
 
dberr_t send_batch (const Parallel_reader::Ctx *ctx, uint64_t n_recs)
 Send a batch of records. More...
 
Counter::Type pending (size_t thread_id) const
 Get the number of rows buffered but not sent. More...
 
bool is_buffer_full (size_t thread_id) const
 Check if the buffer is full. More...
 

Private Attributes

Shards m_n_sent {}
 Counter to track number of records sent to the caller. More...
 
Shards m_n_read {}
 Counter to track number of records processed. More...
 
void ** m_thread_contexts {nullptr}
 Adapter context for each of the spawned threads. More...
 
Init_fn m_init_fn {}
 Callback called by each parallel load thread at the beginning of the parallel load for the scan. More...
 
Load_fn m_load_fn {}
 Callback called by each parallel load thread when processing of rows is required for the scan. More...
 
End_fn m_end_fn {}
 Callback called by each parallel load thread when processing of rows has ended for the scan. More...
 
uint64_t m_batch_size {}
 Number of records to be sent across to the caller in a batch. More...
 
std::vector< Bufferm_buffers {}
 Buffer to store records to be sent to the caller. More...
 
MySQL_row m_mysql_row {}
 Row meta data per scan context. More...
 
row_prebuilt_tm_prebuilt {}
 Prebuilt to use for conversion to MySQL row format. More...
 
std::vector< mem_heap_t *, ut_allocator< mem_heap_t * > > m_blob_heaps {}
 BLOB heap per thread. More...
 
Parallel_reader m_parallel_reader
 Parallel reader to use. More...
 

Static Private Attributes

static constexpr size_t ADAPTER_SEND_BUFFER_SIZE = 2 * 1024 * 1024
 Size of the buffer used to store InnoDB records and sent to the adapter. More...
 

Detailed Description

Traverse an index in the leaf page block list order and send records to adapter.

Member Typedef Documentation

◆ Buffer

using Parallel_reader_adapter::Buffer = std::vector<byte>
private

The row buffer.

◆ End_fn

◆ Init_fn

◆ Load_fn

◆ Shards

Constructor & Destructor Documentation

◆ Parallel_reader_adapter()

Parallel_reader_adapter::Parallel_reader_adapter ( size_t  max_threads,
ulint  rowlen 
)

Constructor.

Parameters
[in]max_threadsMaximum threads to use for all scan contexts.
[in]rowlenRow length.

◆ ~Parallel_reader_adapter()

Parallel_reader_adapter::~Parallel_reader_adapter ( )

Destructor.

Member Function Documentation

◆ add_scan()

dberr_t Parallel_reader_adapter::add_scan ( trx_t trx,
const Parallel_reader::Config config,
Parallel_reader::F &&  f 
)

Add scan context.

Parameters
[in]trxTransaction used for parallel read.
[in]config(Cluster) Index scan configuration.
[in]fCallback function.
Return values
error.

◆ end()

dberr_t Parallel_reader_adapter::end ( Parallel_reader::Ctx ctx,
size_t  thread_id 
)
private

For pushing any left over rows to the caller.

Parameters
[in]ctxParallel read context.
[in]thread_idID of the thread.
Returns
DB_SUCCESS or error code.

◆ init()

dberr_t Parallel_reader_adapter::init ( size_t  thread_id)
private

The callers init function.

Parameters
[in]thread_idID of the thread.
Returns
DB_SUCCESS or error code.

◆ is_buffer_full()

bool Parallel_reader_adapter::is_buffer_full ( size_t  thread_id) const
inlineprivate

Check if the buffer is full.

Parameters
[in]thread_idID of the thread.
Returns
true if the buffer is full.

◆ pending()

Counter::Type Parallel_reader_adapter::pending ( size_t  thread_id) const
inlineprivate

Get the number of rows buffered but not sent.

Parameters
[in]thread_idID of the thread.
Returns
number of buffered items.

◆ process_rows()

dberr_t Parallel_reader_adapter::process_rows ( const Parallel_reader::Ctx ctx)

Convert the record in InnoDB format to MySQL format and send them.

Parameters
[in]ctxParallel read context.
Returns
error code

◆ run()

dberr_t Parallel_reader_adapter::run ( void **  thread_contexts,
Init_fn  init_fn,
Load_fn  load_fn,
End_fn  end_fn 
)

Run the parallel scan.

Parameters
[in]thread_contextsContext for each of the spawned threads
[in]init_fnCallback called by each parallel load thread at the beginning of the parallel load.
[in]load_fnCallback called by each parallel load thread when processing of rows is required.
[in]end_fnCallback called by each parallel load thread when processing of rows has ended.
Returns
DB_SUCCESS or error code.

◆ send_batch()

dberr_t Parallel_reader_adapter::send_batch ( const Parallel_reader::Ctx ctx,
uint64_t  n_recs 
)
private

Send a batch of records.

Parameters
[in]ctxParallel read context.
[in]n_recsNumber of records to send.
Returns
DB_SUCCESS or error code.

◆ set()

void Parallel_reader_adapter::set ( row_prebuilt_t prebuilt)

Set up the query processing state cache.

Parameters
[in]prebuiltThe prebuilt cache for the query.

Member Data Documentation

◆ ADAPTER_SEND_BUFFER_SIZE

constexpr size_t Parallel_reader_adapter::ADAPTER_SEND_BUFFER_SIZE = 2 * 1024 * 1024
staticprivate

Size of the buffer used to store InnoDB records and sent to the adapter.

◆ m_batch_size

uint64_t Parallel_reader_adapter::m_batch_size {}
private

Number of records to be sent across to the caller in a batch.

◆ m_blob_heaps

std::vector<mem_heap_t *, ut_allocator<mem_heap_t *> > Parallel_reader_adapter::m_blob_heaps {}
private

BLOB heap per thread.

◆ m_buffers

std::vector<Buffer> Parallel_reader_adapter::m_buffers {}
private

Buffer to store records to be sent to the caller.

◆ m_end_fn

End_fn Parallel_reader_adapter::m_end_fn {}
private

Callback called by each parallel load thread when processing of rows has ended for the scan.

◆ m_init_fn

Init_fn Parallel_reader_adapter::m_init_fn {}
private

Callback called by each parallel load thread at the beginning of the parallel load for the scan.

◆ m_load_fn

Load_fn Parallel_reader_adapter::m_load_fn {}
private

Callback called by each parallel load thread when processing of rows is required for the scan.

◆ m_mysql_row

MySQL_row Parallel_reader_adapter::m_mysql_row {}
private

Row meta data per scan context.

◆ m_n_read

Shards Parallel_reader_adapter::m_n_read {}
private

Counter to track number of records processed.

◆ m_n_sent

Shards Parallel_reader_adapter::m_n_sent {}
private

Counter to track number of records sent to the caller.

◆ m_parallel_reader

Parallel_reader Parallel_reader_adapter::m_parallel_reader
private

Parallel reader to use.

◆ m_prebuilt

row_prebuilt_t* Parallel_reader_adapter::m_prebuilt {}
private

Prebuilt to use for conversion to MySQL row format.

NOTE: We are sharing this because we don't convert BLOBs yet. There are data members in row_prebuilt_t that cannot be accessed in multi-threaded mode e.g., blob_heap.

row_prebuilt_t is designed for single threaded access and to share it among threads is not recommended unless "you know what you are doing". This is very fragile code as it stands.

To solve the blob heap issue in prebuilt we use per thread m_blob_heaps. Pass the blob heap to the InnoDB to MySQL row format conversion function.

◆ m_thread_contexts

void** Parallel_reader_adapter::m_thread_contexts {nullptr}
private

Adapter context for each of the spawned threads.


The documentation for this class was generated from the following files: