MySQL 8.3.0
Source Code Documentation
ddl::FTS Struct Reference

Full text search index builder. More...

#include <ddl0fts.h>

Classes

struct  Doc_item
 Information about temporary files used in merge sort. More...
 
struct  Inserter
 
struct  Parser
 For parsing and sorting the documents. More...
 

Public Member Functions

 FTS (Context &ctx, dict_index_t *index, dict_table_t *table) noexcept
 Constructor. More...
 
 ~FTS () noexcept
 ~Destructor. More...
 
dberr_t init (size_t n_threads) noexcept
 Create the internal data structures. More...
 
dict_index_tindex () noexcept
 
dict_index_tsort_index () noexcept
 
dberr_t start_parse_threads (Builder *builder) noexcept
 Start the parsing, create the threads. More...
 
dberr_t enqueue (Doc_item *doc_item) noexcept
 For sending the documents to parse to the parsing threads. More...
 
dberr_t check_for_errors () noexcept
 Check for error status after the parsing has finished. More...
 
dberr_t insert (Builder *builder) noexcept
 Start the merging and insert threads. More...
 
dberr_t scan_finished (dberr_t err) noexcept
 Inform the parser threads that the scanning phase is complete so that they can shutdown after emptying the doc item queue. More...
 

Private Types

using Threads = std::vector< std::thread >
 
using Parsers = std::vector< Parser *, ut::allocator< Parser * > >
 

Private Member Functions

dberr_t create (size_t n_threads) noexcept
 Create the data structures required to build the FTS index. More...
 
size_t get_n_parsers () const noexcept
 
void destroy () noexcept
 Destroy the data structures and clean up. More...
 
dberr_t setup_insert_phase () noexcept
 Setup the insert phase inoput files generated by the parsers. More...
 

Static Private Member Functions

static dict_index_tcreate_index (dict_index_t *index, dict_table_t *table, bool *doc_id_32_bit) noexcept
 Create a temporary "fts sort index" used to merge sort the tokenized doc string. More...
 

Private Attributes

Parsers m_parsers {}
 For parsing the documents, there is one per thread. More...
 
Inserterm_inserter {}
 For inserting the rows parsed by the m_parsers. More...
 
Contextm_ctx
 DDL context. More...
 
Dup m_dup
 Duplicate key reporting. More...
 
bool m_doc_id_32_bit {}
 true if document ID should be stored as a 32 bit instead of a 64 bit. More...
 
dict_index_tm_index {}
 DDL index instance. More...
 
dict_table_tm_table {}
 DDL table instance. More...
 
dict_index_tm_sort_index {}
 Temporary index instance with relevant FTS columns. More...
 
Threads m_threads {}
 For tracking parser threads. More...
 

Detailed Description

Full text search index builder.

Member Typedef Documentation

◆ Parsers

using ddl::FTS::Parsers = std::vector<Parser *, ut::allocator<Parser *> >
private

◆ Threads

using ddl::FTS::Threads = std::vector<std::thread>
private

Constructor & Destructor Documentation

◆ FTS()

ddl::FTS::FTS ( Context ctx,
dict_index_t index,
dict_table_t table 
)
noexcept

Constructor.

Parameters
[in,out]ctxDDL context.
[in,out]indexDDL index.
[in,out]tableDDL table.

◆ ~FTS()

ddl::FTS::~FTS ( )
noexcept

~Destructor.

Member Function Documentation

◆ check_for_errors()

dberr_t ddl::FTS::check_for_errors ( )
noexcept

Check for error status after the parsing has finished.

Returns
DB_SUCCESS or error code.

◆ create()

dberr_t ddl::FTS::create ( size_t  n_threads)
privatenoexcept

Create the data structures required to build the FTS index.

Parameters
[in]n_threadsNumber of parser threads.
Returns
DB_SUCCESS or error code.

◆ create_index()

dict_index_t * ddl::FTS::create_index ( dict_index_t index,
dict_table_t table,
bool *  doc_id_32_bit 
)
staticprivatenoexcept

Create a temporary "fts sort index" used to merge sort the tokenized doc string.

The index has three "fields":

  1. Tokenized word,
  2. Doc ID
  3. Word's position in original 'doc'.
Parameters
[in,out]indexIndex to sort.
[in,out]tableTable that the FTS index is created on.
[out]doc_id_32_bitWhether to use 4 bytes instead of 7 bytes integer to store the DOC ID during sort.
Returns
dict_index_t structure for the fts sort index

◆ destroy()

void ddl::FTS::destroy ( )
privatenoexcept

Destroy the data structures and clean up.

◆ enqueue()

dberr_t ddl::FTS::enqueue ( FTS::Doc_item doc_item)
noexcept

For sending the documents to parse to the parsing threads.

Parameters
[in,out]doc_itemDocument to parse, takes ownership.
Returns
DB_SUCCESS or error code, doc_item will be deleted either way.

◆ get_n_parsers()

size_t ddl::FTS::get_n_parsers ( ) const
inlineprivatenoexcept
Returns
the number of parses.

◆ index()

dict_index_t * ddl::FTS::index ( )
inlinenoexcept
Returns
the DDL index.

◆ init()

dberr_t ddl::FTS::init ( size_t  n_threads)
noexcept

Create the internal data structures.

Parameters
[in]n_threadsNumber of parse threads to create.
Returns
DB_SUCCESS or error code.

◆ insert()

dberr_t ddl::FTS::insert ( Builder builder)
noexcept

Start the merging and insert threads.

Parameters
[in,out]builderBuilder instance to use.
Returns
DB_SUCCESS or error code.

◆ scan_finished()

dberr_t ddl::FTS::scan_finished ( dberr_t  err)
noexcept

Inform the parser threads that the scanning phase is complete so that they can shutdown after emptying the doc item queue.

Parameters
[in]errError status of the scanning thread(s).
Returns
DB_SUCCESS or error code.

◆ setup_insert_phase()

dberr_t ddl::FTS::setup_insert_phase ( )
privatenoexcept

Setup the insert phase inoput files generated by the parsers.

Returns
DB_SUCCESS or error code.

◆ sort_index()

dict_index_t * ddl::FTS::sort_index ( )
inlinenoexcept
Returns
the temporary sort index.

◆ start_parse_threads()

dberr_t ddl::FTS::start_parse_threads ( Builder builder)
noexcept

Start the parsing, create the threads.

Returns
DB_SUCCESS or error code.

Member Data Documentation

◆ m_ctx

Context& ddl::FTS::m_ctx
private

DDL context.

◆ m_doc_id_32_bit

bool ddl::FTS::m_doc_id_32_bit {}
private

true if document ID should be stored as a 32 bit instead of a 64 bit.

◆ m_dup

Dup ddl::FTS::m_dup
private

Duplicate key reporting.

◆ m_index

dict_index_t* ddl::FTS::m_index {}
private

DDL index instance.

◆ m_inserter

Inserter* ddl::FTS::m_inserter {}
private

For inserting the rows parsed by the m_parsers.

◆ m_parsers

Parsers ddl::FTS::m_parsers {}
private

For parsing the documents, there is one per thread.

◆ m_sort_index

dict_index_t* ddl::FTS::m_sort_index {}
private

Temporary index instance with relevant FTS columns.

◆ m_table

dict_table_t* ddl::FTS::m_table {}
private

DDL table instance.

◆ m_threads

Threads ddl::FTS::m_threads {}
private

For tracking parser threads.


The documentation for this struct was generated from the following files: