Full text search index builder. More...

#include <ddl0fts.h>

Classes
struct	Doc_item
	Information about temporary files used in merge sort. More...

struct	Inserter

struct	Parser
	For parsing and sorting the documents. More...

Public Member Functions
	FTS (Context &ctx, dict_index_t index, dict_table_t table) noexcept
	Constructor. More...

	~FTS () noexcept
	~Destructor. More...

dberr_t	init (size_t n_threads) noexcept
	Create the internal data structures. More...

dict_index_t *	index () noexcept

dict_index_t *	sort_index () noexcept

dberr_t	start_parse_threads (Builder *builder) noexcept
	Start the parsing, create the threads. More...

dberr_t	enqueue (Doc_item *doc_item) noexcept
	For sending the documents to parse to the parsing threads. More...

dberr_t	check_for_errors () noexcept
	Check for error status after the parsing has finished. More...

dberr_t	insert (Builder *builder) noexcept
	Start the merging and insert threads. More...

dberr_t	scan_finished (dberr_t err) noexcept
	Inform the parser threads that the scanning phase is complete so that they can shutdown after emptying the doc item queue. More...

Private Types
using	Threads = std::vector< std::thread >

using	Parsers = std::vector< Parser , ut::allocator< Parser > >

Private Member Functions
dberr_t	create (size_t n_threads) noexcept
	Create the data structures required to build the FTS index. More...

size_t	get_n_parsers () const noexcept

void	destroy () noexcept
	Destroy the data structures and clean up. More...

dberr_t	setup_insert_phase () noexcept
	Setup the insert phase inoput files generated by the parsers. More...

Static Private Member Functions
static dict_index_t *	create_index (dict_index_t index, dict_table_t table, bool *doc_id_32_bit) noexcept
	Create a temporary "fts sort index" used to merge sort the tokenized doc string. More...

Private Attributes
Parsers	m_parsers {}
	For parsing the documents, there is one per thread. More...

Inserter *	m_inserter {}
	For inserting the rows parsed by the m_parsers. More...

Context &	m_ctx
	DDL context. More...

Dup	m_dup
	Duplicate key reporting. More...

bool	m_doc_id_32_bit {}
	true if document ID should be stored as a 32 bit instead of a 64 bit. More...

dict_index_t *	m_index {}
	DDL index instance. More...

dict_table_t *	m_table {}
	DDL table instance. More...

dict_index_t *	m_sort_index {}
	Temporary index instance with relevant FTS columns. More...

Threads	m_threads {}
	For tracking parser threads. More...

Detailed Description

Full text search index builder.

Member Typedef Documentation

◆ Parsers

using ddl::FTS::Parsers = std::vector<Parser *, ut::allocator<Parser *> >

private

◆ Threads

using ddl::FTS::Threads = std::vector<std::thread>

private

Constructor & Destructor Documentation

◆ FTS()

ddl::FTS::FTS	(	Context &	ctx,
		dict_index_t *	index,
		dict_table_t *	table
	)

noexcept

Constructor.

Parameters

[in,out]	ctx	DDL context.
[in,out]	index	DDL index.
[in,out]	table	DDL table.

◆ ~FTS()

ddl::FTS::~FTS ( )

noexcept

~Destructor.

Member Function Documentation

◆ check_for_errors()

dberr_t ddl::FTS::check_for_errors ( )

noexcept

Check for error status after the parsing has finished.

Returns: DB_SUCCESS or error code.

◆ create()

dberr_t ddl::FTS::create ( size_t n_threads )

privatenoexcept

Create the data structures required to build the FTS index.

Parameters

[in] n_threads Number of parser threads.

Returns: DB_SUCCESS or error code.

◆ create_index()

dict_index_t * ddl::FTS::create_index	(	dict_index_t *	index,
		dict_table_t *	table,
		bool *	doc_id_32_bit
	)

staticprivatenoexcept

Create a temporary "fts sort index" used to merge sort the tokenized doc string.

The index has three "fields":

Tokenized word,
Doc ID
Word's position in original 'doc'.

Parameters

[in,out]	index	Index to sort.
[in,out]	table	Table that the FTS index is created on.
[out]	doc_id_32_bit	Whether to use 4 bytes instead of 7 bytes integer to store the DOC ID during sort.

Returns: dict_index_t structure for the fts sort index

◆ destroy()

void ddl::FTS::destroy ( )

privatenoexcept

Destroy the data structures and clean up.

◆ enqueue()

dberr_t ddl::FTS::enqueue ( FTS::Doc_item * doc_item )

noexcept

For sending the documents to parse to the parsing threads.

Parameters

[in,out] doc_item Document to parse, takes ownership.

Returns: DB_SUCCESS or error code, doc_item will be deleted either way.

◆ get_n_parsers()

size_t ddl::FTS::get_n_parsers ( ) const

inlineprivatenoexcept

Returns: the number of parses.

◆ index()

dict_index_t * ddl::FTS::index ( )

inlinenoexcept

Returns: the DDL index.

◆ init()

dberr_t ddl::FTS::init ( size_t n_threads )

noexcept

Create the internal data structures.

Parameters

[in] n_threads Number of parse threads to create.

Returns: DB_SUCCESS or error code.

◆ insert()

dberr_t ddl::FTS::insert ( Builder * builder )

noexcept

Start the merging and insert threads.

Parameters

[in,out] builder Builder instance to use.

Returns: DB_SUCCESS or error code.

◆ scan_finished()

dberr_t ddl::FTS::scan_finished ( dberr_t err )

noexcept

Inform the parser threads that the scanning phase is complete so that they can shutdown after emptying the doc item queue.

Parameters

[in] err Error status of the scanning thread(s).

Returns: DB_SUCCESS or error code.

◆ setup_insert_phase()

dberr_t ddl::FTS::setup_insert_phase ( )

privatenoexcept

Setup the insert phase inoput files generated by the parsers.

Returns: DB_SUCCESS or error code.

◆ sort_index()

dict_index_t * ddl::FTS::sort_index ( )

inlinenoexcept

Returns: the temporary sort index.

◆ start_parse_threads()

dberr_t ddl::FTS::start_parse_threads ( Builder * builder )

noexcept

Start the parsing, create the threads.

Returns: DB_SUCCESS or error code.

Member Data Documentation

◆ m_ctx

Context& ddl::FTS::m_ctx

private

DDL context.

◆ m_doc_id_32_bit

bool ddl::FTS::m_doc_id_32_bit {}

private

true if document ID should be stored as a 32 bit instead of a 64 bit.

◆ m_dup

Dup ddl::FTS::m_dup

private

Duplicate key reporting.

◆ m_index

dict_index_t* ddl::FTS::m_index {}

private

DDL index instance.

◆ m_inserter

Inserter* ddl::FTS::m_inserter {}

private

For inserting the rows parsed by the m_parsers.

◆ m_parsers

Parsers ddl::FTS::m_parsers {}

private

For parsing the documents, there is one per thread.

◆ m_sort_index

dict_index_t* ddl::FTS::m_sort_index {}

private

Temporary index instance with relevant FTS columns.

◆ m_table

dict_table_t* ddl::FTS::m_table {}

private

DDL table instance.

◆ m_threads

Threads ddl::FTS::m_threads {}

private

For tracking parser threads.

The documentation for this struct was generated from the following files:

storage/innobase/include/ddl0fts.h
storage/innobase/ddl/ddl0fts.cc

Classes

Public Member Functions

Private Types

Private Member Functions

Static Private Member Functions

Private Attributes

Detailed Description

Member Typedef Documentation

◆ Parsers

◆ Threads

Constructor & Destructor Documentation

◆ FTS()

◆ ~FTS()

Member Function Documentation

◆ check_for_errors()

◆ create()

◆ create_index()

◆ destroy()

◆ enqueue()

◆ get_n_parsers()

◆ index()

◆ init()

◆ insert()

◆ scan_finished()

◆ setup_insert_phase()

◆ sort_index()

◆ start_parse_threads()

Member Data Documentation

◆ m_ctx

◆ m_doc_id_32_bit

◆ m_dup

◆ m_index

◆ m_inserter

◆ m_parsers

◆ m_sort_index

◆ m_table

◆ m_threads