Recovery system data structure. More...

#include <log0recv.h>

Classes
struct	Encryption_Key

struct	Mlog_record
	Mini transaction log record. More...

struct	Space
	Every space has its own heap and pages that belong to it. More...

Public Types
using	Pages = std::unordered_map< page_no_t, recv_addr_t *, std::hash< page_no_t >, std::equal_to< page_no_t > >

using	Missing_Ids = std::unordered_set< space_id_t >

using	Spaces = std::unordered_map< space_id_t, Space, std::hash< space_id_t >, std::equal_to< space_id_t > >

using	Encryption_Keys = std::vector< Encryption_Key >

using	Mlog_records = std::vector< Mlog_record, ut::allocator< Mlog_record > >

Public Member Functions
void	save_rec (size_t rec_num, space_id_t space_id, page_no_t page_no, mlog_id_t type, const byte *body, size_t len)
	Save mlog record information. More...

bool	get_saved_rec (size_t rec_num, space_id_t &space_id, page_no_t &page_no, mlog_id_t &type, const byte *&body, size_t &len)
	Return saved mlog record information, if there. More...

Public Attributes
ib_mutex_t	mutex

ib_mutex_t	writer_mutex
	mutex coordinating flushing between recv_writer_thread and the recovery thread. More...

os_event_t	flush_start
	event to activate page cleaner threads More...

os_event_t	flush_end
	event to signal that the page cleaner has finished the request More...

buf_flush_t	flush_type
	type of the flush request. More...

bool	apply_log_recs
	This is true when log rec application to pages is allowed; this flag tells the i/o-handler if it should do log record application. More...

bool	apply_batch_on
	This is true when a log rec application batch is running. More...

byte *	buf
	Buffer for parsing log records. More...

size_t	buf_len
	Size of the parsing buffer. More...

ulint	len
	Amount of data in buf. More...

lsn_t	parse_start_lsn
	This is the lsn from which we were able to start parsing log records and adding them to the hash table; zero if a suitable start point not found yet. More...

lsn_t	checkpoint_lsn
	Checkpoint lsn that was used during recovery (read from file). More...

ulint	bytes_to_ignore_before_checkpoint
	Number of data bytes to ignore until we reach checkpoint_lsn. More...

lsn_t	scanned_lsn
	The log data has been scanned up to this lsn. More...

uint32_t	scanned_epoch_no
	The log data has been scanned up to this epoch_no. More...

ulint	recovered_offset
	Start offset of non-parsed log records in buf. More...

lsn_t	recovered_lsn
	The log records have been parsed up to this lsn. More...

lsn_t	previous_recovered_lsn
	The previous value of recovered_lsn - before we parsed the last mtr. More...

lsn_t	last_block_first_mtr_boundary {}
	Tracks what should be the proper value of first_rec_group field in the header of the block to which recovered_lsn belongs. More...

bool	found_corrupt_log
	Set when finding a corrupt log block or record, or there is a log parsing buffer overflow. More...

bool	found_corrupt_fs
	Set when an inconsistency with the file system contents is detected during log scan or apply. More...

bool	is_cloned_db
	Data directory has been recognized as cloned data directory. More...

bool	is_meb_db
	Data directory has been recognized as data directory from MEB. More...

bool	dblwr_state
	Doublewrite buffer state before MEB recovery starts. More...

Spaces *	spaces
	Hash table of pages, indexed by SpaceID. More...

ulint	n_addrs
	Number of not processed hashed file addresses in the hash table. More...

dblwr::recv::DBLWR *	dblwr
	Doublewrite buffer pages, destroyed after recovery completes. More...

MetadataRecover *	metadata_recover
	We store and merge all table persistent data here during scanning redo logs. More...

Encryption_Keys *	keys
	Encryption Key information per tablespace ID. More...

Missing_Ids	missing_ids
	Tablespace IDs that were ignored during redo log apply. More...

Missing_Ids	deleted
	Tablespace IDs that were explicitly deleted. More...

Mlog_records	saved_recs

Static Public Attributes
static constexpr size_t	MAX_SAVED_MLOG_RECS = 8 * 1024
	While scanning logs for multi-record mini-transaction (mtr), we have two passes. More...

Detailed Description

Recovery system data structure.

Member Typedef Documentation

◆ Encryption_Keys

using recv_sys_t::Encryption_Keys = std::vector<Encryption_Key>

◆ Missing_Ids

using recv_sys_t::Missing_Ids = std::unordered_set<space_id_t>

◆ Mlog_records

using recv_sys_t::Mlog_records = std::vector<Mlog_record, ut::allocator<Mlog_record> >

◆ Pages

using recv_sys_t::Pages = std::unordered_map<page_no_t, recv_addr_t *, std::hash<page_no_t>, std::equal_to<page_no_t> >

◆ Spaces

using recv_sys_t::Spaces = std::unordered_map<space_id_t, Space, std::hash<space_id_t>, std::equal_to<space_id_t> >

Member Function Documentation

◆ get_saved_rec()

bool recv_sys_t::get_saved_rec	(	size_t	rec_num,
		space_id_t &	space_id,
		page_no_t &	page_no,
		mlog_id_t &	type,
		const byte *&	body,
		size_t &	len
	)

inline

Return saved mlog record information, if there.

Works only in single threaded recovery scanner.

Parameters

[in]	rec_num	record number in multi record group
[out]	space_id	space ID for the log record
[out]	page_no	page number for the log record
[out]	type	log record type
[out]	body	pointer to log record body in recovery buffer
[out]	len	length of the log record

Returns: true iff saved record data is found. mutex protecting the fields apply_log_recs, n_addrs, and the state field in each recv_addr struct

◆ save_rec()

void recv_sys_t::save_rec	(	size_t	rec_num,
		space_id_t	space_id,
		page_no_t	page_no,
		mlog_id_t	type,
		const byte *	body,
		size_t	len
	)

inline

Save mlog record information.

Silently returns if cannot save. Works only in single threaded recovery scanner.

Parameters

[in]	rec_num	record number in multi record group
[in]	space_id	space ID for the log record
[in]	page_no	page number for the log record
[in]	type	log record type
[in]	body	pointer to log record body in recovery buffer
[in]	len	length of the log record

Member Data Documentation

◆ apply_batch_on

bool recv_sys_t::apply_batch_on

This is true when a log rec application batch is running.

◆ apply_log_recs

bool recv_sys_t::apply_log_recs

This is true when log rec application to pages is allowed; this flag tells the i/o-handler if it should do log record application.

◆ buf

byte* recv_sys_t::buf

Buffer for parsing log records.

◆ buf_len

size_t recv_sys_t::buf_len

Size of the parsing buffer.

◆ bytes_to_ignore_before_checkpoint

ulint recv_sys_t::bytes_to_ignore_before_checkpoint

Number of data bytes to ignore until we reach checkpoint_lsn.

◆ checkpoint_lsn

lsn_t recv_sys_t::checkpoint_lsn

Checkpoint lsn that was used during recovery (read from file).

◆ dblwr

dblwr::recv::DBLWR* recv_sys_t::dblwr

Doublewrite buffer pages, destroyed after recovery completes.

◆ dblwr_state

bool recv_sys_t::dblwr_state

Doublewrite buffer state before MEB recovery starts.

We restore to this state after MEB recovery completes and disable the doublewrite buffer during MEB recovery.

◆ deleted

Missing_Ids recv_sys_t::deleted

Tablespace IDs that were explicitly deleted.

◆ flush_end

os_event_t recv_sys_t::flush_end

event to signal that the page cleaner has finished the request

◆ flush_start

os_event_t recv_sys_t::flush_start

event to activate page cleaner threads

◆ flush_type

buf_flush_t recv_sys_t::flush_type

type of the flush request.

BUF_FLUSH_LRU: flush end of LRU, keeping free blocks. BUF_FLUSH_LIST: flush all of blocks.

◆ found_corrupt_fs

bool recv_sys_t::found_corrupt_fs

Set when an inconsistency with the file system contents is detected during log scan or apply.

◆ found_corrupt_log

bool recv_sys_t::found_corrupt_log

Set when finding a corrupt log block or record, or there is a log parsing buffer overflow.

◆ is_cloned_db

bool recv_sys_t::is_cloned_db

Data directory has been recognized as cloned data directory.

◆ is_meb_db

bool recv_sys_t::is_meb_db

Data directory has been recognized as data directory from MEB.

◆ keys

Encryption_Keys* recv_sys_t::keys

Encryption Key information per tablespace ID.

◆ last_block_first_mtr_boundary

lsn_t recv_sys_t::last_block_first_mtr_boundary {}

Tracks what should be the proper value of first_rec_group field in the header of the block to which recovered_lsn belongs.

It might be also zero, in which case it means we do not know.

◆ len

ulint recv_sys_t::len

Amount of data in buf.

◆ MAX_SAVED_MLOG_RECS

constexpr size_t recv_sys_t::MAX_SAVED_MLOG_RECS = 8 * 1024

staticconstexpr

While scanning logs for multi-record mini-transaction (mtr), we have two passes.

In first pass, we check if all the logs of the mtr is present in current recovery buffer or not. If yes, then in second pass we go through the logs again the add to hash table for apply. To avoid parsing multiple times, we save the parsed records in first pass and reuse them in second pass.

Parsing of redo log takes significant amount of time and this optimization of avoiding second parse gave about 1.8x speed up on recovery scan time of 1G of redo log from sysbench rw test.

There is currently no limit for maximum number of logs in an mtr. Practically, from sysbench rw test recovery with 1G of redo log to recover from the record count were spread from 3 - 1235 with majority between 600 - 700. So, it is likely by saving 1k records we could avoid most of the re-parsing overhead. Considering possible bigger number of records in other load and future changes the limit for number of saved records is kept at 8k. The same value from the contribution patch. The memory requirement 32 x 8k = 256k seems fine as one time overhead for the entire instance.

◆ metadata_recover

MetadataRecover* recv_sys_t::metadata_recover

We store and merge all table persistent data here during scanning redo logs.

◆ missing_ids

Missing_Ids recv_sys_t::missing_ids

Tablespace IDs that were ignored during redo log apply.

◆ mutex

ib_mutex_t recv_sys_t::mutex

◆ n_addrs

ulint recv_sys_t::n_addrs

Number of not processed hashed file addresses in the hash table.

◆ parse_start_lsn

lsn_t recv_sys_t::parse_start_lsn

This is the lsn from which we were able to start parsing log records and adding them to the hash table; zero if a suitable start point not found yet.

◆ previous_recovered_lsn

lsn_t recv_sys_t::previous_recovered_lsn

The previous value of recovered_lsn - before we parsed the last mtr.

It is equal to recovered_lsn before we parsed any mtr. This is used to find moments in which recovered_lsn moves to the next block in which case we should update the last_block_first_rec_group (described below).

◆ recovered_lsn

lsn_t recv_sys_t::recovered_lsn

The log records have been parsed up to this lsn.

◆ recovered_offset

ulint recv_sys_t::recovered_offset

Start offset of non-parsed log records in buf.

◆ saved_recs

Mlog_records recv_sys_t::saved_recs

◆ scanned_epoch_no

uint32_t recv_sys_t::scanned_epoch_no

The log data has been scanned up to this epoch_no.

◆ scanned_lsn

lsn_t recv_sys_t::scanned_lsn

The log data has been scanned up to this lsn.

◆ spaces

Spaces* recv_sys_t::spaces

Hash table of pages, indexed by SpaceID.

◆ writer_mutex

ib_mutex_t recv_sys_t::writer_mutex

mutex coordinating flushing between recv_writer_thread and the recovery thread.

The documentation for this struct was generated from the following file:

storage/innobase/include/log0recv.h

Classes

Public Types

Public Member Functions

Public Attributes

Static Public Attributes