MySQL 9.0.0
Source Code Documentation
binlog::Log_sanitizer Class Referenceabstract

Class used to recover binary / relay log file. More...

#include <log_sanitizer.h>

Inheritance diagram for binlog::Log_sanitizer:
[legend]

Public Member Functions

 Log_sanitizer ()
 Ctor. More...
 
virtual ~Log_sanitizer ()=default
 Dtor. More...
 
my_off_t get_valid_pos () const
 Retrieves the position of the last binlog/relay log event that ended a transaction or position after the RLE/FDE/SE that comes from the source. More...
 
std::pair< my_off_t, bool > get_valid_source_pos () const
 Retrieves the last valid source position of an event in read from the binary log / relay log file, which may be: More...
 
std::pair< std::string, bool > get_valid_source_file () const
 Retrieves the updated name of the binlog source file. More...
 
bool is_log_malformed () const
 Retrieves whether or not the log was correctly processed in full. More...
 
std::string const & get_failure_message () const
 Retrieves the textual representation of the encontered failure, if any. More...
 
std::string get_valid_file () const
 
bool is_log_truncation_needed () const
 Checks whether a valid sanitized log file needs truncation of the last, partially written transaction or events that cannot be safely read. More...
 
bool is_fatal_error () const
 Checks whether the fatal error occurred during log sanitization (OOM / decompression error which we cannot handle) More...
 

Protected Member Functions

virtual PSI_memory_keyget_memory_key () const =0
 Function used to obtain memory key for derived classes. More...
 
template<class Type_reader >
void process_logs (Type_reader &reader)
 This function goes through the opened file and searches for a valid position in a binary log file. More...
 
template<class Type_reader >
void process_logs (Type_reader &reader, const std::list< std::string > &list_of_files, MYSQL_BIN_LOG &log)
 This function goes iterates over the relay log files in the 'list_of_files' container, starting from the most recent one. More...
 
template<class Type_reader >
void process_logs (Type_reader &reader, MYSQL_BIN_LOG &log)
 This function will obtain the list of relay log files using the object of MYSQL_BIN_LOG class and iterate over them to find the last valid position within a relay log file. More...
 
template<class Type_reader >
bool process_one_log (Type_reader &reader, const std::string &filename)
 Reads and validates one log file. More...
 
void process_query_event (Query_log_event const &ev)
 Invoked when a Query_log_event is read from the binary log file reader. More...
 
void process_xid_event (Xid_log_event const &ev)
 Invoked when a Xid_log_event is read from the binary log file reader. More...
 
void process_xa_prepare_event (XA_prepare_log_event const &ev)
 Invoked when a XA_prepare_log_event is read from the binary log file reader. More...
 
void process_start ()
 Invoked when a BEGIN or an ‘XA START’ is found in a Query_log_event. More...
 
void process_commit ()
 Invoked when a COMMIT is found in a Query_log_event. More...
 
void process_rollback ()
 Invoked when a ROLLBACK is found in a Query_log_event. More...
 
void process_atomic_ddl (Query_log_event const &ev)
 Invoked when a DDL is found in a Query_log_event. More...
 
void process_xa_commit (std::string const &query)
 Invoked when an XA COMMIT is found in a Query_log_event. More...
 
void process_xa_rollback (std::string const &query)
 Invoked when an XA ROLLBACK is found in a Query_log_event. More...
 
void add_external_xid (std::string const &query, enum_ha_recover_xa_state state)
 Parses the provided string for an XID and adds it to the externally coordinated transactions map, along side the provided state. More...
 

Protected Attributes

bool m_validation_started {true}
 Indicates whether validation has started. More...
 
my_off_t m_valid_pos {0}
 Position of the last binlog/relay log event that ended a transaction. More...
 
my_off_t m_valid_source_pos {0}
 Position of the last binlog event that ended a transaction (source position which corresponds to m_valid_pos) More...
 
std::string m_valid_source_file {""}
 Currently processed binlog file set in case source rotation event is encountered. More...
 
std::string m_valid_file {""}
 Last log file containing finished transaction. More...
 
bool m_in_transaction {false}
 Whether or not the event being processed is within a transaction. More...
 
bool m_is_malformed {false}
 Whether or not the binary log is malformed/corrupted or error occurred. More...
 
bool m_fatal_error {false}
 Whether or not the binary log has a fatal error. More...
 
std::string m_failure_message {""}
 Textual representation of the encountered failure. More...
 
MEM_ROOT m_mem_root
 Memory pool to use for the XID lists. More...
 
Mem_root_allocator< my_xidm_set_alloc
 Memory pool allocator to use with the normal transaction list. More...
 
Mem_root_allocator< std::pair< const XID, XID_STATE::xa_states > > m_map_alloc
 Memory pool allocator to use with the XA transaction list. More...
 
Xid_commit_list m_internal_xids
 List of normal transactions fully written to the binary log. More...
 
Xa_state_list::list m_external_xids
 List of XA transactions and states that appear in the binary log. More...
 
bool m_is_log_truncation_needed {false}
 Information on whether log needs to be truncated, i.e. More...
 
bool m_has_valid_pos {false}
 Indicator on whether a valid position has been found in the log file. More...
 
bool m_has_valid_source_pos {false}
 Indicator on whether a valid source position has been found in the log file. More...
 
my_off_t m_last_file_size {0}
 Last opened file size. More...
 

Detailed Description

Class used to recover binary / relay log file.

This base class is responsible for finding the last valid position of a relay log / binary log file, meaning, the position of the last finished event which occurs outside of transaction boundary. Validation starts when first reliable position has been found, i.e.:

  • source rotation event
  • source FDE
  • source STOP event
  • first finished transaction:
    • Query log event with: COMMIT / ROLLBACK / XA COMMIT / XA ROLLBACK / atomic DDL
    • XID Log event Validation ends at the end of the binlog file / relay log file or in case further reading is not possible. Binary log recovery: Binary log file always start with an FDE which is the first and valid position within a file. Binary log files are never removed by a log sanitizer. Relay log recovery: If no valid position has been found in any of the relay log files, Log sanitizer will keep all of the relay log files. In case a valid position has been found in any of the first relay log files, relay log files that do not contain a valid position outside of a transaction boundary, will be removed.

Constructor & Destructor Documentation

◆ Log_sanitizer()

binlog::Log_sanitizer::Log_sanitizer ( )

Ctor.

◆ ~Log_sanitizer()

virtual binlog::Log_sanitizer::~Log_sanitizer ( )
virtualdefault

Dtor.

Member Function Documentation

◆ add_external_xid()

void binlog::Log_sanitizer::add_external_xid ( std::string const &  query,
enum_ha_recover_xa_state  state 
)
protected

Parses the provided string for an XID and adds it to the externally coordinated transactions map, along side the provided state.

Parameters
queryThe query to search and retrieve the XID from
stateThe state to add to the map, along side the XID

◆ get_failure_message()

std::string const & binlog::Log_sanitizer::get_failure_message ( ) const

Retrieves the textual representation of the encontered failure, if any.

Returns
the string containing the textual representation of the failure, an empty string otherwise.

◆ get_memory_key()

virtual PSI_memory_key & binlog::Log_sanitizer::get_memory_key ( ) const
protectedpure virtual

Function used to obtain memory key for derived classes.

Returns
Reference to a memory key

Implemented in binlog::Binlog_recovery, and rpl::Relay_log_sanitizer.

◆ get_valid_file()

std::string binlog::Log_sanitizer::get_valid_file ( ) const
inline

◆ get_valid_pos()

my_off_t binlog::Log_sanitizer::get_valid_pos ( ) const

Retrieves the position of the last binlog/relay log event that ended a transaction or position after the RLE/FDE/SE that comes from the source.

Returns
The position of the last binlog event that ended a transaction

◆ get_valid_source_file()

std::pair< std::string, bool > binlog::Log_sanitizer::get_valid_source_file ( ) const

Retrieves the updated name of the binlog source file.

Returns
Updated source file or empty string; indicator equal to true in case filename is valid

◆ get_valid_source_pos()

std::pair< my_off_t, bool > binlog::Log_sanitizer::get_valid_source_pos ( ) const

Retrieves the last valid source position of an event in read from the binary log / relay log file, which may be:

  • source position of the event ending a transaction
  • source position written in the source RLE
    Returns
    The position of the last binlog event that ended a transaction and indicator whether this position is valid

◆ is_fatal_error()

bool binlog::Log_sanitizer::is_fatal_error ( ) const

Checks whether the fatal error occurred during log sanitization (OOM / decompression error which we cannot handle)

Returns
true in case fatal error occurred, false otherwise

◆ is_log_malformed()

bool binlog::Log_sanitizer::is_log_malformed ( ) const

Retrieves whether or not the log was correctly processed in full.

Returns
true if the log processing ended with errors, false otherwise.

◆ is_log_truncation_needed()

bool binlog::Log_sanitizer::is_log_truncation_needed ( ) const

Checks whether a valid sanitized log file needs truncation of the last, partially written transaction or events that cannot be safely read.

Returns
true in case log file needs to be truncated, false otherwise

◆ process_atomic_ddl()

void binlog::Log_sanitizer::process_atomic_ddl ( Query_log_event const &  ev)
protected

Invoked when a DDL is found in a Query_log_event.

Actions taken to process the statement:

  • If m_in_transaction flag is set to true, m_is_malformed is set to true, indicating that the binary log is malformed.
  • The XID of the transaction is extracted and added to the list of internally coordinated transactions m_internal_xids.
  • If the XID already exists in the list, m_is_malformed is set to true, indicating that the binary log is malformed.
    Parameters
    evThe Query_log_event to process

◆ process_commit()

void binlog::Log_sanitizer::process_commit ( )
protected

Invoked when a COMMIT is found in a Query_log_event.

Actions taken to process the statement:

  • If m_in_transaction flag is set to false, m_is_malformed is set to true, indicating that the binary log is malformed.
  • The m_in_transaction flag is set to false, indicating that the event starts a transaction.

◆ process_logs() [1/3]

template<class Type_reader >
void binlog::Log_sanitizer::process_logs ( Type_reader &  reader)
protected

This function goes through the opened file and searches for a valid position in a binary log file.

It also gathers information about XA transactions which will be used during the binary log recovery

Parameters
readerLog reader, must be opened

◆ process_logs() [2/3]

template<class Type_reader >
void binlog::Log_sanitizer::process_logs ( Type_reader &  reader,
const std::list< std::string > &  list_of_files,
MYSQL_BIN_LOG log 
)
protected

This function goes iterates over the relay log files in the 'list_of_files' container, starting from the most recent one.

It gathers information about XA transactions and performs a small validation of the log files. Validation starts in case a first reliable position has been found (FDE/RLE/SE from the source or the end of a transaction), and proceeds till the end of file or until a read error has occurred. In case a valid position has been found within a file, relay log files that were created after this file will be removed. In case no valid position has been found within a file, sanitizer will iterate over events in the previous (older) relay log file. In case no valid position has been found in any of the files listed in the 'list_of_files' container, relay log files won't be removed. It may happen e.g. in case we cannot decrypt events.

Parameters
readerRelay log file reader object
list_of_filesThe list of relay logs we know, obtained from the relay log index
logMYSQL_BIN_LOG object used to manipulate relay log files

◆ process_logs() [3/3]

template<class Type_reader >
void binlog::Log_sanitizer::process_logs ( Type_reader &  reader,
MYSQL_BIN_LOG log 
)
protected

This function will obtain the list of relay log files using the object of MYSQL_BIN_LOG class and iterate over them to find the last valid position within a relay log file.

It will remove relay log files that contain only parts of the last, partially written transaction

Parameters
readerRelay log file reader object
logMYSQL_BIN_LOG object used to manipulate relay log files

◆ process_one_log()

template<class Type_reader >
bool binlog::Log_sanitizer::process_one_log ( Type_reader &  reader,
const std::string &  filename 
)
protected

Reads and validates one log file.

Parameters
[in]filenameName of the log file to process
[in]readerReference to reader able to read processed log file
Returns
true if processed log contains a valid log position outside of transaction boundaries

◆ process_query_event()

void binlog::Log_sanitizer::process_query_event ( Query_log_event const &  ev)
protected

Invoked when a Query_log_event is read from the binary log file reader.

The underlying query string is inspected to determine if the SQL command starts or ends a transaction. The following commands are searched for:

  • BEGIN
  • COMMIT
  • ROLLBACK
  • DDL
  • XA START
  • XA COMMIT
  • XA ROLLBACK Check below for the description of the action that is taken for each.
    Parameters
    evThe Query_log_event to process

◆ process_rollback()

void binlog::Log_sanitizer::process_rollback ( )
protected

Invoked when a ROLLBACK is found in a Query_log_event.

Actions taken to process the statement:

  • If m_in_transaction flag is set to false, m_is_malformed is set to true, indicating that the binary log is malformed.
  • The m_in_transaction flag is set to false, indicating that the event starts a transaction.

◆ process_start()

void binlog::Log_sanitizer::process_start ( )
protected

Invoked when a BEGIN or an ‘XA START’ is found in a Query_log_event.

Actions taken to process the statement:

  • If m_in_transaction flag is set to true, m_is_malformed is set to true, indicating that the binary log is malformed.
  • The m_in_transaction flag is set to true, indicating that the event starts a transaction.

◆ process_xa_commit()

void binlog::Log_sanitizer::process_xa_commit ( std::string const &  query)
protected

Invoked when an XA COMMIT is found in a Query_log_event.

Actions taken to process the statement:

  • If m_in_transaction flag is set to true, m_is_malformed is set to true, indicating that the binary log is malformed.
  • The m_in_transaction flag is set to false, indicating that the event ends a transaction.
  • The XID of the transaction is extracted and added to the list of externally coordinated transactions m_external_xids, alongside the state COMMITTED.
  • If the XID already exists in the list associated with a state other than PREPARED, m_is_malformed is set to true, indicating that the binary log is malformed.
    Parameters
    queryThe query string to process

◆ process_xa_prepare_event()

void binlog::Log_sanitizer::process_xa_prepare_event ( XA_prepare_log_event const &  ev)
protected

Invoked when a XA_prepare_log_event is read from the binary log file reader.

Actions taken to process the event:

  • If m_in_transaction flag is set to false, m_is_malformed is set to true, indicating that the binary log is malformed.
  • The m_in_transaction flag is set to false, indicating that the event ends a transaction.
  • The XID of the transaction is extracted and added to the list of externally coordinated transactions m_external_xids, along side the state COMMITTED if the event represents an XA COMMIT ONE_PHASE or PREPARED if not.
  • If the XID already exists in the list associated with a state other than COMMITTED or ROLLEDBACK, m_is_malformed is set to true, indicating that the binary log is malformed.
    Parameters
    evThe XA_prepare_log_event to process

◆ process_xa_rollback()

void binlog::Log_sanitizer::process_xa_rollback ( std::string const &  query)
protected

Invoked when an XA ROLLBACK is found in a Query_log_event.

Actions taken to process the statement:

  • If m_in_transaction flag is set to true, m_is_malformed is set to true, indicating that the binary log is malformed.
  • The m_in_transaction flag is set to false, indicating that the event ends a transaction.
  • The XID of the transaction is extracted and added to the list of externally coordinated transactions m_external_xids, along side the state ROLLEDBACK.
  • If the XID already exists in the list associated with a state other than PREPARED, m_is_malformed is set to true, indicating that the binary log is malformed.
    Parameters
    queryThe query string to process

◆ process_xid_event()

void binlog::Log_sanitizer::process_xid_event ( Xid_log_event const &  ev)
protected

Invoked when a Xid_log_event is read from the binary log file reader.

Actions taken to process the event:

  • If m_in_transaction flag is set to false, m_is_malformed is set to true, indicating that the binary log is malformed.
  • The m_in_transaction flag is set to false, indicating that the event ends a transaction.
  • The XID of the transaction is extracted and added to the list of internally coordinated transactions m_internal_xids.
  • If the XID already exists in the list, m_is_malformed is set to true, indicating that the binary log is malformed.
    Parameters
    evThe Xid_log_event to process

Member Data Documentation

◆ m_external_xids

Xa_state_list::list binlog::Log_sanitizer::m_external_xids
protected

List of XA transactions and states that appear in the binary log.

◆ m_failure_message

std::string binlog::Log_sanitizer::m_failure_message {""}
protected

Textual representation of the encountered failure.

◆ m_fatal_error

bool binlog::Log_sanitizer::m_fatal_error {false}
protected

Whether or not the binary log has a fatal error.

◆ m_has_valid_pos

bool binlog::Log_sanitizer::m_has_valid_pos {false}
protected

Indicator on whether a valid position has been found in the log file.

◆ m_has_valid_source_pos

bool binlog::Log_sanitizer::m_has_valid_source_pos {false}
protected

Indicator on whether a valid source position has been found in the log file.

◆ m_in_transaction

bool binlog::Log_sanitizer::m_in_transaction {false}
protected

Whether or not the event being processed is within a transaction.

◆ m_internal_xids

Xid_commit_list binlog::Log_sanitizer::m_internal_xids
protected

List of normal transactions fully written to the binary log.

◆ m_is_log_truncation_needed

bool binlog::Log_sanitizer::m_is_log_truncation_needed {false}
protected

Information on whether log needs to be truncated, i.e.

log is not ending at transaction boundary or we cannot read it till the end

◆ m_is_malformed

bool binlog::Log_sanitizer::m_is_malformed {false}
protected

Whether or not the binary log is malformed/corrupted or error occurred.

◆ m_last_file_size

my_off_t binlog::Log_sanitizer::m_last_file_size {0}
protected

Last opened file size.

◆ m_map_alloc

Mem_root_allocator<std::pair<const XID, XID_STATE::xa_states> > binlog::Log_sanitizer::m_map_alloc
protected

Memory pool allocator to use with the XA transaction list.

◆ m_mem_root

MEM_ROOT binlog::Log_sanitizer::m_mem_root
protected

Memory pool to use for the XID lists.

◆ m_set_alloc

Mem_root_allocator<my_xid> binlog::Log_sanitizer::m_set_alloc
protected

Memory pool allocator to use with the normal transaction list.

◆ m_valid_file

std::string binlog::Log_sanitizer::m_valid_file {""}
protected

Last log file containing finished transaction.

◆ m_valid_pos

my_off_t binlog::Log_sanitizer::m_valid_pos {0}
protected

Position of the last binlog/relay log event that ended a transaction.

◆ m_valid_source_file

std::string binlog::Log_sanitizer::m_valid_source_file {""}
protected

Currently processed binlog file set in case source rotation event is encountered.

◆ m_valid_source_pos

my_off_t binlog::Log_sanitizer::m_valid_source_pos {0}
protected

Position of the last binlog event that ended a transaction (source position which corresponds to m_valid_pos)

◆ m_validation_started

bool binlog::Log_sanitizer::m_validation_started {true}
protected

Indicates whether validation has started.

In case of relay log sanitization, we start validation when we are sure that we are at transaction boundary and we are able to recover source position, meaning, when we detect:

  • first encountered Rotation Event, that comes from the source
  • end of a transaction (Xid event, QLE containing COMMIT/ROLLBACK/XA COMMIT/XA ROLLBACK)
  • an atomic DDL transaction Since binary logs always start at transaction boundary, when doing a binary log recovery, we start validation right away. By default, we are assuming that we are in the binary log recovery procedure

The documentation for this class was generated from the following files: