MySQL  8.0.27
Source Code Documentation
Commit_order_manager Class Reference

On a replica and only on a replica, this class is responsible for commiting the applyed transactions in the same order as was observed on the source. More...

#include <rpl_replica_commit_order_manager.h>

Public Member Functions

 Commit_order_manager (uint32 worker_numbers)
 
 Commit_order_manager (const Commit_order_manager &)=delete
 
 ~Commit_order_manager ()
 
Commit_order_manageroperator= (const Commit_order_manager &)=delete
 
void init_worker_context (Slave_worker &worker)
 Initializes the MDL context for a given worker in the commit order queue. More...
 
void register_trx (Slave_worker *worker)
 Register the worker into commit order queue when coordinator dispatches a transaction to the worker. More...
 
bool visit_lock_graph (Commit_order_lock_graph &wait_for_commit, MDL_wait_for_graph_visitor &gvisitor)
 Determines if the worker holding the commit order wait ticket `wait_for_commit is in deadlock with the MDL context encapsulated in the visitor parameter. More...
 

Static Public Member Functions

static void check_and_report_deadlock (THD *thd_self, THD *thd_wait_for)
 Check if order commit deadlock happens. More...
 
static bool wait (THD *thd)
 Wait for its turn to commit or unregister. More...
 
static void wait_and_finish (THD *thd, bool error)
 Wait for its turn to unregister and signal the next one to go ahead. More...
 
static bool get_rollback_status (THD *thd)
 Get transaction rollback status. More...
 
static void finish_one (THD *thd)
 Unregister the thread from the commit order queue and signal the next thread to awake. More...
 
static bool wait_for_its_turn_before_flush_stage (THD *thd)
 Determines whether current thread needs to wait for its turn to commit and unregister from the commit order queue. More...
 

Private Member Functions

bool wait_on_graph (Slave_worker *worker)
 Determines if the worker passed as a parameter must wait on the MDL graph for other workers to commit and, if it must, will wait for it's turn to commit. More...
 
bool wait (Slave_worker *worker)
 Wait for its turn to commit or unregister. More...
 
void finish_one (Slave_worker *worker)
 Unregister the thread from the commit order queue and signal the next thread to awake. More...
 
void finish (Slave_worker *worker)
 Unregister the transaction from the commit order queue and signal the next one to go ahead. More...
 
void reset_server_status (THD *first_thd)
 Reset server_status value of the commit group. More...
 
bool get_rollback_status ()
 Get rollback status. More...
 
void set_rollback_status ()
 Set rollback status to true. More...
 
void unset_rollback_status ()
 Unset rollback status to false. More...
 
void report_deadlock (Slave_worker *worker)
 
void flush_engine_and_signal_threads (Slave_worker *worker)
 Flush record of transactions for all the waiting threads and then awake them from their wait. More...
 

Private Attributes

std::atomic< bool > m_rollback_trx
 
cs::apply::Commit_order_queue m_workers
 

Detailed Description

On a replica and only on a replica, this class is responsible for commiting the applyed transactions in the same order as was observed on the source.

The key components of the commit order management are:

  • This class, that wraps the commit order management, allowing for API clients to schedule workers for committing, make workers wait for their turn to commit, finish up a scheduled worker task and allow for others to progress.
  • A commit order queue of type cs::apply::Commit_order_queue that holds the sequence by which worker threads should commit and the committing order state for each of the schduled workers.
  • The MDL infra-structure which allows for: one worker to wait for another to finish when transactions need to be committed in order; detect deadlocks involving workers waiting on each other for their turn to commit and non-worker threads waiting on meta-data locks held by worker threads.

The worker thread progress stages relevant to the commit order management are:

  • REGISTERED: the worker thread as been added to the commit order queue by the coordinator and is allowed to start applying the transaction.
  • FINISHED APPLYING: the worker thread just finished applying the transaction and checks if it needs to wait for a preceding worker to finish commiting.
  • REQUESTED GRANT: the worker thread waits on the MDL graph for the preceding worker to finish commiting.
  • WAITED: the worker thread finished waiting (either is the first in the commit order queue or has just been grantted permission to continue).
  • RELEASE NEXT: the worker thread removes itself from the commit order queue, checks if there is any worker waiting on the commit order and releases such worker iff is the preceding worker for the waiting worker.
  • FINISHED: the worker marks itself as available to take on another transaction to apply.

The progress of the worker within the stages:

                               +-------------------------+
                               |                         |
                               v                         |
                          [REGISTERED]                   |
                               |                         |
                               v                         |
                      [FINISHED APPLYING]                |
                               |                         |
                          Worker is                      |
                      first in the queue?                |
                             /   \                       |
                        yes /     \ no                   |
                           /       v                     |
                           \    [REQUESTED GRANT]        |
                            \     /                      |
                             \   /                       |
                              \ /                        |
                               |                         |
                               v                         |
                           [WAITED]                      |
                               |                         |
                               v                         |
                        [RELEASE NEXT]                   |
                               |                         |
                               v                         |
                          [FINISHED]                     |
                               |                         |
                               +-------------------------+

Lock-free structures and atomic access to variables are used to manage the commit order queue and to keep the worker stage transitions. This means that there is no atomicity in regards to changes performed in the queue or in the MDL graph within a given stage. Hence, stages maybe skipped and sequentially scheduled worker threads may overlap in the same stage.

In the context of the following tables, let W1 be a worker that is scheduled to commit before some other worker W2.

The behavior of W2 (rows) towards W1 (columns) in regards to thread synchronization, based on the stage of each thread: +---------—+--------------------------------------------------------------—+ | \ W1 | REGISTERED | FINISHED | REQUESTED | WAITED | RELEASE | FINISHED | | W2 \ | | APPLYING | GRANT | | NEXT | | +---------—+---------—+-------—+--------—+-----—+------—+-------—+ | REGISTERED | | | | | | | +---------—+---------—+-------—+--------—+-----—+------—+-------—+ | FIN. APPL. | | | | | | | +---------—+---------—+-------—+--------—+-----—+------—+-------—+ | REQ. GRANT | WAIT | WAIT | WAIT | WAIT | WAIT | | +---------—+---------—+-------—+--------—+-----—+------—+-------—+ | WAITED | | | | | | | +---------—+---------—+-------—+--------—+-----—+------—+-------—+ | REL. NEXT | | | | | WAIT | | +---------—+---------—+-------—+--------—+-----—+------—+-------—+ | FINISHED | | | | | | | +---------------------------------------------------------------------------—+

The W2 wait when both worker threads are in the RELEASE NEXT stage happens in the case W2 never entered the REQUESTED GRANT stage. This case may happen if W1 being in RELEASE NEXT removes itself from the queue before W2 enters FINISHED APPLYING and then W2 reaches the RELEASE NEXT stage before W1 exits it:

       [W1]                                [W2]

stage = RELEASE NEXT stage = REGISTERED | | v | queue.pop() v | stage = FINISHED_APPLYING | | v v next_worker.stage queue.front() == W2 == FINISHED_APPLYING | | | | v | stage = WAITED | | | v | stage = RELEASE NEXT | | v v next_worker.release() queue.pop()

The commit order queue includes mechanisms that block the poping until the preceding worker finishes the releasing operation. This wait will only be active for the amount of time that takes for W1 to change the values of the MDL graph structures needed to release W2, which is a very small amount of cycles.

The behavior of W1 (rows) towards W2 (columns)in regards to thread synchronization, based on the stage of each thread: +---------—+--------------------------------------------------------------—+ | \ W2 | REGISTERED | FINISHED | REQUESTED | WAITED | RELEASE | FINISHED | | W1 \ | | APPLYING | GRANT | | NEXT | | +---------—+---------—+-------—+--------—+-----—+------—+-------—+ | REGISTERED | | | | | | | +---------—+---------—+-------—+--------—+-----—+------—+-------—+ | FIN. APPL. | | | | | | | +---------—+---------—+-------—+--------—+-----—+------—+-------—+ | REQ. GRANT | | | | | | | +---------—+---------—+-------—+--------—+-----—+------—+-------—+ | WAITED | | | | | | | +---------—+---------—+-------—+--------—+-----—+------—+-------—+ | REL. NEXT | | GRANT | GRANT | | | | +---------—+---------—+-------—+--------—+-----—+------—+-------—+ | FINISHED | | | | | | | +---------------------------------------------------------------------------—+

The W1 grant to W2 may happen when W2 is either in the FINISHED APPLYING or REQUESTED GRANT stages. W1 must also signal the grant when W2 is in FINISHED APPLYING because W1 has no way to determine if W2 has already evaluated the first element of the queue or not, that is, W1 can't determine if W2 will proceed to the REQUESTED GRANT or to the WAITED stage. Therefore, W1 will signal in both cases.

Constructor & Destructor Documentation

◆ Commit_order_manager() [1/2]

Commit_order_manager::Commit_order_manager ( uint32  worker_numbers)

◆ Commit_order_manager() [2/2]

Commit_order_manager::Commit_order_manager ( const Commit_order_manager )
delete

◆ ~Commit_order_manager()

Commit_order_manager::~Commit_order_manager ( )
default

Member Function Documentation

◆ check_and_report_deadlock()

void Commit_order_manager::check_and_report_deadlock ( THD thd_self,
THD thd_wait_for 
)
static

Check if order commit deadlock happens.

Worker1(trx1) Worker2(trx2) ============= ============= ... ... Engine acquires lock A ... Engine acquires lock A(waiting for trx1 to release it. COMMIT(waiting for trx2 to commit first).

Currently, there are two corner cases can cause the deadlock.

  • Case 1 CREATE TABLE t1(c1 INT PRIMARY KEY, c2 INT, INDEX(c2)) ENGINE = InnoDB; INSERT INTO t1 VALUES(1, NULL),(2, 2), (3, NULL), (4, 4), (5, NULL), (6, 6)

    INSERT INTO t1 VALUES(7, NULL); DELETE FROM t1 WHERE c2 <= 3;

  • Case 2 ANALYZE TABLE t1; INSERT INTO t2 SELECT * FROM mysql.innodb_table_stats

Since this is not a real lock deadlock, it could not be handled by engine. slave need to handle it separately. Worker1(trx1) Worker2(trx2) ============= ============= ... ... Engine acquires lock A ... Engine acquires lock A.

  1. found trx1 is holding the lock.
  2. report the lock wait to server code by calling thd_report_row_lock_wait(). Then this function is called to check if it causes a order commit deadlock. Report the deadlock to worker1.
  3. waiting for trx1 to release it. COMMIT(waiting for trx2 to commit first). Found the deadlock flag set by worker2 and then return with ER_LOCK_DEADLOCK.

Rollback the transaction Get lock A and go ahead. ... Retry the transaction

To conclude, The transaction A which is waiting for transaction B to commit and is holding a lock which is required by transaction B will be rolled back and try again later.

Parameters
[in]thd_selfThe THD object of self session which is acquiring a lock hold by another session.
[in]thd_wait_forThe THD object of a session which is holding a lock being acquired by current session.

◆ finish()

void Commit_order_manager::finish ( Slave_worker worker)
private

Unregister the transaction from the commit order queue and signal the next one to go ahead.

Parameters
[in]workerThe worker which is executing the transaction.

◆ finish_one() [1/2]

void Commit_order_manager::finish_one ( Slave_worker worker)
private

Unregister the thread from the commit order queue and signal the next thread to awake.

Parameters
[in]workerThe worker which is executing the transaction.

◆ finish_one() [2/2]

void Commit_order_manager::finish_one ( THD thd)
static

Unregister the thread from the commit order queue and signal the next thread to awake.

Parameters
[in]thdThe THD object of current thread.

◆ flush_engine_and_signal_threads()

void Commit_order_manager::flush_engine_and_signal_threads ( Slave_worker worker)
private

Flush record of transactions for all the waiting threads and then awake them from their wait.

It also calls gtid_state->update_commit_group() which updates both the THD and the Gtid_state for whole commit group to reflect that the transaction set of transactions has ended.

Parameters
[in]workerThe worker which is executing the transaction.

◆ get_rollback_status() [1/2]

bool Commit_order_manager::get_rollback_status ( )
private

Get rollback status.

Return values
trueTransactions in the queue should rollback.
falseTransactions in the queue shouldn't rollback.

◆ get_rollback_status() [2/2]

bool Commit_order_manager::get_rollback_status ( THD thd)
static

Get transaction rollback status.

Parameters
[in]thdThe THD object of current thread.
Return values
trueCurrent transaction should rollback.
falseCurrent transaction shouldn't rollback.

◆ init_worker_context()

void Commit_order_manager::init_worker_context ( Slave_worker worker)

Initializes the MDL context for a given worker in the commit order queue.

Parameters
workerThe worker to initialize the context for

◆ operator=()

Commit_order_manager& Commit_order_manager::operator= ( const Commit_order_manager )
delete

◆ register_trx()

void Commit_order_manager::register_trx ( Slave_worker worker)

Register the worker into commit order queue when coordinator dispatches a transaction to the worker.

Parameters
[in]workerThe worker which the transaction will be dispatched to.

◆ report_deadlock()

void Commit_order_manager::report_deadlock ( Slave_worker worker)
private

◆ reset_server_status()

void Commit_order_manager::reset_server_status ( THD first_thd)
private

Reset server_status value of the commit group.

Parameters
[in]first_thdThe first thread of the commit group that needs server_status to be updated.

◆ set_rollback_status()

void Commit_order_manager::set_rollback_status ( )
private

Set rollback status to true.

◆ unset_rollback_status()

void Commit_order_manager::unset_rollback_status ( )
private

Unset rollback status to false.

◆ visit_lock_graph()

bool Commit_order_manager::visit_lock_graph ( Commit_order_lock_graph wait_for_commit,
MDL_wait_for_graph_visitor gvisitor 
)

Determines if the worker holding the commit order wait ticket `wait_for_commit is in deadlock with the MDL context encapsulated in the visitor parameter.

Parameters
wait_for_commit
gvisitorThe MDL graph visitor to check for deadlocks against.
Returns
true if a deadlock has been found and false otherwise.

◆ wait() [1/2]

bool Commit_order_manager::wait ( Slave_worker worker)
private

Wait for its turn to commit or unregister.

Parameters
[in]workerThe worker which is executing the transaction.
Return values
falseAll previous transactions succeed, so this transaction can go ahead and commit.
trueOne or more previous transactions rollback, so this transaction should rollback.

◆ wait() [2/2]

bool Commit_order_manager::wait ( THD thd)
static

Wait for its turn to commit or unregister.

Parameters
[in]thdThe THD object of current thread.
Return values
falseAll previous transactions succeed, so this transaction can go ahead and commit.
trueThe transaction is marked to rollback.

◆ wait_and_finish()

void Commit_order_manager::wait_and_finish ( THD thd,
bool  error 
)
static

Wait for its turn to unregister and signal the next one to go ahead.

In case error happens while processing transaction, notify the following transaction to rollback.

Parameters
[in]thdThe THD object of current thread.
[in]errorIf true failure in transaction execution

◆ wait_for_its_turn_before_flush_stage()

bool Commit_order_manager::wait_for_its_turn_before_flush_stage ( THD thd)
static

Determines whether current thread needs to wait for its turn to commit and unregister from the commit order queue.

The sql commands ALTER TABLE, DROP TABLE, DROP DB, OPTIMIZE TABLE, ANALYZE TABLE and REPAIR TABLE are allowed to wait for its turn to commit and unregister from the commit order queue as exception in MYSQL_BIN_LOG::ordered_commit(), as these transactions have multiple commits and so not determined if the call is ending transaction.

Parameters
[in]thdThe THD object of current thread.
Return values
trueAllow thread to wait for it turn
falseDo not allow thread to wait for it turn

◆ wait_on_graph()

bool Commit_order_manager::wait_on_graph ( Slave_worker worker)
private

Determines if the worker passed as a parameter must wait on the MDL graph for other workers to commit and, if it must, will wait for it's turn to commit.

Parameters
workerThe worker to determine the commit waiting status for.
Returns
false if the worker is ready to commit, true if not.

Member Data Documentation

◆ m_rollback_trx

std::atomic<bool> Commit_order_manager::m_rollback_trx
private

◆ m_workers

cs::apply::Commit_order_queue Commit_order_manager::m_workers
private

The documentation for this class was generated from the following files: