On a replica and only on a replica, this class is responsible for committing the applied transactions in the same order as was observed on the source. More...

#include <rpl_replica_commit_order_manager.h>

Public Member Functions
	Commit_order_manager (uint32 worker_numbers)

	Commit_order_manager (const Commit_order_manager &)=delete

	~Commit_order_manager ()

Commit_order_manager &	operator= (const Commit_order_manager &)=delete

void	init_worker_context (Slave_worker &worker)
	Initializes the MDL context for a given worker in the commit order queue. More...

void	register_trx (Slave_worker *worker)
	Register the worker into commit order queue when coordinator dispatches a transaction to the worker. More...

bool	visit_lock_graph (Commit_order_lock_graph &wait_for_commit, MDL_wait_for_graph_visitor &gvisitor)
	Determines if the worker holding the commit order wait ticket `wait_for_commit is in deadlock with the MDL context encapsulated in the visitor parameter. More...

Static Public Member Functions
static void	check_and_report_deadlock (THD thd_self, THD thd_wait_for)
	Check if order commit deadlock happens. More...

static bool	wait (THD *thd)
	Wait for its turn to commit or unregister. More...

static void	wait_and_finish (THD *thd, bool error)
	Wait for its turn to unregister and signal the next one to go ahead. More...

static bool	get_rollback_status (THD *thd)
	Get transaction rollback status. More...

static void	finish_one (THD *thd)
	Unregister the thread from the commit order queue and signal the next thread to awake. More...

static bool	wait_for_its_turn_before_flush_stage (THD *thd)
	Determines whether current thread needs to wait for its turn to commit and unregister from the commit order queue. More...

Private Member Functions
bool	wait_on_graph (Slave_worker *worker)
	Determines if the worker passed as a parameter must wait on the MDL graph for other workers to commit and, if it must, will wait for it's turn to commit. More...

bool	wait (Slave_worker *worker)
	Wait for its turn to commit or unregister. More...

void	finish_one (Slave_worker *worker)
	Unregister the thread from the commit order queue and signal the next thread to awake. More...

void	finish (Slave_worker *worker)
	Unregister the transaction from the commit order queue and signal the next one to go ahead. More...

void	reset_server_status (THD *first_thd)
	Reset server_status value of the commit group. More...

bool	get_rollback_status ()
	Get rollback status. More...

void	set_rollback_status ()
	Set rollback status to true. More...

void	unset_rollback_status ()
	Unset rollback status to false. More...

void	report_deadlock (Slave_worker *worker)

void	flush_engine_and_signal_threads (Slave_worker *worker)
	Flush record of transactions for all the waiting threads and then awake them from their wait. More...

Private Attributes
std::atomic< bool >	m_rollback_trx

cs::apply::Commit_order_queue	m_workers

Detailed Description

On a replica and only on a replica, this class is responsible for committing the applied transactions in the same order as was observed on the source.

The key components of the commit order management are:

This class, that wraps the commit order management, allowing for API clients to schedule workers for committing, make workers wait for their turn to commit, finish up a scheduled worker task and allow for others to progress.
A commit order queue of type cs::apply::Commit_order_queue that holds the sequence by which worker threads should commit and the committing order state for each of the scheduled workers.
The MDL infra-structure which allows for: one worker to wait for another to finish when transactions need to be committed in order; detect deadlocks involving workers waiting on each other for their turn to commit and non-worker threads waiting on meta-data locks held by worker threads.

The worker thread progress stages relevant to the commit order management are:

REGISTERED: the worker thread as been added to the commit order queue by the coordinator and is allowed to start applying the transaction.
FINISHED APPLYING: the worker thread just finished applying the transaction and checks if it needs to wait for a preceding worker to finish committing.
REQUESTED GRANT: the worker thread waits on the MDL graph for the preceding worker to finish committing.
WAITED: the worker thread finished waiting (either is the first in the commit order queue or has just been grantted permission to continue).
RELEASE NEXT: the worker thread removes itself from the commit order queue, checks if there is any worker waiting on the commit order and releases such worker iff is the preceding worker for the waiting worker.
FINISHED: the worker marks itself as available to take on another transaction to apply.

The progress of the worker within the stages:

                               +-------------------------+
                               |                         |
                               v                         |
                          [REGISTERED]                   |
                               |                         |
                               v                         |
                      [FINISHED APPLYING]                |
                               |                         |
                          Worker is                      |
                      first in the queue?                |
                             /   \                       |
                        yes /     \ no                   |
                           /       v                     |
                           \    [REQUESTED GRANT]        |
                            \     /                      |
                             \   /                       |
                              \ /                        |
                               |                         |
                               v                         |
                           [WAITED]                      |
                               |                         |
                               v                         |
                        [RELEASE NEXT]                   |
                               |                         |
                               v                         |
                          [FINISHED]                     |
                               |                         |
                               +-------------------------+

Lock-free structures and atomic access to variables are used to manage the commit order queue and to keep the worker stage transitions. This means that there is no atomicity in regards to changes performed in the queue or in the MDL graph within a given stage. Hence, stages maybe skipped and sequentially scheduled worker threads may overlap in the same stage.

In the context of the following tables, let W1 be a worker that is scheduled to commit before some other worker W2.

The W2 wait when both worker threads are in the RELEASE NEXT stage happens in the case W2 never entered the REQUESTED GRANT stage. This case may happen if W1 being in RELEASE NEXT removes itself from the queue before W2 enters FINISHED APPLYING and then W2 reaches the RELEASE NEXT stage before W1 exits it:

       [W1]                                [W2]

The commit order queue includes mechanisms that block the popping until the preceding worker finishes the releasing operation. This wait will only be active for the amount of time that takes for W1 to change the values of the MDL graph structures needed to release W2, which is a very small amount of cycles.

The W1 grant to W2 may happen when W2 is either in the FINISHED APPLYING or REQUESTED GRANT stages. W1 must also signal the grant when W2 is in FINISHED APPLYING because W1 has no way to determine if W2 has already evaluated the first element of the queue or not, that is, W1 can't determine if W2 will proceed to the REQUESTED GRANT or to the WAITED stage. Therefore, W1 will signal in both cases.

Constructor & Destructor Documentation

◆ Commit_order_manager() [1/2]

Commit_order_manager::Commit_order_manager ( uint32 worker_numbers )

◆ Commit_order_manager() [2/2]

Commit_order_manager::Commit_order_manager ( const Commit_order_manager & )

delete

◆ ~Commit_order_manager()

Commit_order_manager::~Commit_order_manager ( )

default

Member Function Documentation

◆ check_and_report_deadlock()

void Commit_order_manager::check_and_report_deadlock	(	THD *	thd_self,
		THD *	thd_wait_for
	)

static

Check if order commit deadlock happens.

Worker1(trx1) Worker2(trx2) ============= ============= ... ... Engine acquires lock A ... Engine acquires lock A(waiting for trx1 to release it. COMMIT(waiting for trx2 to commit first).

Currently, there are two corner cases can cause the deadlock.

Case 1 CREATE TABLE t1(c1 INT PRIMARY KEY, c2 INT, INDEX(c2)) ENGINE = InnoDB; INSERT INTO t1 VALUES(1, NULL),(2, 2), (3, NULL), (4, 4), (5, NULL), (6, 6)

INSERT INTO t1 VALUES(7, NULL); DELETE FROM t1 WHERE c2 <= 3;
Case 2 ANALYZE TABLE t1; INSERT INTO t2 SELECT * FROM mysql.innodb_table_stats

Since this is not a real lock deadlock, it could not be handled by engine. slave need to handle it separately. Worker1(trx1) Worker2(trx2) ============= ============= ... ... Engine acquires lock A ... Engine acquires lock A.

found trx1 is holding the lock.
report the lock wait to server code by calling thd_report_row_lock_wait(). Then this function is called to check if it causes a order commit deadlock. Report the deadlock to worker1.
waiting for trx1 to release it. COMMIT(waiting for trx2 to commit first). Found the deadlock flag set by worker2 and then return with ER_LOCK_DEADLOCK.

Rollback the transaction Get lock A and go ahead. ... Retry the transaction

To conclude, The transaction A which is waiting for transaction B to commit and is holding a lock which is required by transaction B will be rolled back and try again later.

Parameters

[in]	thd_self	The THD object of self session which is acquiring a lock hold by another session.
[in]	thd_wait_for	The THD object of a session which is holding a lock being acquired by current session.

◆ finish()

void Commit_order_manager::finish ( Slave_worker * worker )

private

Unregister the transaction from the commit order queue and signal the next one to go ahead.

Parameters

[in] worker The worker which is executing the transaction.

◆ finish_one() [1/2]

void Commit_order_manager::finish_one ( Slave_worker * worker )

private

Unregister the thread from the commit order queue and signal the next thread to awake.

Parameters

[in] worker The worker which is executing the transaction.

◆ finish_one() [2/2]

void Commit_order_manager::finish_one ( THD * thd )

static

Unregister the thread from the commit order queue and signal the next thread to awake.

Parameters

[in] thd The THD object of current thread.

◆ flush_engine_and_signal_threads()

void Commit_order_manager::flush_engine_and_signal_threads ( Slave_worker * worker )

private

Flush record of transactions for all the waiting threads and then awake them from their wait.

It also calls gtid_state->update_commit_group() which updates both the THD and the Gtid_state for whole commit group to reflect that the transaction set of transactions has ended.

Parameters

[in] worker The worker which is executing the transaction.

◆ get_rollback_status() [1/2]

bool Commit_order_manager::get_rollback_status ( )

private

Get rollback status.

Return values

true	Transactions in the queue should rollback.
false	Transactions in the queue shouldn't rollback.

◆ get_rollback_status() [2/2]

bool Commit_order_manager::get_rollback_status ( THD * thd )

static

Get transaction rollback status.

Parameters

[in] thd The THD object of current thread.

Return values

true	Current transaction should rollback.
false	Current transaction shouldn't rollback.

◆ init_worker_context()

void Commit_order_manager::init_worker_context ( Slave_worker & worker )

Initializes the MDL context for a given worker in the commit order queue.

Parameters

worker The worker to initialize the context for

◆ operator=()

Commit_order_manager & Commit_order_manager::operator= ( const Commit_order_manager & )

delete

◆ register_trx()

void Commit_order_manager::register_trx ( Slave_worker * worker )

Register the worker into commit order queue when coordinator dispatches a transaction to the worker.

Parameters

[in] worker The worker which the transaction will be dispatched to.

◆ report_deadlock()

void Commit_order_manager::report_deadlock ( Slave_worker * worker )

private

◆ reset_server_status()

void Commit_order_manager::reset_server_status ( THD * first_thd )

private

Reset server_status value of the commit group.

Parameters

[in] first_thd The first thread of the commit group that needs server_status to be updated.

◆ set_rollback_status()

void Commit_order_manager::set_rollback_status ( )

private

Set rollback status to true.

◆ unset_rollback_status()

void Commit_order_manager::unset_rollback_status ( )

private

Unset rollback status to false.

◆ visit_lock_graph()

bool Commit_order_manager::visit_lock_graph	(	Commit_order_lock_graph &	wait_for_commit,
		MDL_wait_for_graph_visitor &	gvisitor
	)

Determines if the worker holding the commit order wait ticket `wait_for_commit is in deadlock with the MDL context encapsulated in the visitor parameter.

Parameters

wait_for_commit	The wait ticket being held by the worker thread.
gvisitor	The MDL graph visitor to check for deadlocks against.

Returns: true if a deadlock has been found and false otherwise.

◆ wait() [1/2]

bool Commit_order_manager::wait ( Slave_worker * worker )

private

Wait for its turn to commit or unregister.

Parameters

[in] worker The worker which is executing the transaction.

Return values

false	All previous transactions succeed, so this transaction can go ahead and commit.
true	One or more previous transactions rollback, so this transaction should rollback.

◆ wait() [2/2]

bool Commit_order_manager::wait ( THD * thd )

static

Wait for its turn to commit or unregister.

Parameters

[in] thd The THD object of current thread.

Return values

false	All previous transactions succeed, so this transaction can go ahead and commit.
true	The transaction is marked to rollback.

◆ wait_and_finish()

void Commit_order_manager::wait_and_finish	(	THD *	thd,
		bool	error
	)

static

Wait for its turn to unregister and signal the next one to go ahead.

In case error happens while processing transaction, notify the following transaction to rollback.

Parameters

[in]	thd	The THD object of current thread.
[in]	error	If true failure in transaction execution

◆ wait_for_its_turn_before_flush_stage()

bool Commit_order_manager::wait_for_its_turn_before_flush_stage ( THD * thd )

static

Determines whether current thread needs to wait for its turn to commit and unregister from the commit order queue.

The sql commands ALTER TABLE, DROP TABLE, DROP DB, OPTIMIZE TABLE, ANALYZE TABLE and REPAIR TABLE are allowed to wait for its turn to commit and unregister from the commit order queue as exception in MYSQL_BIN_LOG::ordered_commit(), as these transactions have multiple commits and so not determined if the call is ending transaction.

Parameters

[in] thd The THD object of current thread.

Return values

true	Allow thread to wait for it turn
false	Do not allow thread to wait for it turn

◆ wait_on_graph()

bool Commit_order_manager::wait_on_graph ( Slave_worker * worker )

private

Determines if the worker passed as a parameter must wait on the MDL graph for other workers to commit and, if it must, will wait for it's turn to commit.

Parameters

worker The worker to determine the commit waiting status for.

Returns: false if the worker is ready to commit, true if not.

Member Data Documentation

◆ m_rollback_trx

std::atomic<bool> Commit_order_manager::m_rollback_trx

private

◆ m_workers

cs::apply::Commit_order_queue Commit_order_manager::m_workers

private

The documentation for this class was generated from the following files:

Public Member Functions

Static Public Member Functions

Private Member Functions

Private Attributes

Detailed Description

Constructor & Destructor Documentation

◆ Commit_order_manager() [1/2]

◆ Commit_order_manager() [2/2]

◆ ~Commit_order_manager()

Member Function Documentation

◆ check_and_report_deadlock()

◆ finish()

◆ finish_one() [1/2]

◆ finish_one() [2/2]

◆ flush_engine_and_signal_threads()

◆ get_rollback_status() [1/2]

◆ get_rollback_status() [2/2]

◆ init_worker_context()

◆ operator=()

◆ register_trx()

◆ report_deadlock()

◆ reset_server_status()

◆ set_rollback_status()

◆ unset_rollback_status()

◆ visit_lock_graph()

◆ wait() [1/2]

◆ wait() [2/2]

◆ wait_and_finish()

◆ wait_for_its_turn_before_flush_stage()

◆ wait_on_graph()

Member Data Documentation

◆ m_rollback_trx

◆ m_workers