Redo log buffer implementation, including functions to: More...

#include <atomic>
#include <cstring>
#include "log0buf.h"
#include "log0chkp.h"
#include "log0log.h"
#include "log0recv.h"
#include "log0sys.h"
#include "log0test.h"
#include "log0types.h"
#include "log0write.h"
#include "my_dbug.h"
#include "srv0srv.h"
#include "srv0start.h"
#include "ut0byte.h"

Functions
Reservation of space in the redo log
static void	log_wait_for_space_after_reserving (log_t &log, const Log_handle &handle)
	Waits until there is free space in log buffer up to reserved handle.end_sn. More...

static void	log_wait_for_space_in_log_buf (log_t &log, sn_t end_sn)
	Waits until there is free space in the log buffer. More...

void	log_update_buf_limit (log_t &log)
	Updates limit used when writing to log buffer. More...

void	log_update_buf_limit (log_t &log, lsn_t write_lsn)
	Updates limit used when writing to log buffer, according to provided write_lsn. More...

Log_handle	log_buffer_reserve (log_t &log, size_t len)
	Reserves space in the redo log for following write operations. More...

Locking for the redo log
static void	log_buffer_s_lock_wait (log_t &log, const sn_t start_sn)
	Waits for the start_sn unlocked and allowed to write to the buffer. More...

static sn_t	log_buffer_s_lock_enter_reserve (log_t &log, size_t len)
	Acquires an s-lock on the log buffer and reserves `len` bytes in it. More...

static void	log_buffer_s_lock_exit (log_t &log)
	Releases the log buffer s-lock. More...

lsn_t	log_buffer_wait_for_ready_for_write_lsn (log_t &log, lsn_t end_lsn, uint32_t *waits_on_event)
	Wait until log_buffer_ready_for_write_lsn() returns at least min_lsn. More...

void	log_buffer_x_lock_enter (log_t &log)
	Acquires the log buffer x-lock. More...

void	log_buffer_x_lock_exit (log_t &log)
	Releases the log buffer x-lock. More...

Writing to the redo log buffer
lsn_t	log_buffer_write (log_t &log, const byte *str, size_t str_len, lsn_t start_lsn)
	Writes data to the log buffer. More...

void	log_buffer_write_completed (log_t &log, lsn_t start_lsn, lsn_t end_lsn, bool is_last_block)
	Adds a link start_lsn -> end_lsn to the log recent written buffer. More...

void	log_buffer_set_first_record_group (log_t &log, lsn_t rec_group_end_lsn)
	Modifies header of log block in the log buffer, which contains a given lsn value, and sets offset to the first group of log records within the block. More...

void	log_buffer_flush_to_disk (log_t &log, bool sync)
	Write to the log file up to the last log entry. More...

void	log_buffer_sync_in_background ()
	Writes the log buffer to the log file. More...

void	log_buffer_get_last_block (log_t &log, lsn_t &last_lsn, byte *last_block, uint32_t &block_len)
	Get last redo block from redo buffer and end LSN. More...

Traversing links in the redo log recent buffers

void	log_advance_ready_for_write_lsn (log_t &log)
	Advances log_buffer_ready_for_write_lsn() using links in the recent written buffer. More...

Detailed Description

Redo log buffer implementation, including functions to:

Reserve space in the redo log buffer,
Write to the reserved space in the log buffer,
Add link to the log recent written buffer,

Function Documentation

◆ log_advance_ready_for_write_lsn()

void log_advance_ready_for_write_lsn ( log_t & log )

Advances log_buffer_ready_for_write_lsn() using links in the recent written buffer.

It's used by the log writer thread only.

Parameters

[in] log redo log

◆ log_buffer_flush_to_disk()

void log_buffer_flush_to_disk	(	log_t &	log,
		bool	sync = `true`
	)

Write to the log file up to the last log entry.

Parameters

[in,out]	log	redo log
[in]	sync	whether we want the written log also to be flushed to disk.

◆ log_buffer_get_last_block()

void log_buffer_get_last_block	(	log_t &	log,
		lsn_t &	last_lsn,
		byte *	last_block,
		uint32_t &	block_len
	)

Get last redo block from redo buffer and end LSN.

Note that it takes x-lock on the log buffer for a short period. Out values are always set, even when provided last_block is nullptr.

Parameters

[in,out]	log	redo log
[out]	last_lsn	end lsn of last mtr
[out]	last_block	last redo block
[in,out]	block_len	length in bytes

◆ log_buffer_reserve()

Log_handle log_buffer_reserve	(	log_t &	log,
		size_t	len
	)

Reserves space in the redo log for following write operations.

Space is reserved for a given number of data bytes. Additionally bytes for required headers and footers of log blocks are reserved.

After the space is reserved, range of lsn values from a start_lsn to an end_lsn is assigned. The log writer thread cannot proceed further than to the start_lsn, until a link start_lsn -> end_lsn has been added to the log recent written buffer.

NOTE that the link is added after data is written to the reserved space in the log buffer. It is very critical to do all these steps as fast as possible, because very likely the log writer thread is waiting for the link.

Parameters

[in,out]	log	redo log
[in]	len	number of data bytes to reserve for write

Returns: handle that represents the reservation

◆ log_buffer_s_lock_enter_reserve()

static sn_t log_buffer_s_lock_enter_reserve	(	log_t &	log,
		size_t	len
	)

inlinestatic

Acquires an s-lock on the log buffer and reserves len bytes in it.

The corresponding unlock operation is log_buffer_s_lock_exit() which is called during the log_buffer_write_completed(..) for the last fragment of data copied to the reserved buffer - to be specific it happens after the data is already copied to the log buffer, but right before the last missing link is added to log.recent_written. The current implementation pretends rw_lock_s_lock() is taken, while actually only the debug info and PSI info typical for rw_lock_t are maintained, but no actual rw_lock_t is being used for synchronization - that is achieved by using log_t::sn (to inform s-locking threads about x-locker) and log_t::recently_written (to inform x-locking thread when s-lockers have finished). The resize does not use a read/write lock (rw_lock_t) due to performance impact it would cause.

Parameters

[in,out]	log	redo log
[in]	len	number of data bytes to reserve for write

Returns: start sn of reserved

◆ log_buffer_s_lock_exit()

static void log_buffer_s_lock_exit ( log_t & log )

inlinestatic

Releases the log buffer s-lock.

Parameters

[in,out] log redo log

◆ log_buffer_s_lock_wait()

static void log_buffer_s_lock_wait	(	log_t &	log,
		const sn_t	start_sn
	)

inlinestatic

Waits for the start_sn unlocked and allowed to write to the buffer.

Parameters

[in,out]	log	redo log
[in]	start_sn	target sn value to start to write

◆ log_buffer_set_first_record_group()

void log_buffer_set_first_record_group	(	log_t &	log,
		lsn_t	rec_group_end_lsn
	)

Modifies header of log block in the log buffer, which contains a given lsn value, and sets offset to the first group of log records within the block.

This is used by mtr after writing a log record group which ends at lsn belonging to different log block than lsn at which the group was started. When write was finished at the last data byte of log block, it is considered ended in the next log block, because the next data byte belongs to that block.

During recovery, when recovery is started in the middle of some group of log records, it first looks for the beginning of the next group.

Parameters

[in,out]	log	redo log
[in]	rec_group_end_lsn	lsn at which the first log record group starts within the block containing this lsn value

◆ log_buffer_sync_in_background()

void log_buffer_sync_in_background ( )

Writes the log buffer to the log file.

It is intended to be called from background master thread periodically. If the log writer threads are active, this function writes nothing.

◆ log_buffer_wait_for_ready_for_write_lsn()

lsn_t log_buffer_wait_for_ready_for_write_lsn	(	log_t &	log,
		lsn_t	min_lsn,
		uint32_t *	waits_on_event = `nullptr`
	)

Wait until log_buffer_ready_for_write_lsn() returns at least min_lsn.

Parameters

[in]	log	redo log
[in]	min_lsn	The awaited minimum value of log_buffer_ready_for_write_lsn().
[out]	waits_on_event	If not null, then the pointed integer will be set to the number of times this function had to wait on closer_event.

Returns: The seen value of log_buffer_ready_for_write_lsn() which was at least equal to min_lsn.

◆ log_buffer_write()

lsn_t log_buffer_write	(	log_t &	log,
		const byte *	str,
		size_t	str_len,
		lsn_t	start_lsn
	)

Writes data to the log buffer.

The space in the redo log has to be reserved before calling to this function and lsn pointing to inside the reserved range of lsn values has to be provided.

The write does not have to cover the whole reserved space, but may not overflow it. If it does not cover, then returned value should be used to start the next write operation. Note that finally we must use exactly all the reserved space.

Parameters

[in,out]	log	redo log
[in]	str	memory to write data from
[in]	str_len	number of bytes to write
[in]	start_lsn	lsn to start writing at (the reserved space)

Returns: end_lsn after writing the data (in the reserved space), could be used to start the next write operation if there is still free space in the reserved space

◆ log_buffer_write_completed()

void log_buffer_write_completed	(	log_t &	log,
		lsn_t	start_lsn,
		lsn_t	end_lsn,
		bool	is_last_block
	)

Adds a link start_lsn -> end_lsn to the log recent written buffer.

This function must be called after the data has been written to the fragment of log buffer represented by range [start_lsn, end_lsn). After the link is added, the log writer may write the data to disk.

NOTE that still dirty pages for the [start_lsn, end_lsn) are not added to flush lists when this function is called.

Parameters

[in,out]	log	redo log
[in]	start_lsn	start_lsn of the link to add
[in]	end_lsn	end_lsn of the link to add
[in]	is_last_block	A flag to let log writer know that this [start_lsn,end_lsn) range was the last fragment of the range reserved by this thread earlier through log_buffer_reserve(..) , i.e. this thread has finished writing to the log buffer.

◆ log_buffer_x_lock_enter()

void log_buffer_x_lock_enter ( log_t & log )

Acquires the log buffer x-lock.

Parameters

[in,out] log redo log

◆ log_buffer_x_lock_exit()

void log_buffer_x_lock_exit ( log_t & log )

Releases the log buffer x-lock.

Parameters

[in,out] log redo log

◆ log_update_buf_limit() [1/2]

void log_update_buf_limit ( log_t & log )

Updates limit used when writing to log buffer.

Note that the log buffer may have space for log records for which we still do not have space in log files (for larger lsn values).

Parameters

[in,out] log redo log

◆ log_update_buf_limit() [2/2]

void log_update_buf_limit	(	log_t &	log,
		lsn_t	write_lsn
	)

Updates limit used when writing to log buffer, according to provided write_lsn.

It must be <= log.write_lsn.load() to protect from log buffer overwrites.

Parameters

[in,out]	log	redo log
[in]	write_lsn	value <= log.write_lsn.load()

◆ log_wait_for_space_after_reserving()

static void log_wait_for_space_after_reserving	(	log_t &	log,
		const Log_handle &	handle
	)

static

Waits until there is free space in log buffer up to reserved handle.end_sn.

If there was no space, it basically waits for log writer thread which copies data from log buffer to log files and advances log.write_lsn, reclaiming space in the log buffer (it's a ring buffer).

There is a special case - if it turned out, that log buffer is too small for the reserved range of lsn values, it resizes the log buffer.

It's used during reservation of lsn values, when the reserved handle.end_sn is greater than log.buf_limit_sn.

Parameters

[in,out]	log	redo log
[in]	handle	handle for the reservation

◆ log_wait_for_space_in_log_buf()

static void log_wait_for_space_in_log_buf	(	log_t &	log,
		sn_t	end_sn
	)

static

Waits until there is free space in the log buffer.

The free space has to be available for range of sn values ending at the provided sn.

Parameters

[in]	log	redo log
[in]	end_sn	end of the range of sn values

Functions

Detailed Description

Function Documentation

◆ log_advance_ready_for_write_lsn()

◆ log_buffer_flush_to_disk()

◆ log_buffer_get_last_block()

◆ log_buffer_reserve()

◆ log_buffer_s_lock_enter_reserve()

◆ log_buffer_s_lock_exit()

◆ log_buffer_s_lock_wait()

◆ log_buffer_set_first_record_group()

◆ log_buffer_sync_in_background()

◆ log_buffer_wait_for_ready_for_write_lsn()

◆ log_buffer_write()

◆ log_buffer_write_completed()

◆ log_buffer_x_lock_enter()

◆ log_buffer_x_lock_exit()

◆ log_update_buf_limit() [1/2]

◆ log_update_buf_limit() [2/2]

◆ log_wait_for_space_after_reserving()

◆ log_wait_for_space_in_log_buf()