Redo log buffer implementation, including functions to: More...

#include <atomic>
#include <cstring>
#include "log0buf.h"
#include "log0chkp.h"
#include "log0log.h"
#include "log0recv.h"
#include "log0sys.h"
#include "log0test.h"
#include "log0types.h"
#include "log0write.h"
#include "my_dbug.h"
#include "srv0srv.h"
#include "srv0start.h"
#include "ut0byte.h"

Functions
Reservation of space in the redo log
static void	log_wait_for_space_after_reserving (log_t &log, const Log_handle &handle)
	Waits until there is free space in log buffer up to reserved handle.end_sn. More...

static void	log_wait_for_space_in_log_buf (log_t &log, sn_t end_sn)
	Waits until there is free space in the log buffer. More...

void	log_update_buf_limit (log_t &log)
	Updates limit used when writing to log buffer. More...

void	log_update_buf_limit (log_t &log, lsn_t write_lsn)
	Updates limit used when writing to log buffer, according to provided write_lsn. More...

Log_handle	log_buffer_reserve (log_t &log, size_t len)
	Reserves space in the redo log for following write operations. More...

Locking for the redo log
static void	log_buffer_s_lock_wait (log_t &log, const sn_t start_sn)
	Waits for the start_sn unlocked and allowed to write to the buffer. More...

static sn_t	log_buffer_s_lock_enter_reserve (log_t &log, size_t len)
	Acquires the log buffer s-lock. More...

static void	log_buffer_s_lock_exit_close (log_t &log, lsn_t start_lsn, lsn_t end_lsn)
	Releases the log buffer s-lock. More...

void	log_buffer_x_lock_enter (log_t &log)
	Acquires the log buffer x-lock. More...

void	log_buffer_x_lock_exit (log_t &log)
	Releases the log buffer x-lock. More...

Writing to the redo log buffer
lsn_t	log_buffer_write (log_t &log, const byte *str, size_t str_len, lsn_t start_lsn)
	Writes data to the log buffer. More...

void	log_buffer_write_completed (log_t &log, lsn_t start_lsn, lsn_t end_lsn)
	Adds a link start_lsn -> end_lsn to the log recent written buffer. More...

void	log_wait_for_space_in_log_recent_closed (log_t &log, lsn_t lsn)
	Waits until there is free space in the log recent closed buffer for any links start_lsn -> end_lsn, which start at provided start_lsn. More...

void	log_buffer_close (log_t &log, const Log_handle &handle)
	Adds a link start_lsn -> end_lsn to the log recent closed buffer. More...

void	log_buffer_set_first_record_group (log_t &log, lsn_t rec_group_end_lsn)
	Modifies header of log block in the log buffer, which contains a given lsn value, and sets offset to the first group of log records within the block. More...

void	log_buffer_flush_to_disk (log_t &log, bool sync)
	Write to the log file up to the last log entry. More...

void	log_buffer_flush_to_disk (bool sync)
	Requests flush of the log buffer. More...

void	log_buffer_sync_in_background ()
	Writes the log buffer to the log file. More...

void	log_buffer_get_last_block (log_t &log, lsn_t &last_lsn, byte *last_block, uint32_t &block_len)
	Get last redo block from redo buffer and end LSN. More...

Traversing links in the redo log recent buffers

void	log_advance_ready_for_write_lsn (log_t &log)
	Advances log.buf_ready_for_write_lsn using links in the recent written buffer. More...

Detailed Description

Redo log buffer implementation, including functions to:

Reserve space in the redo log buffer,
Write to the reserved space in the log buffer,
Add link to the log recent written buffer,
Add link to the log recent closed buffer.

Function Documentation

◆ log_advance_ready_for_write_lsn()

void log_advance_ready_for_write_lsn ( log_t & log )

Advances log.buf_ready_for_write_lsn using links in the recent written buffer.

It's used by the log writer thread only.

Parameters

[in] log redo log

◆ log_buffer_close()

void log_buffer_close	(	log_t &	log,
		const Log_handle &	handle
	)

Adds a link start_lsn -> end_lsn to the log recent closed buffer.

This is called after all dirty pages related to [start_lsn, end_lsn) have been added to corresponding flush lists. For detailed explanation -

See also: log0write.cc.

Parameters

[in,out]	log	redo log
[in]	handle	handle for the reservation of space

◆ log_buffer_flush_to_disk() [1/2]

void log_buffer_flush_to_disk ( bool sync = true )

Requests flush of the log buffer.

Parameters

[in] sync true: wait until the flush is done

◆ log_buffer_flush_to_disk() [2/2]

void log_buffer_flush_to_disk	(	log_t &	log,
		bool	sync = `true`
	)

Write to the log file up to the last log entry.

Parameters

[in,out]	log	redo log
[in]	sync	whether we want the written log also to be flushed to disk.

◆ log_buffer_get_last_block()

void log_buffer_get_last_block	(	log_t &	log,
		lsn_t &	last_lsn,
		byte *	last_block,
		uint32_t &	block_len
	)

Get last redo block from redo buffer and end LSN.

Note that it takes x-lock on the log buffer for a short period. Out values are always set, even when provided last_block is nullptr.

Parameters

[in,out]	log	redo log
[out]	last_lsn	end lsn of last mtr
[out]	last_block	last redo block
[in,out]	block_len	length in bytes

◆ log_buffer_reserve()

Log_handle log_buffer_reserve	(	log_t &	log,
		size_t	len
	)

Reserves space in the redo log for following write operations.

Space is reserved for a given number of data bytes. Additionally bytes for required headers and footers of log blocks are reserved.

After the space is reserved, range of lsn values from a start_lsn to an end_lsn is assigned. The log writer thread cannot proceed further than to the start_lsn, until a link start_lsn -> end_lsn has been added to the log recent written buffer.

NOTE that the link is added after data is written to the reserved space in the log buffer. It is very critical to do all these steps as fast as possible, because very likely the log writer thread is waiting for the link.

Parameters

[in,out]	log	redo log
[in]	len	number of data bytes to reserve for write

Returns: handle that represents the reservation

◆ log_buffer_s_lock_enter_reserve()

static sn_t log_buffer_s_lock_enter_reserve	(	log_t &	log,
		size_t	len
	)

inlinestatic

Acquires the log buffer s-lock.

And reserve space in the log buffer. The corresponding unlock operation is adding link to log.recent_closed.

Parameters

[in,out]	log	redo log
[in]	len	number of data bytes to reserve for write

Returns: start sn of reserved

◆ log_buffer_s_lock_exit_close()

static void log_buffer_s_lock_exit_close	(	log_t &	log,
		lsn_t	start_lsn,
		lsn_t	end_lsn
	)

inlinestatic

Releases the log buffer s-lock.

Parameters

[in,out]	log	redo log
[in]	start_lsn	start lsn of the reservation
[in]	end_lsn	end lsn of the reservation

◆ log_buffer_s_lock_wait()

static void log_buffer_s_lock_wait	(	log_t &	log,
		const sn_t	start_sn
	)

inlinestatic

Waits for the start_sn unlocked and allowed to write to the buffer.

Parameters

[in,out]	log	redo log
[in]	start_sn	target sn value to start to write

◆ log_buffer_set_first_record_group()

void log_buffer_set_first_record_group	(	log_t &	log,
		lsn_t	rec_group_end_lsn
	)

Modifies header of log block in the log buffer, which contains a given lsn value, and sets offset to the first group of log records within the block.

This is used by mtr after writing a log record group which ends at lsn belonging to different log block than lsn at which the group was started. When write was finished at the last data byte of log block, it is considered ended in the next log block, because the next data byte belongs to that block.

During recovery, when recovery is started in the middle of some group of log records, it first looks for the beginning of the next group.

Parameters

[in,out]	log	redo log
[in]	rec_group_end_lsn	lsn at which the first log record group starts within the block containing this lsn value

◆ log_buffer_sync_in_background()

void log_buffer_sync_in_background ( )

Writes the log buffer to the log file.

It is intended to be called from background master thread periodically. If the log writer threads are active, this function writes nothing.

◆ log_buffer_write()

lsn_t log_buffer_write	(	log_t &	log,
		const byte *	str,
		size_t	str_len,
		lsn_t	start_lsn
	)

Writes data to the log buffer.

The space in the redo log has to be reserved before calling to this function and lsn pointing to inside the reserved range of lsn values has to be provided.

The write does not have to cover the whole reserved space, but may not overflow it. If it does not cover, then returned value should be used to start the next write operation. Note that finally we must use exactly all the reserved space.

Parameters

[in,out]	log	redo log
[in]	str	memory to write data from
[in]	str_len	number of bytes to write
[in]	start_lsn	lsn to start writing at (the reserved space)

Returns: end_lsn after writing the data (in the reserved space), could be used to start the next write operation if there is still free space in the reserved space

◆ log_buffer_write_completed()

void log_buffer_write_completed	(	log_t &	log,
		lsn_t	start_lsn,
		lsn_t	end_lsn
	)

Adds a link start_lsn -> end_lsn to the log recent written buffer.

This function must be called after the data has been written to the fragment of log buffer represented by range [start_lsn, end_lsn). After the link is added, the log writer may write the data to disk.

NOTE that still dirty pages for the [start_lsn, end_lsn) are not added to flush lists when this function is called.

Parameters

[in,out]	log	redo log
[in]	start_lsn	start_lsn of the link to add
[in]	end_lsn	end_lsn of the link to add

◆ log_buffer_x_lock_enter()

void log_buffer_x_lock_enter ( log_t & log )

Acquires the log buffer x-lock.

Parameters

[in,out] log redo log

◆ log_buffer_x_lock_exit()

void log_buffer_x_lock_exit ( log_t & log )

Releases the log buffer x-lock.

Parameters

[in,out] log redo log

◆ log_update_buf_limit() [1/2]

void log_update_buf_limit ( log_t & log )

Updates limit used when writing to log buffer.

Note that the log buffer may have space for log records for which we still do not have space in log files (for larger lsn values).

Parameters

[in,out] log redo log

◆ log_update_buf_limit() [2/2]

void log_update_buf_limit	(	log_t &	log,
		lsn_t	write_lsn
	)

Updates limit used when writing to log buffer, according to provided write_lsn.

It must be <= log.write_lsn.load() to protect from log buffer overwrites.

Parameters

[in,out]	log	redo log
[in]	write_lsn	value <= log.write_lsn.load()

◆ log_wait_for_space_after_reserving()

static void log_wait_for_space_after_reserving	(	log_t &	log,
		const Log_handle &	handle
	)

static

Waits until there is free space in log buffer up to reserved handle.end_sn.

If there was no space, it basically waits for log writer thread which copies data from log buffer to log files and advances log.write_lsn, reclaiming space in the log buffer (it's a ring buffer).

There is a special case - if it turned out, that log buffer is too small for the reserved range of lsn values, it resizes the log buffer.

It's used during reservation of lsn values, when the reserved handle.end_sn is greater than log.buf_limit_sn.

Parameters

[in,out]	log	redo log
[in]	handle	handle for the reservation

◆ log_wait_for_space_in_log_buf()

static void log_wait_for_space_in_log_buf	(	log_t &	log,
		sn_t	end_sn
	)

static

Waits until there is free space in the log buffer.

The free space has to be available for range of sn values ending at the provided sn.

Parameters

[in]	log	redo log
[in]	end_sn	end of the range of sn values

◆ log_wait_for_space_in_log_recent_closed()

void log_wait_for_space_in_log_recent_closed	(	log_t &	log,
		lsn_t	lsn
	)

Waits until there is free space in the log recent closed buffer for any links start_lsn -> end_lsn, which start at provided start_lsn.

It does not add any link.

This is called just before dirty pages for [start_lsn, end_lsn) are added to flush lists. That's because we need to guarantee, that the delay until dirty page is added to flush list is limited. For detailed explanation -

See also: log0write.cc.

Parameters

[in,out]	log	redo log
[in]	lsn	lsn on which we wait (for any link: lsn -> x)

Functions

Detailed Description

Function Documentation

◆ log_advance_ready_for_write_lsn()

◆ log_buffer_close()

◆ log_buffer_flush_to_disk() [1/2]

◆ log_buffer_flush_to_disk() [2/2]

◆ log_buffer_get_last_block()

◆ log_buffer_reserve()

◆ log_buffer_s_lock_enter_reserve()

◆ log_buffer_s_lock_exit_close()

◆ log_buffer_s_lock_wait()

◆ log_buffer_set_first_record_group()

◆ log_buffer_sync_in_background()

◆ log_buffer_write()

◆ log_buffer_write_completed()

◆ log_buffer_x_lock_enter()

◆ log_buffer_x_lock_exit()

◆ log_update_buf_limit() [1/2]

◆ log_update_buf_limit() [2/2]

◆ log_wait_for_space_after_reserving()

◆ log_wait_for_space_in_log_buf()

◆ log_wait_for_space_in_log_recent_closed()