MySQL 8.0.32
Source Code Documentation
log0chkp.cc File Reference

Redo log checkpointing. More...

#include <chrono>
#include <cstring>
#include "arch0arch.h"
#include "buf0buf.h"
#include "buf0flu.h"
#include "dict0dict.h"
#include "log0buf.h"
#include "log0chkp.h"
#include "log0encryption.h"
#include "log0files_io.h"
#include "log0log.h"
#include "log0recv.h"
#include "log0sys.h"
#include "log0test.h"
#include "log0types.h"
#include "log0write.h"
#include "mach0data.h"
#include "my_dbug.h"
#include "os0event.h"
#include "srv0mon.h"
#include "srv0srv.h"
#include "srv0start.h"
#include "ut0byte.h"

Functions

Log - coordination with buffer pool and oldest_lsn
static void log_update_available_for_checkpoint_lsn (log_t &log)
 Updates lsn available for checkpoint. More...
 
static lsn_t log_compute_available_for_checkpoint_lsn (const log_t &log)
 Calculates lsn at which we might write a next checkpoint. More...
 
Log - making checkpoints
static bool log_should_checkpoint (log_t &log)
 Checks if checkpoint should be written. More...
 
static void log_consider_checkpoint (log_t &log)
 Considers writing next checkpoint. More...
 
static void log_consider_sync_flush (log_t &log)
 Considers requesting page cleaners to execute sync flush. More...
 
static void log_checkpoint (log_t &log)
 Makes a checkpoint. More...
 
static std::chrono::steady_clock::duration log_checkpoint_time_elapsed (const log_t &log)
 Calculates time that elapsed since last checkpoint. More...
 
static void log_request_checkpoint_low (log_t &log, lsn_t requested_lsn)
 Requests a checkpoint written for lsn greater or equal to provided one. More...
 
static void log_request_checkpoint_in_next_file_low (log_t &log)
 Requests a checkpoint written in the next log file (not in the one, to which current log.last_checkpoint_lsn belongs to). More...
 
static void log_wait_for_checkpoint (const log_t &log, lsn_t lsn)
 Waits for checkpoint advanced to at least that lsn. More...
 
static bool log_request_sync_flush (const log_t &log, lsn_t new_oldest)
 Requests for urgent flush of dirty pages, to advance oldest_lsn in flush lists to provided value. More...
 
void log_set_dict_max_allowed_checkpoint_lsn (log_t &log, lsn_t max_lsn)
 Updates the field log.dict_max_allowed_checkpoint_lsn. More...
 
static lsn_t log_determine_checkpoint_lsn (log_t &log)
 
dberr_t log_files_next_checkpoint (log_t &log, lsn_t next_checkpoint_lsn)
 Writes the next checkpoint to the log file, by writing a single checkpoint header with the checkpoint lsn. More...
 
Log_checkpoint_header_no log_next_checkpoint_header (Log_checkpoint_header_no checkpoint_header_no)
 Provides opposite checkpoint header number to the given checkpoint header number. More...
 
dberr_t log_files_write_checkpoint_low (log_t &log, Log_file_handle &checkpoint_file_handle, Log_checkpoint_header_no checkpoint_header_no, lsn_t checkpoint_lsn)
 Writes checkpoint to the file containing the written checkpoint_lsn. More...
 
dberr_t log_files_write_first_data_block_low (log_t &log, Log_file_handle &file_handle, lsn_t checkpoint_lsn, lsn_t file_start_lsn)
 Writes the first data block to the log file using the provided handle to the opened log file. More...
 
static bool log_request_checkpoint_validate (const log_t &log)
 
void log_request_checkpoint (log_t &log, bool sync)
 Requests a fuzzy checkpoint write (for currently available lsn). More...
 
void log_request_checkpoint_in_next_file (log_t &log)
 Requests a checkpoint written in the next log file (not in the one, to which current log.last_checkpoint_lsn belongs to). More...
 
bool log_request_latest_checkpoint (log_t &log, lsn_t &requested_lsn)
 Requests a checkpoint at the current lsn. More...
 
bool log_make_latest_checkpoint (log_t &log)
 Make a checkpoint at the current lsn. More...
 
bool log_make_latest_checkpoint ()
 Make a checkpoint at the current lsn. More...
 
lsn_t log_sync_flush_lsn (log_t &log)
 Computes lsn up to which sync flush should be done or returns 0 if there is no need to execute sync flush now. More...
 
void log_checkpointer (log_t *log_ptr)
 The log checkpointer thread routine. More...
 
lsn_t log_get_checkpoint_age (const log_t &log)
 Calculates age of current checkpoint as number of bytes since last checkpoint. More...
 
Log - free check
sn_t log_concurrency_margin (lsn_t log_capacity, bool &is_safe)
 Computes concurrency margin to be used within log_free_check calls, for a given redo log capacity (soft_logical_capacity). More...
 
void log_update_concurrency_margin (log_t &log)
 Updates log.concurrency_margin and log.concurrency_margin_is_safe for the current capacity of the redo log and current innodb_thread_concurrency value. More...
 
void log_update_limits_low (log_t &log)
 Updates log.free_check_limit_sn in the log. More...
 
void log_set_dict_persist_margin (log_t &log, sn_t margin)
 Updates log.dict_persist_margin and recompute free check limit. More...
 
lsn_t log_free_check_margin (const log_t &log)
 Provides current margin used in the log_free_check calls. More...
 
lsn_t log_free_check_capacity (const log_t &log, lsn_t free_check_margin)
 Computes capacity of redo log available until log_free_check() needs to wait. More...
 
lsn_t log_free_check_capacity (const log_t &log)
 Computes capacity of redo log available until log_free_check() needs to wait. More...
 
void log_free_check_wait (log_t &log)
 Waits until there is free space in log files which includes concurrency margin required for all threads. More...
 
void log_free_check_validate ()
 Performs debug checks to validate some of the assumptions. More...
 

Detailed Description

Redo log checkpointing.

File consists of four groups:

  1. Coordination between log and buffer pool (oldest_lsn).
  2. Making checkpoints (including the log_checkpointer thread).
  3. Free check.

Function Documentation

◆ log_checkpoint()

static void log_checkpoint ( log_t log)
static

Makes a checkpoint.

Note that this function does not flush dirty blocks from the buffer pool. It only checks what is lsn of the oldest modification in the buffer pool, and writes information about the lsn in log files.

Parameters
[in,out]logredo log

◆ log_checkpoint_time_elapsed()

static std::chrono::steady_clock::duration log_checkpoint_time_elapsed ( const log_t log)
static

Calculates time that elapsed since last checkpoint.

Returns
Time duration elapsed since the last checkpoint

◆ log_checkpointer()

void log_checkpointer ( log_t log_ptr)

The log checkpointer thread routine.

Parameters
[in,out]log_ptrpointer to redo log

◆ log_compute_available_for_checkpoint_lsn()

static lsn_t log_compute_available_for_checkpoint_lsn ( const log_t log)
static

Calculates lsn at which we might write a next checkpoint.

It does the best effort, but possibly the maximum allowed lsn, could be even bigger. That's because the order of dirty pages in flush lists has been relaxed, and we don't want to spend time on traversing the whole flush lists here.

Note that some flush lists could be empty, and some additions of dirty pages could be pending (threads have written data to the log buffer and became scheduled out just before adding the dirty pages). That's why the calculated value cannot be larger than the log.buf_dirty_pages_added_up_to_lsn (only up to this lsn value we are sure, that all the dirty pages have been added).

It is guaranteed, that the returned value will not be smaller than the log.last_checkpoint_lsn.

Returns
lsn for which we might write the checkpoint

◆ log_concurrency_margin()

sn_t log_concurrency_margin ( lsn_t  log_capacity,
bool &  is_safe 
)

Computes concurrency margin to be used within log_free_check calls, for a given redo log capacity (soft_logical_capacity).

Parameters
[in]log_capacityredo log capacity (soft)
[out]is_safetrue iff the computed margin wasn't truncated because of too small log_capacity
Returns
the computed margin

◆ log_consider_checkpoint()

static void log_consider_checkpoint ( log_t log)
static

Considers writing next checkpoint.

Checks if checkpoint should be written (using log_should_checkpoint()) and writes the checkpoint if that's the case.

◆ log_consider_sync_flush()

static void log_consider_sync_flush ( log_t log)
static

Considers requesting page cleaners to execute sync flush.

◆ log_determine_checkpoint_lsn()

static lsn_t log_determine_checkpoint_lsn ( log_t log)
static

◆ log_files_next_checkpoint()

dberr_t log_files_next_checkpoint ( log_t log,
lsn_t  lsn 
)

Writes the next checkpoint to the log file, by writing a single checkpoint header with the checkpoint lsn.

Flushes the file after the write and updates the log.last_checkpoint_lsn.

Remarks
Note that two checkpoint headers are used alternately for consecutive checkpoints. If InnoDB crashed during the write, it would still have the previous checkpoint info and recovery would work.
Parameters
[in,out]logredo log
[in]lsnwrites checkpoint at this lsn
Returns
DB_SUCCESS or error

◆ log_files_write_checkpoint_low()

dberr_t log_files_write_checkpoint_low ( log_t log,
Log_file_handle checkpoint_file_handle,
Log_checkpoint_header_no  checkpoint_header_no,
lsn_t  next_checkpoint_lsn 
)

Writes checkpoint to the file containing the written checkpoint_lsn.

The checkpoint is written to the given checkpoint header. Unless InnoDB is starting: checkpointer, writer and files mutexes must be acquired before calling this function.

Parameters
[in,out]logredo log
[in]checkpoint_file_handlehandle to opened file
[in]checkpoint_header_nocheckpoint header to be written
[in]next_checkpoint_lsnthe checkpoint lsn to write
Returns
DB_SUCCESS or error

◆ log_files_write_first_data_block_low()

dberr_t log_files_write_first_data_block_low ( log_t log,
Log_file_handle file_handle,
lsn_t  checkpoint_lsn,
lsn_t  file_start_lsn 
)

Writes the first data block to the log file using the provided handle to the opened log file.

The block is addressed by the given checkpoint_lsn, filled with 0x00 and its data length points to checkpoint_lsn inside, making the block logically empty.

Remarks
This is used only during creation of new log files.
Parameters
[in,out]logredo log
[in]file_handlehandle to the opened log file
[in]checkpoint_lsnthe checkpoint lsn
[in]file_start_lsnstart_lsn of the file
Returns
DB_SUCCESS or error

◆ log_free_check_capacity() [1/2]

lsn_t log_free_check_capacity ( const log_t log)

Computes capacity of redo log available until log_free_check() needs to wait.

It calls log_free_check_margin(log) to obtain the current log_free_check_margin.

Parameters
[in]logredo log
Returns
lsn capacity up to free_check_wait happens

◆ log_free_check_capacity() [2/2]

lsn_t log_free_check_capacity ( const log_t log,
lsn_t  free_check_margin 
)

Computes capacity of redo log available until log_free_check() needs to wait.

It uses a provided size of the log_free_check_margin.

Parameters
[in]logredo log
[in]free_check_marginsize of the log_free_check_margin;
See also
log_free_check_margin(log)
Returns
lsn capacity up to free_check_wait happens

◆ log_free_check_margin()

lsn_t log_free_check_margin ( const log_t log)

Provides current margin used in the log_free_check calls.

It is a sum of dict_persist_margin and concurrency_margin.

Parameters
[in]logredo log
Returns
margin that would be used in log_free_check()

◆ log_free_check_validate()

void log_free_check_validate ( )

Performs debug checks to validate some of the assumptions.

◆ log_free_check_wait()

void log_free_check_wait ( log_t log)

Waits until there is free space in log files which includes concurrency margin required for all threads.

You should rather use log_free_check().

Parameters
[in]logredo log

◆ log_get_checkpoint_age()

lsn_t log_get_checkpoint_age ( const log_t log)

Calculates age of current checkpoint as number of bytes since last checkpoint.

This includes bytes for headers and footers of all log blocks. The calculation is based on the latest written checkpoint lsn, and the current lsn, which points to the first non reserved data byte. Note that the current lsn could not fit the free space in the log files. This means that the checkpoint age could potentially be larger than capacity of the log files. However we do the best effort to avoid such situations, and if they happen, user threads wait until the space is reclaimed.

Parameters
[in]logredo log
Returns
checkpoint age as number of bytes

◆ log_make_latest_checkpoint() [1/2]

bool log_make_latest_checkpoint ( )

Make a checkpoint at the current lsn.

Reads current lsn and waits until all dirty pages have been flushed up to that lsn. Afterwards requests a checkpoint write and waits until it is finished.

Returns
true iff current lsn was greater than last checkpoint lsn

◆ log_make_latest_checkpoint() [2/2]

bool log_make_latest_checkpoint ( log_t log)

Make a checkpoint at the current lsn.

Reads current lsn and waits until all dirty pages have been flushed up to that lsn. Afterwards requests a checkpoint write and waits until it is finished.

Parameters
[in,out]logredo log
Returns
true iff current lsn was greater than last checkpoint lsn

◆ log_next_checkpoint_header()

Log_checkpoint_header_no log_next_checkpoint_header ( Log_checkpoint_header_no  checkpoint_header_no)

Provides opposite checkpoint header number to the given checkpoint header number.

Parameters
[in]checkpoint_header_nothe given checkpoint header number
Returns
the opposite checkpoint header number

◆ log_request_checkpoint()

void log_request_checkpoint ( log_t log,
bool  sync 
)

Requests a fuzzy checkpoint write (for currently available lsn).

Parameters
[in,out]logredo log
[in]syncwhether request is sync (function should wait)

◆ log_request_checkpoint_in_next_file()

void log_request_checkpoint_in_next_file ( log_t log)

Requests a checkpoint written in the next log file (not in the one, to which current log.last_checkpoint_lsn belongs to).

Parameters
[in,out]logredo log

◆ log_request_checkpoint_in_next_file_low()

void log_request_checkpoint_in_next_file_low ( log_t log)
static

Requests a checkpoint written in the next log file (not in the one, to which current log.last_checkpoint_lsn belongs to).

Prior to calling this function, caller must acquire the log.limits_mutex !

Parameters
[in,out]logredo log

◆ log_request_checkpoint_low()

static void log_request_checkpoint_low ( log_t log,
lsn_t  requested_lsn 
)
static

Requests a checkpoint written for lsn greater or equal to provided one.

The log.checkpointer_mutex has to be acquired before it is called, and it is not released within this function.

Parameters
[in,out]logredo log
[in]requested_lsnprovided lsn (checkpoint should be not older)

◆ log_request_checkpoint_validate()

static bool log_request_checkpoint_validate ( const log_t log)
static

◆ log_request_latest_checkpoint()

bool log_request_latest_checkpoint ( log_t log,
lsn_t requested_lsn 
)

Requests a checkpoint at the current lsn.

Parameters
[in,out]logredo log
[out]requested_lsnlsn for which checkpoint was requested, or stays unmodified if it wasn't requested
Returns
true iff requested (false if checkpoint_lsn was already at that lsn)

◆ log_request_sync_flush()

static bool log_request_sync_flush ( const log_t log,
lsn_t  new_oldest 
)
static

Requests for urgent flush of dirty pages, to advance oldest_lsn in flush lists to provided value.

This should force page cleaners to perform the sync-flush in which case the innodb_max_io_capacity is not respected. This should be called when we are close to running out of space in redo log (close to free_check_limit_sn).

Parameters
[in]logredo log
[in]new_oldestoldest_lsn to stop flush at (or greater)
Return values
truerequested page flushing
falsedid not request page flushing (either because it is unit test for redo log or sync flushing is disabled by sys_var: innodb_flush_sync)

◆ log_set_dict_max_allowed_checkpoint_lsn()

void log_set_dict_max_allowed_checkpoint_lsn ( log_t log,
lsn_t  max_lsn 
)

Updates the field log.dict_max_allowed_checkpoint_lsn.

This is limitation for lsn at which checkpoint might be written, imposed by cached changes to the DD table buffer. It is called from DD code.

Parameters
[in,out]logredo log
[in]max_lsnnew value for the limitation

◆ log_set_dict_persist_margin()

void log_set_dict_persist_margin ( log_t log,
sn_t  margin 
)

Updates log.dict_persist_margin and recompute free check limit.

Parameters
[in,out]logredo log
[in]marginnew value for log.dict_persist_margin

◆ log_should_checkpoint()

static bool log_should_checkpoint ( log_t log)
static

Checks if checkpoint should be written.

Checks time elapsed since the last checkpoint, age of the last checkpoint and if there was any extra request to write the checkpoint (e.g. coming from log_make_latest_checkpoint()).

Returns
true if checkpoint should be written

◆ log_sync_flush_lsn()

lsn_t log_sync_flush_lsn ( log_t log)

Computes lsn up to which sync flush should be done or returns 0 if there is no need to execute sync flush now.

Parameters
[in,out]logredo log
Returns
lsn for which we want to have oldest_lsn >= lsn in each BP, or 0 if there is no need for sync flush

◆ log_update_available_for_checkpoint_lsn()

static void log_update_available_for_checkpoint_lsn ( log_t log)
static

Updates lsn available for checkpoint.

Parameters
[in,out]logredo log

◆ log_update_concurrency_margin()

void log_update_concurrency_margin ( log_t log)

Updates log.concurrency_margin and log.concurrency_margin_is_safe for the current capacity of the redo log and current innodb_thread_concurrency value.

Parameters
[in,out]logredo log

◆ log_update_limits_low()

void log_update_limits_low ( log_t log)

Updates log.free_check_limit_sn in the log.

The log_limits_mutex must be acquired before a call (unless srv_is_being_started is true).

Parameters
[in,out]logredo log

◆ log_wait_for_checkpoint()

static void log_wait_for_checkpoint ( const log_t log,
lsn_t  lsn 
)
static

Waits for checkpoint advanced to at least that lsn.

Parameters
[in]logredo log
[in]lsnlsn up to which we are waiting