MySQL 8.3.0
Source Code Documentation
log0log.cc File Reference

Redo log system - provides durability for unflushed modifications to contents of data pages. More...

#include <time.h>
#include <cstdio>
#include <cstring>
#include "arch0arch.h"
#include "buf0dblwr.h"
#include "fil0fil.h"
#include "log0buf.h"
#include "log0chkp.h"
#include "log0files_capacity.h"
#include "log0files_governor.h"
#include "log0log.h"
#include "log0meb.h"
#include "log0pre_8_0_30.h"
#include "log0recv.h"
#include "log0sys.h"
#include "log0write.h"
#include "mtr0mtr.h"
#include "os0event.h"
#include "os0thread-create.h"
#include "os0thread.h"
#include "srv0srv.h"
#include "srv0start.h"
#include "sync0sync.h"
#include "sync0types.h"
#include "ut0byte.h"
#include "ut0link_buf.h"
#include "ut0new.h"

Functions

Allocation and deallocation of log_sys
static void log_sys_create ()
 Allocates the log system and initializes all log mutexes and log events. More...
 
static void log_sys_free ()
 Free the log system data structures. More...
 
static void log_fix_first_rec_group (lsn_t block_lsn, Log_data_block_header &block_header, lsn_t start_lsn)
 
dberr_t log_start (log_t &log, lsn_t checkpoint_lsn, lsn_t start_lsn, bool allow_checkpoints)
 Starts the initialized redo log system using a provided checkpoint_lsn and current lsn. More...
 
Log - log system initialization.
static dberr_t log_sys_handle_creator (log_t &log)
 Handles the log creator name stored on the disk (in the redo log files). More...
 
static dberr_t log_sys_check_format (const log_t &log)
 Recognizes log format and emits corresponding message to the error log. More...
 
static dberr_t log_sys_check_directory (const Log_files_context &ctx, std::string &path, bool &found_files)
 Checks if the redo log directory exists, can be listed and contains at least one redo log file. More...
 
dberr_t log_sys_init (bool expect_no_files, lsn_t flushed_lsn, lsn_t &new_files_lsn)
 Initializes log_sys and finds existing redo log files, or creates a new set of redo log files. More...
 
void log_sys_close ()
 Close the log system and free all the related memory. More...
 
Allocation / deallocation of buffers
static void log_allocate_buffer (log_t &log)
 Calculates proper size for the log buffer and allocates the log buffer. More...
 
static void log_deallocate_buffer (log_t &log)
 Deallocates the log buffer. More...
 
static void log_allocate_write_ahead_buffer (log_t &log)
 Allocates the log write-ahead buffer (aligned to system page for easier migrations between NUMA nodes). More...
 
static void log_deallocate_write_ahead_buffer (log_t &log)
 Deallocates the log write-ahead buffer. More...
 
static void log_allocate_flush_events (log_t &log)
 Allocates the array with flush events. More...
 
static void log_deallocate_flush_events (log_t &log)
 Deallocates the array with flush events. More...
 
static void log_deallocate_write_events (log_t &log)
 Deallocates the array with write events. More...
 
static void log_allocate_write_events (log_t &log)
 Allocates the array with write events. More...
 
static void log_allocate_recent_written (log_t &log)
 Allocates the log recent written buffer. More...
 
static void log_deallocate_recent_written (log_t &log)
 Deallocates the log recent written buffer. More...
 
static void log_allocate_recent_closed (log_t &log)
 Allocates the log recent closed buffer. More...
 
static void log_deallocate_recent_closed (log_t &log)
 Deallocates the log recent closed buffer. More...
 
static void log_reset_encryption_buffer (log_t &log)
 Resets the log encryption buffer (used to write encryption headers). More...
 
Resizing of buffers
static void log_calc_buf_size (log_t &log)
 Calculates proper size of the log buffer and updates related fields. More...
 
bool log_buffer_resize_low (log_t &log, size_t new_size, lsn_t end_lsn)
 Changes size of the log buffer. More...
 
bool log_buffer_resize (log_t &log, size_t new_size)
 Changes size of the log buffer. More...
 
void log_write_ahead_resize (log_t &log, size_t new_size)
 Resizes the write ahead buffer in the redo log. More...
 
Start / stop of background threads
static void log_pause_writer_threads (log_t &log)
 Pauses writer, flusher and notifiers and switches user threads to write log as former version. More...
 
static void log_resume_writer_threads (log_t &log)
 Resumes writer, flusher and notifiers and switches user threads not to write log. More...
 
void log_writer_thread_active_validate ()
 Validates that the log writer thread is active. More...
 
void log_background_write_threads_active_validate (const log_t &log)
 Validates that the log writer, flusher threads are active. More...
 
void log_background_threads_active_validate (const log_t &log)
 Validates that all the log background threads are active. More...
 
void log_background_threads_inactive_validate ()
 Validates that all the log background threads are inactive. More...
 
void log_start_background_threads (log_t &log)
 Starts all the log background threads. More...
 
void log_stop_background_threads (log_t &log)
 Stops all the log background threads. More...
 
void log_stop_background_threads_nowait (log_t &log)
 Marks the flag which tells log threads to stop and wakes them. More...
 
void log_make_empty_and_stop_background_threads (log_t &log)
 Function similar to. More...
 
void log_wake_threads (log_t &log)
 Wakes up all log threads which are alive. More...
 
void log_control_writer_threads (log_t &log)
 Pause / resume the log writer, the log flusher, the log write notifier and the log flush notifier threads based on innodb_log_writer_threads value. More...
 
Status printing
void log_print (const log_t &log, FILE *file)
 Prints information about important lsn values used in the redo log, and some statistics about speed of writing and flushing of data. More...
 
void log_refresh_stats (log_t &log)
 Refreshes the statistics used to print per-second averages in log_print(). More...
 
void log_update_exported_variables (const log_t &log)
 
Log position locking (for replication)
void log_position_lock (log_t &log)
 Lock redo log. More...
 
void log_position_unlock (log_t &log)
 Unlock the locked redo log. More...
 
void log_position_collect_lsn_info (const log_t &log, lsn_t *current_lsn, lsn_t *checkpoint_lsn)
 Collect coordinates in the locked redo log. More...
 
Log - persisting flags
template<typename T >
static void log_update_flags (log_t &log, T update_function)
 Updates and persists the updated log flags to disk. More...
 
void log_persist_enable (log_t &log)
 Enable redo logging and persist the information. More...
 
void log_persist_disable (log_t &log)
 Disable redo logging and persist the information. More...
 
void log_persist_crash_safe (log_t &log)
 Persist the information that it is safe to restart server. More...
 
void log_persist_initialized (log_t &log)
 Marks the redo log files as belonging to the initialized data directory with initialized set of redo log files. More...
 
void log_crash_safe_validate (log_t &log)
 Asserts that the log is not marked as crash-unsafe. More...
 

Variables

ut::aligned_pointer< log_t, ut::INNODB_CACHE_LINE_SIZE > * log_sys_object
 Redo log system. More...
 
log_tlog_sys
 Redo log system (singleton). More...
 

Detailed Description

Redo log system - provides durability for unflushed modifications to contents of data pages.

This file covers general maintenance, including:

  1. Allocation and deallocation of the redo log data structures.
  2. Initialization and shutdown of the redo log.
  3. Start / stop for the log background threads.
  4. Runtime updates of server variables.
  5. Extending size of the redo log buffers.
  6. Locking redo position (for replication).

Created 12/9/1995 Heikki Tuuri

Function Documentation

◆ log_allocate_buffer()

static void log_allocate_buffer ( log_t log)
static

Calculates proper size for the log buffer and allocates the log buffer.

Parameters
[out]logredo log

◆ log_allocate_flush_events()

static void log_allocate_flush_events ( log_t log)
static

Allocates the array with flush events.

Parameters
[out]logredo log

◆ log_allocate_recent_closed()

static void log_allocate_recent_closed ( log_t log)
static

Allocates the log recent closed buffer.

Parameters
[out]logredo log

◆ log_allocate_recent_written()

static void log_allocate_recent_written ( log_t log)
static

Allocates the log recent written buffer.

Parameters
[out]logredo log

◆ log_allocate_write_ahead_buffer()

static void log_allocate_write_ahead_buffer ( log_t log)
static

Allocates the log write-ahead buffer (aligned to system page for easier migrations between NUMA nodes).

Parameters
[out]logredo log

◆ log_allocate_write_events()

static void log_allocate_write_events ( log_t log)
static

Allocates the array with write events.

Parameters
[out]logredo log

◆ log_background_threads_active_validate()

void log_background_threads_active_validate ( const log_t log)

Validates that all the log background threads are active.

Used only to assert, that the state is correct.

Parameters
[in]logredo log

◆ log_background_threads_inactive_validate()

void log_background_threads_inactive_validate ( )

Validates that all the log background threads are inactive.

Used only to assert, that the state is correct.

◆ log_background_write_threads_active_validate()

void log_background_write_threads_active_validate ( const log_t log)

Validates that the log writer, flusher threads are active.

Used only to assert, that the state is correct.

Parameters
[in]logredo log

◆ log_buffer_resize()

bool log_buffer_resize ( log_t log,
size_t  new_size 
)

Changes size of the log buffer.

This is a thread-safe version. It is used by SET GLOBAL innodb_log_buffer_size = X.

Parameters
[in,out]logredo log
[in]new_sizerequested new size
Returns
true iff succeeded in resize

◆ log_buffer_resize_low()

bool log_buffer_resize_low ( log_t log,
size_t  new_size,
lsn_t  end_lsn 
)

Changes size of the log buffer.

This is a non-thread-safe version which might be invoked only when there are no concurrent possible writes to the log buffer. It is used in log_buffer_reserve() when a requested size to reserve is larger than size of the log buffer.

Parameters
[in,out]logredo log
[in]new_sizerequested new size
[in]end_lsnmaximum lsn written to log buffer
Returns
true iff succeeded in resize

◆ log_calc_buf_size()

static void log_calc_buf_size ( log_t log)
static

Calculates proper size of the log buffer and updates related fields.

Calculations are based on current value of srv_log_buffer_size. Note, that the proper size of the log buffer should be a power of two.

Parameters
[out]logredo log

◆ log_control_writer_threads()

void log_control_writer_threads ( log_t log)

Pause / resume the log writer, the log flusher, the log write notifier and the log flush notifier threads based on innodb_log_writer_threads value.

Note
Calls to this function should be protected externally by some mutex. The caller innodb_log_writer_threads_update() is protected by LOCK_global_system_variables in mysqld.

◆ log_crash_safe_validate()

void log_crash_safe_validate ( log_t log)

Asserts that the log is not marked as crash-unsafe.

Parameters
[in,out]logredo log

◆ log_deallocate_buffer()

static void log_deallocate_buffer ( log_t log)
static

Deallocates the log buffer.

Parameters
[out]logredo log

◆ log_deallocate_flush_events()

static void log_deallocate_flush_events ( log_t log)
static

Deallocates the array with flush events.

Parameters
[out]logredo log

◆ log_deallocate_recent_closed()

static void log_deallocate_recent_closed ( log_t log)
static

Deallocates the log recent closed buffer.

Parameters
[out]logredo log

◆ log_deallocate_recent_written()

static void log_deallocate_recent_written ( log_t log)
static

Deallocates the log recent written buffer.

Parameters
[out]logredo log

◆ log_deallocate_write_ahead_buffer()

static void log_deallocate_write_ahead_buffer ( log_t log)
static

Deallocates the log write-ahead buffer.

Parameters
[out]logredo log

◆ log_deallocate_write_events()

static void log_deallocate_write_events ( log_t log)
static

Deallocates the array with write events.

Parameters
[out]logredo log

◆ log_fix_first_rec_group()

static void log_fix_first_rec_group ( lsn_t  block_lsn,
Log_data_block_header block_header,
lsn_t  start_lsn 
)
static

◆ log_make_empty_and_stop_background_threads()

void log_make_empty_and_stop_background_threads ( log_t log)

Function similar to.

See also
log_stop_background_threads() except that it stops all the log threads in such a way, that the redo log will be logically empty after the threads are stopped.
Note
It is caller responsibility to ensure that all threads other than the log_files_governor cannot produce new redo log records when this function is being called.

◆ log_pause_writer_threads()

static void log_pause_writer_threads ( log_t log)
static

Pauses writer, flusher and notifiers and switches user threads to write log as former version.

NOTE: These pause/resume functions should be protected by mutex while serving. The caller innodb_log_writer_threads_update() is protected by LOCK_global_system_variables in mysqld.

Parameters
[out]logredo log

◆ log_persist_crash_safe()

void log_persist_crash_safe ( log_t log)

Persist the information that it is safe to restart server.

Parameters
[in,out]logredo log

◆ log_persist_disable()

void log_persist_disable ( log_t log)

Disable redo logging and persist the information.

Parameters
[in,out]logredo log

◆ log_persist_enable()

void log_persist_enable ( log_t log)

Enable redo logging and persist the information.

Parameters
[in,out]logredo log

◆ log_persist_initialized()

void log_persist_initialized ( log_t log)

Marks the redo log files as belonging to the initialized data directory with initialized set of redo log files.

Flushes the log_flags without the flag LOG_HEADER_FLAG_NOT_INITIALIZED to the newest redo log file.

Parameters
[in,out]logredo log

◆ log_position_collect_lsn_info()

void log_position_collect_lsn_info ( const log_t log,
lsn_t current_lsn,
lsn_t checkpoint_lsn 
)

Collect coordinates in the locked redo log.

Parameters
[in]loglocked redo log
[out]current_lsnstores current lsn there
[out]checkpoint_lsnstores checkpoint lsn there

◆ log_position_lock()

void log_position_lock ( log_t log)

Lock redo log.

Both current lsn and checkpoint lsn will not change until the redo log is unlocked.

Parameters
[in,out]logredo log to lock

◆ log_position_unlock()

void log_position_unlock ( log_t log)

Unlock the locked redo log.

Parameters
[in,out]logredo log to unlock

◆ log_print()

void log_print ( const log_t log,
FILE *  file 
)

Prints information about important lsn values used in the redo log, and some statistics about speed of writing and flushing of data.

Parameters
[in]logredo log for which print information
[out]filefile where to print

◆ log_refresh_stats()

void log_refresh_stats ( log_t log)

Refreshes the statistics used to print per-second averages in log_print().

Parameters
[in,out]logredo log

◆ log_reset_encryption_buffer()

static void log_reset_encryption_buffer ( log_t log)
static

Resets the log encryption buffer (used to write encryption headers).

Parameters
[out]logredo log

◆ log_resume_writer_threads()

static void log_resume_writer_threads ( log_t log)
static

Resumes writer, flusher and notifiers and switches user threads not to write log.

Parameters
[out]logredo log

◆ log_start()

dberr_t log_start ( log_t log,
lsn_t  checkpoint_lsn,
lsn_t  start_lsn,
bool  allow_checkpoints = true 
)

Starts the initialized redo log system using a provided checkpoint_lsn and current lsn.

Block for current_lsn must be properly initialized in the log buffer prior to calling this function. Therefore a proper value of first_rec_group must be set for that block before log_start is called.

Parameters
[in,out]logredo log
[in]checkpoint_lsncheckpoint lsn
[in]start_lsncurrent lsn to start at
[in]allow_checkpointstrue iff allows writing newer checkpoints
Returns
DB_SUCCESS or error

◆ log_start_background_threads()

void log_start_background_threads ( log_t log)

Starts all the log background threads.

This can be called only, when the threads are inactive. This should never be called concurrently. This may not be called during read-only mode.

Parameters
[in,out]logredo log

◆ log_stop_background_threads()

void log_stop_background_threads ( log_t log)

Stops all the log background threads.

This can be called only, when the threads are active. This should never be called concurrently. This may not be called in read-only mode. Note that is is impossible to start log background threads in such case.

Parameters
[in,out]logredo log

◆ log_stop_background_threads_nowait()

void log_stop_background_threads_nowait ( log_t log)

Marks the flag which tells log threads to stop and wakes them.

Does not wait until they are stopped.

Parameters
[in,out]logredo log

◆ log_sys_check_directory()

static dberr_t log_sys_check_directory ( const Log_files_context ctx,
std::string &  path,
bool &  found_files 
)
static

Checks if the redo log directory exists, can be listed and contains at least one redo log file.

Remarks
Redo log file is recognized only by looking at the file name, accordingly to the configured ruleset (
See also
log_configure()).
Parameters
[in]ctxredo log files context
[in,out]pathpath to redo log directory checked during this call
[out]found_filestrue iff found at least one redo log file
Return values
DB_SUCCESSif succeeded to list the log directory
DB_ERRORif failed to list the existing log directory
DB_NOT_FOUNDif the log directory does not exist

◆ log_sys_check_format()

static dberr_t log_sys_check_format ( const log_t log)
static

Recognizes log format and emits corresponding message to the error log.

Returns DB_ERROR if format is too old and no longer supported or format is not the current one and innodb_force_recovery != 0 is passed.

Parameters
[in]logredo log
Returns
DB_SUCCESS or error

◆ log_sys_close()

void log_sys_close ( )

Close the log system and free all the related memory.

◆ log_sys_create()

static void log_sys_create ( )
static

Allocates the log system and initializes all log mutexes and log events.

◆ log_sys_free()

static void log_sys_free ( )
static

Free the log system data structures.

Deallocate all the related memory.

◆ log_sys_handle_creator()

static dberr_t log_sys_handle_creator ( log_t log)
static

Handles the log creator name stored on the disk (in the redo log files).

Does whatever is required to handle the creator name and decides if further initialization of InnoDB might be allowed.

Remarks
Recognizes log creator by its name and marks that within recv_sys by setting recv_sys->is_meb_db or recv_sys->is_cloned_db. Disables the dblwr if log files were created by MEB. Emits message to the error log when the recognized creator name is not an usual MySQL's creator name.
Parameters
[in]logredo log with log.m_creator_name read from files
Return values
DB_SUCCESSif further initialization might be continued (note that potentially the creator name could be recognized as an unknown one)
DB_ERRORif further initialization must not be continued (for example because a foreign creator name was detected but read-only mode is turned on)

◆ log_sys_init()

dberr_t log_sys_init ( bool  expect_no_files,
lsn_t  flushed_lsn,
lsn_t new_files_lsn 
)

Initializes log_sys and finds existing redo log files, or creates a new set of redo log files.

New redo log files are created in following cases:

  • there are no existing redo log files in the log directory,
  • existing set of redo log files is not marked as fully initialized (flag LOG_HEADER_FLAG_NOT_INITIALIZED exists in the newest file).

After this call, the log_sys global variable is allocated and initialized. InnoDB might start recovery then.

Remarks
The redo log files are not resized in this function, because before resizing log files, InnoDB must run recovery and ensure log files are logically empty. The redo resize is currently the only scenario in which the initialized log_sys might become closed by log_sys_close() and then re-initialized by another call to log_sys_init().
Note
Note that the redo log system is NOT ready for user writes after this call is finished. The proper order of calls looks like this:
  • log_sys_init(),
  • log_start(),
  • log_start_background_threads() and this sequence is executed inside srv_start() in srv0start.cc (interleaved with remaining logic of the srv_start())
Note this function also verifies that REDO logs are in known format.
Parameters
[in]expect_no_filestrue means we should return DB_ERROR if log files are present in the directory before proceeding any further
[in]flushed_lsnlsn at which new redo log files might be started if they had to be created during this call; this should be lsn stored in the system tablespace header at offset FIL_PAGE_FILE_FLUSH_LSN if the data directory has been initialized;
[out]new_files_lsnupdated to the lsn of the first checkpoint created in the new log files if new log files are created; else: 0
Returns
DB_SUCCESS or error

◆ log_update_exported_variables()

void log_update_exported_variables ( const log_t log)

◆ log_update_flags()

template<typename T >
static void log_update_flags ( log_t log,
update_function 
)
static

Updates and persists the updated log flags to disk.

Parameters
[in,out]logredo log
[in]update_functionfunctor called on existing flags, supposed to make changes

◆ log_wake_threads()

void log_wake_threads ( log_t log)

Wakes up all log threads which are alive.

Parameters
[in,out]logredo log

◆ log_write_ahead_resize()

void log_write_ahead_resize ( log_t log,
size_t  new_size 
)

Resizes the write ahead buffer in the redo log.

Parameters
[in,out]logredo log
[in]new_sizenew size (in bytes)

◆ log_writer_thread_active_validate()

void log_writer_thread_active_validate ( )

Validates that the log writer thread is active.

Used only to assert, that the state is correct.

Variable Documentation

◆ log_sys

log_t* log_sys

Redo log system (singleton).

◆ log_sys_object

Redo log system.

Singleton used to populate global pointer.