WL#7806: InnoDB: Log-based discovery of built-in tablespaces

Affects: Server-8.0   —   Status: Complete   —   Priority: Medium

This is follow-up to
WL#7142 InnoDB: Simplify tablespace discovery during crash recovery

Write MLOG_FILE_NAME for all persistent tablespaces, not only for *.ibd files.
Currently, this includes the following:

• The InnoDB system tablespace (ibdata*)
• The InnoDB undo log tablespaces (undo*)

On startup, the InnoDB system tablespace and the InnoDB redo log will be opened.
If there are redo log records to be applied since the latest checkpoint,
any tablespaces requiring cleanup will be opened and recovered based on
MLOG_FILE_NAME records in the redo log.

If the MLOG_FILE_NAME records for the system tablespace disagree with
the server configuration affecting the data file names for the system
tablespace, recovery will be aborted with an error message, before
applying any redo log.

After recovery, any undo log tablespaces for which no redo log records were
applied will be opened based on existing mechanism. The system tablespace will
remain open at all times.
FR1. InnoDB must refuse to start up if some system tablespace files
or undo tablespace files are deleted or renamed after killing the server.

FR2. The InnoDB redo log volume (bytes per user transaction) will not grow
significantly. (A few more MLOG_FILE_NAME entries emitted for each log checkpoint.)

FR3. InnoDB will not generate any redo log when the server is not modifying any
pages.

FR4. There should be no significant performance regression.
WL#7142 introduced MLOG_FILE_NAME for user-created tablespaces (*.ibd files).
The log records were not being emitted for predefined redo-logged tablespaces
(the system tablespace and undo log tablespaces).

This WL#7806 will remove the function is_predefined_tablespace() that was
introduced in WL#7142, and treat all redo-logged tablespaces will be treated
in the same way.

This change allows InnoDB crash recovery to notice more cases when the
redo logs do not match the system files. Before WL#7806, we could without any
warning lose changes to undo log tablespaces if the files were missing
(for example, if the startup parameter innodb_undo_directory had been changed).
WL#7806 InnoDB: Log-based discovery of built-in tablespaces

This is follow-up to
WL#7142 InnoDB: Simplify tablespace discovery during crash recovery

We will write MLOG_FILE_NAME for all persistent tablespaces, not only
for *.ibd files. Currently, this includes the following:

* The InnoDB system tablespace (ibdata*)
* The InnoDB undo log tablespaces (undo*)

On startup, only the InnoDB system tablespace and the InnoDB redo log
will be opened. (Any additional undo tablespaces will not be opened yet.)
If there are any redo log records to be applied since the latest
checkpoint, any tablespaces requiring cleanup will be opened and
recovered based on MLOG_FILE_NAME records in the redo log.

If the MLOG_FILE_NAME records for the system tablespace disagree with
the server configuration affecting the data file names for the system
tablespace, recovery will be aborted with an error message, before
applying any redo log.

After recovery, any undo log tablespaces for which no redo log records were
applied will be opened based on existing mechanism. The system tablespace will
remain open at all times.

is_predefined_tablespace(): Remove. All redo-logged tablespaces will
be treated in the same way.

mtr_t::m_undo_space, mtr_t::set_undo_space(): New field and method, to
associate an undo tablespace associate with the mini-transaction.

mtr_t::m_modifies_sys_space, mtr_t::set_sys_modified(): New field and
method, to note that the mini-transaction is modifying the system
tablespace.

mtr_t::set_spaces(): A kind of copy constructor that copies the
information on modified tablespaces from another mini-transaction.

mtr_t::is_undo_space(): A debug method to ensure that set_undo_space()
has been called.

mtr_t::Command::prepare_write(): Invoke fil_spaces_lookup() before
log_mutex_enter(), to look up all tablespaces that were flagged as
modified by the mini-transaction, for invoking
fil_names_write_if_was_clean() after log_mutex_enter().

mtr_write_log_t::m_len: Remove. We will write the entire log.

srv_undo_tablespaces, srv_undo_tablespaces_open: Set the initial value
to 0 on server startup, so that fil_space_belongs_in_lru() will behave
in a predictable way during redo log apply.

trx_rseg_t: Note that space,page_no are constant and need not be
protected by mutex.

fil_load_single_file_tablespace(): Renamed from
fil_load_single_table_tablespace().

fil_space_system_check(): New function, to check that MLOG_FILE_NAME
records match the system tablespace data files.

fil_space_undo_check(): New function, to reopen possibly existing undo
log files after redo log apply has completed.

fil_spaces_lookup(): New function, to replace previous usage of
fil_names_write(). At mini-transaction commit, this looks up all
modified redo logged tablespaces.

fil_names_write_if_was_clean(), fil_names_dirty_and_write(): Update
fil_space_t::max_lsn, and write MLOG_FILE_NAME records if max_lsn was
previously 0, indicating that this is the first time of modifying the
tablespaces since the previous checkpoint.

fil_names_write(): Renamed from fil_names_write_low(). We will get the
fil_space_t* looked up by fil_spaces_lookup().

fil_name_parse(): Support undo tablespaces and multi-file system
tablespace.