#include <fil0innodb_pages_persistence.h>

Inheritance diagram for ib::fil::Pages_persistence:

Public Member Functions
void	redo_create_tablespace (Tablespaces_nodes_interface::Tablespace_id, uint32_t, const char *) override
	Redo a create tablespace storage operation for a tablespace. More...

void	redo_delete_tablespace (Tablespaces_nodes_interface::Tablespace_id, const char *) override
	Redo a delete tablespace storage operation for a tablespace. More...

void	redo_rename_tablespace (Tablespace_id, const char , const char ) override
	Redo a rename operation for a tablespace. More...

Status	init () override
	Allocate and construct helper data structures and start persisting changes to pages, but do not start persisting the progress of that operation: the values reported by get_checkpoint_lsn() should not change, and ib::redo::handler->do_not_need_smaller_than(lsn) should not be called. More...

Status	assume_checkpoint_lsn (Lsn min_needed_lsn) override
	The caller asks the implementation to assume that redo log below min_needed_lsn is not available and is not needed anymore because pages have all changes below this lsn already persisted. More...

void	enable_checkpointing () override
	From now on, the implementation is permitted to bump the min needed lsn value reported and persisted - that is, not only can it persist changes to pages but moreover it can persist the state of progress of this operation. More...

void	enable_periodical_checkpoints () override
	From now on, not only advancing the min needed lsn is permitted, but also encouraged. More...

void	disable_checkpointing () override
	From now on, the implementation is no longer permitted to bump the min needed lsn. More...

void	deinit () override
	Deinitialize whatever was initialized in init(). More...

void	mtr_has_dirtied_pages (Lsn start_lsn, Lsn end_lsn, ::Flush_observer observer, ut::Function_reference< void(ut::Function_reference< void(buf_block_t )>)> iterate_over_dirty_pages) override
	The caller informs that it is now committing an mtr which has dirtied some pages. More...

void	page_became_dirty (struct buf_block_t *buf_block) override
	A callback that is called from buf_flush_note_oldest_modification() once for each page that became dirty, assuming buf_flush_note_oldest_modification() is called at all from mtr_has_dirtied_pages(), for example by calling buf_flush_note_modification() from visitor passed to iterate_over_dirty_pages. More...

void	persist_tablespace (Tablespace_id space_id, const trx_t *trx) override
	Makes sure that for a specified tablespace all changes to pages are persisted. More...

void	persist_tablespaces (::Flush_observer *observer) override
	Makes sure that all changes to pages that are observed by a specified flush observer are persisted. More...

ut::Expected< std::vector< Tablespaces_nodes_interface::Tablespace_id >, Status >	recover_pages (Lsn &clean_shutdown_lsn) override
	Run recovery of all tablespaces assuming they already contain all the changes at least up to the provided clean_shutdown_lsn. More...

Status	recover_tables () override
	Run recovery of tables (in particular: mysql.innodb_dynamic_metadata) based on information gathered during recover_pages(). More...

void	page_is_to_be_evicted (Tablespace_id space_id, Page_number page_no, Lsn modification_lsn) override
	A callback that is called once the page is freed from the BufferPool. More...

Lsn	get_checkpoint_lsn () const override
	Returns the checkpoint lsn. More...

void	request_sharp_checkpoint () override
	The caller request the implementation to ensure that when it returns, it no longer needs the redo log generated so far, i.e. More...

ib::Sys_var_handler_interface &	config_handler () override
	Returns an object responsible for handling updates of dynamic sys-vars related to pages persistence, such as: More...

ib::Monitoring_interface &	get_monitoring () override
	Returns an object responsible for reporting values of monitors related to pages persistence, such as: More...

Public Member Functions inherited from ib::fil::Pages_persistence_interface
virtual	~Pages_persistence_interface ()=default

Private Member Functions
dberr_t	scan_tablespaces ()
	Scan on disk tablespace files in recover() which is called during server bootstrap. More...

Private Attributes
MetadataRecover *	m_dict_metadata {}
	PTM information gathered during recover_pages() to be applied during recover_tables() More...

Additional Inherited Members
Public Types inherited from ib::fil::Pages_persistence_interface
enum class	Status { SUCCESS = 0 , IO_ERROR }

using	Page_number = Tablespace_node_handle_interface::Page_number
	Type used for numbering the pages in the tablespace. More...

using	Tablespace_id = Tablespaces_nodes_interface::Tablespace_id
	Type used for giving the tablespaces unique number. More...

using	Lsn = ib::redo::Lsn

Member Function Documentation

◆ assume_checkpoint_lsn()

Pages_persistence::Status ib::fil::Pages_persistence::assume_checkpoint_lsn ( Lsn min_needed_lsn )

overridevirtual

The caller asks the implementation to assume that redo log below min_needed_lsn is not available and is not needed anymore because pages have all changes below this lsn already persisted.

Also, the next change to come will be min_needed_lsn. In other words the redo log is logically empty: all changes were already persisted to pages. That is, the checkpoint lsn is at min_needed_lsn and no dirty page will be added with a lower lsn in future. This is currently used only when initializing a new instance.

Parameters

[in] min_needed_lsn the current snapshot of the database (its tablespaces), the start and the end of the redo log, the only available version of it

Return values

SUCCESS	everything went fine: the min_needed_lsn value was stored in Redo log handler's metadata, and get_checkpoint_lsn() would return min_needed_lsn if called
IO_ERROR	could not persist the value of min needed lsn (make a checkpoint)

Implements ib::fil::Pages_persistence_interface.

◆ config_handler()

ib::Sys_var_handler_interface & ib::fil::Pages_persistence::config_handler ( )

overridevirtual

Returns an object responsible for handling updates of dynamic sys-vars related to pages persistence, such as:

innodb_log_checkpoint_fuzzy_now
innodb_checkpoint_disabled

Implements ib::fil::Pages_persistence_interface.

◆ deinit()

void ib::fil::Pages_persistence::deinit ( )

overridevirtual

Deinitialize whatever was initialized in init().

This is called when shutting down in a planned way (as opposed to the destructor which might be called indirectly from mysql_exit on abort, in which case the order of destruction can be problematic - for example PFS or some other piece of infrastructure might be no longer available)

Implements ib::fil::Pages_persistence_interface.

◆ disable_checkpointing()

void ib::fil::Pages_persistence::disable_checkpointing ( )

overridevirtual

From now on, the implementation is no longer permitted to bump the min needed lsn.

That is, the log checkpointer thread should be stopped now.

Implements ib::fil::Pages_persistence_interface.

◆ enable_checkpointing()

void ib::fil::Pages_persistence::enable_checkpointing ( )

overridevirtual

From now on, the implementation is permitted to bump the min needed lsn value reported and persisted - that is, not only can it persist changes to pages but moreover it can persist the state of progress of this operation.

That is, the log checkpointer can be started now.

Implements ib::fil::Pages_persistence_interface.

◆ enable_periodical_checkpoints()

void ib::fil::Pages_persistence::enable_periodical_checkpoints ( )

overridevirtual

From now on, not only advancing the min needed lsn is permitted, but also encouraged.

Before a call to this function it should only be done if really needed for correctness, but avoided otherwise to make the behaviour more deterministic. From now on, unsolicited, periodical advancements are no longer a problem and can actually help in reclaiming redo log space. That is, the log checkpointer can start periodical checkpoints.

Implements ib::fil::Pages_persistence_interface.

◆ get_checkpoint_lsn()

Pages_persistence::Lsn ib::fil::Pages_persistence::get_checkpoint_lsn ( ) const

overridevirtual

Returns the checkpoint lsn.

This function should be monotone in time. The returned value itself should be crash-resistant, that is, after the crash the implementation should return get_checkpoint_lsn() value at least as large as returned now.

Returns: the oldest lsn in the redo log still needed by this implementation of pages persistence

Implements ib::fil::Pages_persistence_interface.

◆ get_monitoring()

ib::Monitoring_interface & ib::fil::Pages_persistence::get_monitoring ( )

overridevirtual

Returns an object responsible for reporting values of monitors related to pages persistence, such as:

MONITOR_OVLD_LSN_BUF_DIRTY_PAGES_ADDED
MONITOR_OVLD_BUF_OLDEST_LSN_APPROX
MONITOR_OVLD_BUF_OLDEST_LSN_LWM
MONITOR_OVLD_MAX_AGE_ASYNC
MONITOR_OVLD_MAX_AGE_SYNC
Returns
The instance to use to obtain values of relevant monitors

Implements ib::fil::Pages_persistence_interface.

◆ init()

Pages_persistence::Status ib::fil::Pages_persistence::init ( )

overridevirtual

Allocate and construct helper data structures and start persisting changes to pages, but do not start persisting the progress of that operation: the values reported by get_checkpoint_lsn() should not change, and ib::redo::handler->do_not_need_smaller_than(lsn) should not be called.

That is: page cleaners can start now, but log check pointer can not.

Returns: IO_ERROR due to one of plenty possible IO errors, SUCCESS otherwise

Implements ib::fil::Pages_persistence_interface.

◆ mtr_has_dirtied_pages()

void ib::fil::Pages_persistence::mtr_has_dirtied_pages	(	Lsn	start_lsn,
		Lsn	end_lsn,
		::Flush_observer *	observer,
		ut::Function_reference< void(ut::Function_reference< void(buf_block_t *)>)>	iterate_over_dirty_pages
	)

overridevirtual

The caller informs that it is now committing an mtr which has dirtied some pages.

The pages are still latched in BP, but it will not dirty any more pages, and the range of lsns is already assigned to it (unless the redo logging is not enabled for this mtr, i.e. MTR_LOG_SHORT_INSERTS, MTR_LOG_NO_REDO or MTR_LOG_NONE was used). This is a good moment to add dirtied pages to flush lists.

Parameters

[in]	start_lsn	the start lsn of the committing mtr. It is 0 iff this mtr is not redo logged.
[in]	end_lsn	the end lsn of the committing mtr. It is 0 iff this mtr is not redo logged.
[in]	observer	the Flush observer (if any) attached to this mtr
[in]	iterate_over_dirty_pages	a function that can be used to iterate over all pages dirtied committing mtr by passing a visitor callback to it. The visitor will be called AT LEAST once for each dirtied page.

Implements ib::fil::Pages_persistence_interface.

◆ page_became_dirty()

void ib::fil::Pages_persistence::page_became_dirty ( struct buf_block_t * buf_block )

overridevirtual

A callback that is called from buf_flush_note_oldest_modification() once for each page that became dirty, assuming buf_flush_note_oldest_modification() is called at all from mtr_has_dirtied_pages(), for example by calling buf_flush_note_modification() from visitor passed to iterate_over_dirty_pages.

TODO: is it really needed if we have mtr_has_dirtied_pages() ?

Parameters

[in] buf_block Block for Page that became dirty.

Implements ib::fil::Pages_persistence_interface.

◆ page_is_to_be_evicted()

void ib::fil::Pages_persistence::page_is_to_be_evicted	(	Tablespace_id	space_id,
		Page_number	page_no,
		Lsn	modification_lsn
	)

overridevirtual

A callback that is called once the page is freed from the BufferPool.

The page is already removed from the Page Hash table, but the mutex that guards the page object and latch protecting the Page Hash table, are still held, so this page will not be read in again while this method is executing.

Parameters

[in]	space_id	The id of the tablespace containing the evicted page
[in]	page_no	Zero-based number of the evicted page within its tablespace
[in]	modification_lsn	The highest modification LSN of the page that is being evicted - that is there are no modifications to this page in the range [modification_lsn, peek_first_unassigned_lsn), as otherwise the page would be dirty or latched and could not be evicted. In other words, if you want to bring this page back to BP in the same state, you need to ensure it is recovered at least up to modification_lsn.

Implements ib::fil::Pages_persistence_interface.

◆ persist_tablespace()

void ib::fil::Pages_persistence::persist_tablespace	(	Tablespace_id	space_id,
		const trx_t *	trx
	)

overridevirtual

Makes sure that for a specified tablespace all changes to pages are persisted.

This requires all pages from the Buffer Pool (for the specified tablespace) that were dirty before this call, to be made clean. This will remove such pages from the flush_list, but not from the Buffer Pool itself (and thus will not remove from LRU list). In case the specified transaction is interrupted before or during the call, whatever pages were not yet processed, will not be touched and thus will remain in both lists.

Parameters

[in]	space_id	The id of the tablespace to persist its pages.
[in]	trx	Transaction, if any, to monitor for interrupted operation.

Implements ib::fil::Pages_persistence_interface.

◆ persist_tablespaces()

void ib::fil::Pages_persistence::persist_tablespaces ( ::Flush_observer * observer )

overridevirtual

Makes sure that all changes to pages that are observed by a specified flush observer are persisted.

This requires all such pages from the Buffer Pool that were dirty before this call, to be made clean. This will remove such pages from the flush_list, but not from the Buffer Pool itself (and thus will not remove from LRU list). In case the transaction that is linked to the specified FlushObserver instance is interrupted before or during the call, all pages that were not flushed before noticing the interruption, will be removed from the flush_list, causing them to be considered clean. They will not be ever written back even when evicted from the LRU. This means that the changes observed by the FlushObserver must not be redologged, and any future modifications of this page must start from initializing it before use.

Parameters

[in] observer The Flush observer used to monitor the flushing process.

Implements ib::fil::Pages_persistence_interface.

◆ recover_pages()

ut::Expected< std::vector< ib::fil::Tablespaces_nodes_interface::Tablespace_id >, Pages_persistence::Status > ib::fil::Pages_persistence::recover_pages ( Lsn & clean_shutdown_lsn )

overridevirtual

Run recovery of all tablespaces assuming they already contain all the changes at least up to the provided clean_shutdown_lsn.

Note that this is not the value of min needed lsn which the persistence implementation keeps track of more precisely by itself, nor even the (perhaps lower) value stored when making a checkpoint via store_metadata(0,..) - it is just the last value InnoDB was aware of at last clean shutdown obtained via get_checkpoint_lsn() and stored in system tablespace's header. The init() method must be called before attempting recover_pages(..). If CLONE is supported, then it should call:

arch_init(),
Arch_page_sys::post_recovery_init() as well.
Parameters

[in,out] clean_shutdown_lsn The lsn at the last clean shutdown the InnoDB knows about. This method will set it to the recovered lsn.

Return values

List of tablespaces IDs found for all discovered tablespaces

IO_ERROR if could not perform recovery due to one of plenty possible IO errors

Implements ib::fil::Pages_persistence_interface.

◆ recover_tables()

Pages_persistence::Status ib::fil::Pages_persistence::recover_tables ( )

overridevirtual

Run recovery of tables (in particular: mysql.innodb_dynamic_metadata) based on information gathered during recover_pages().

Used to update PTM's B-tree using the information from redo log. TODO: recover_tables() does not really belong to pages persistence, as tables are a high-level concept, and recovering them is basically running SQL on top of already recovered B-tree. However, this is how InnoDB currently handles PTM

the redo log contains logical records which are interpreted as requests to REPLACE INTO mysql.innodb_dynamic_metadata. Therefore recovery is two-stage: first we recover_pages() to get physically consistent B-tree, then we apply collected PTM changes extracted from those records. Once we implement WL#15550 we will no longer produce this kind of redo log records. If we additionally assume that upgrade will require a clean shutdown then the redo log will no longer contain such PTM redo log records, and we will no longer have to handle them and will be able to remove this method.

Implements ib::fil::Pages_persistence_interface.

◆ redo_create_tablespace()

void ib::fil::Pages_persistence::redo_create_tablespace	(	Tablespaces_nodes_interface::Tablespace_id	space_id,
		uint32_t	flags,
		const char *	path
	)

inlineoverridevirtual

Redo a create tablespace storage operation for a tablespace.

This operation is done when REDOs are replayed and must be successful.

Parameters

[in]	space_id	Tablespace id
[in]	flags	Tablespace flags
[in]	path	path of tablespace storage

Implements ib::fil::Pages_persistence_interface.

◆ redo_delete_tablespace()

void ib::fil::Pages_persistence::redo_delete_tablespace	(	Tablespaces_nodes_interface::Tablespace_id	space_id,
		const char *	path
	)

inlineoverridevirtual

Redo a delete tablespace storage operation for a tablespace.

This operation is done when REDOs are replayed and must be successful.

Parameters

[in]	space_id	Tablespace id
[in]	path	path of tablespace storage

Implements ib::fil::Pages_persistence_interface.

◆ redo_rename_tablespace()

void ib::fil::Pages_persistence::redo_rename_tablespace	(	Tablespace_id	space_id,
		const char *	old_path,
		const char *	new_path
	)

inlineoverridevirtual

Redo a rename operation for a tablespace.

This operation is done when REDOs are replayed and must be successful.

Parameters

[in]	space_id	Tablespace id
[in]	old_path	current path tablespace storage
[in]	new_path	target path of tablespace storage

Implements ib::fil::Pages_persistence_interface.

◆ request_sharp_checkpoint()

void ib::fil::Pages_persistence::request_sharp_checkpoint ( )

overridevirtual

The caller request the implementation to ensure that when it returns, it no longer needs the redo log generated so far, i.e.

the value returned by subsequent calls to get_checkpoint_lsn() must be at least the value of ib::redo::handler->peek_first_unassigned_lsn() would have returned before the call to request_sharp_checkpoint(). This method is used from:

various debug hooks
places which want to ensure redo log is logically empty because:
- it's MEB or other "archival" situation where we need a clear start of log
- it is a slow or normal shutdown
after we did some non-redo-logged changes to tablespaces
we've imported a new tablespace
we've initialized legacy double-write pages and want them written to disc, and would not like to have to "recover" changes to them
we've initialized new undo log rsegs - TODO: I don't understand this part

Implements ib::fil::Pages_persistence_interface.

◆ scan_tablespaces()

dberr_t ib::fil::Pages_persistence::scan_tablespaces ( )

private

Scan on disk tablespace files in recover() which is called during server bootstrap.

Member Data Documentation

◆ m_dict_metadata

MetadataRecover* ib::fil::Pages_persistence::m_dict_metadata {}

private

PTM information gathered during recover_pages() to be applied during recover_tables()

The documentation for this class was generated from the following files:

storage/innobase/include/fil0innodb_pages_persistence.h
storage/innobase/fil/fil0innodb_pages_persistence.cc

List	of tablespaces IDs found for all discovered tablespaces
IO_ERROR	if could not perform recovery due to one of plenty possible IO errors

Public Member Functions

Private Member Functions

Private Attributes

Additional Inherited Members

Member Function Documentation

◆ assume_checkpoint_lsn()

◆ config_handler()

◆ deinit()

◆ disable_checkpointing()

◆ enable_checkpointing()

◆ enable_periodical_checkpoints()

◆ get_checkpoint_lsn()

◆ get_monitoring()

◆ init()

◆ mtr_has_dirtied_pages()

◆ page_became_dirty()

◆ page_is_to_be_evicted()

◆ persist_tablespace()

◆ persist_tablespaces()

◆ recover_pages()

◆ recover_tables()

◆ redo_create_tablespace()

◆ redo_delete_tablespace()

◆ redo_rename_tablespace()

◆ request_sharp_checkpoint()

◆ scan_tablespaces()

Member Data Documentation

◆ m_dict_metadata