The tablespace memory cache. More...

Public Types
using	Fil_shards = std::vector< Fil_shard * >

Public Member Functions
	Fil_system (size_t n_shards, size_t max_open)
	Constructor. More...

	~Fil_system ()
	Destructor. More...

fil_space_t *	space_acquire (space_id_t space_id, bool silent)
	Acquire a tablespace when it could be dropped concurrently. More...

dberr_t	prepare_open_for_business (bool read_only_mode)
	Update the DD if any files were moved to a new location. More...

void	flush_file_spaces ()
	Flush to disk the writes in file spaces possibly cached by the OS (note: spaces of type FIL_TYPE_TEMPORARY are skipped) More...

void	purge ()
	Clean up the shards. More...

size_t	count_undo_deleted (space_id_t undo_num)
	Count how many truncated undo space IDs are still tracked in the buffer pool and the file_system cache. More...

bool	is_deleted (space_id_t space_id) noexcept
	Check if a particular undo space_id for a page in the buffer pool has been deleted recently. More...

fil_space_t *	get_space_by_name (const char *name)
	Fetch the fil_space_t instance that maps to the name. More...

bool	is_greater_than_max_id (space_id_t space_id) const
	Check a space ID against the maximum known tablespace ID. More...

bool	update_maximum_space_id (space_id_t space_id)
	Update the maximum known space ID if it's smaller than max_id. More...

bool	assign_new_space_id (space_id_t *space_id)
	Assigns a new space id for a new single-table tablespace. More...

void	wait_while_ios_in_progress () const
	Allows other threads to advance work while we wait for I/Os to complete. More...

bool	close_file_in_all_LRU ()
	Tries to close a file in all the LRU lists. More...

void	close_all_files ()
	Close all open files. More...

size_t	get_minimum_limit_for_open_files (size_t n_files_not_belonging_in_lru) const
	Returns minimum open files limit to be set to allow the specified number of non-LRU files opened. More...

bool	set_open_files_limit (size_t &new_max_open_files)
	Changes the maximum opened files limit. More...

size_t	get_open_files_limit () const
	Returns maximum number of allowed opened files. More...

dberr_t	iterate (Fil_iterator::Function &f)
	Iterate through all persistent tablespace files (FIL_TYPE_TABLESPACE) returning the nodes via callback function cbk. More...

size_t	encryption_rotate ()
	Rotate the tablespace keys by new master key. More...

void	encryption_reencrypt (const std::vector< space_id_t > &sid_vector)
	Reencrypt tablespace keys by current master key. More...

bool	lookup_for_recovery (space_id_t space_id)
	Lookup the tablespace ID for recovery and DDL log apply. More...

dberr_t	open_for_recovery (space_id_t space_id)
	Open a tablespace that has a redo log record to apply. More...

void	moved (dd::Object_id object_id, space_id_t space_id, const char *space_name, const std::string &old_path, const std::string &new_path, bool dd_flag_missing)
	Note that a file has been relocated. More...

Fil_shard *	shard_by_id (space_id_t space_id) const
	Fil_shard by space ID. More...

void	mutex_acquire_all () const
	Acquire all the mutexes. More...

void	mutex_release_all () const
	Release all the mutexes. More...

bool	validate () const
	Checks the consistency of the tablespace cache. More...

bool	mutex_owned_all () const
	Check if all mutexes are owned. More...

dberr_t	rename_tablespace_name_in_cache (space_id_t space_id, const char old_name, const char new_name)
	Rename a tablespace in tablespace cache. More...

Static Public Member Functions
static size_t	get_limit_for_non_lru_files (size_t open_files_limit)
	Returns maximum number of allowed non-LRU files opened for a specified open files limit. More...

static bool	space_belongs_in_LRU (const fil_space_t *space)
	Determines if a file belongs to the least-recently-used list. More...

Private Member Functions
fil_load_status	ibd_open_for_recovery (space_id_t space_id, const std::string &path, fil_space_t *&space)
	Open an ibd tablespace and add it to the InnoDB data structures. More...

	Fil_system (Fil_system &&)=delete

	Fil_system (const Fil_system &)=delete

Fil_system &	operator= (const Fil_system &)=delete

Private Attributes
Fil_shards	m_shards
	Fil_shards managed. More...

fil::detail::Open_files_limit	m_open_files_limit

std::atomic< space_id_t >	m_max_assigned_id
	Maximum space id in the existing tables, or assigned during the time mysqld has been up; at an InnoDB startup we scan the data dictionary and set here the maximum of the space id's of the tables there. More...

dd_fil::Tablespaces	m_moved
	List of tablespaces that have been relocated. More...

std::mutex	m_moved_mutex
	Mutex protecting the list of relocated tablespaces. More...

std::atomic_size_t	m_next_shard_to_close_from_LRU {}
	Next index (modulo number of shards) to try to close a file from the LRU list to distribute closures evenly between the shards. More...

std::atomic_size_t	m_n_files_not_belonging_in_lru {}
	Current number of files that are not belonging in LRU. More...

ib::Throttler	m_MANY_NON_LRU_FILES_OPENED_throttler {}
	Throttles messages about high files not belonging in LRU count, the warning ER_IB_WARN_MANY_NON_LRU_FILES_OPENED. More...

ib::Throttler	m_TRYING_TO_OPEN_FILE_FOR_LONG_TIME_throttler {}
	Throttles messages about long waiting for opened files limit, the warning ER_IB_MSG_TRYING_TO_OPEN_FILE_FOR_LONG_TIME. More...

ib::Throttler	m_ACCESSING_NONEXISTINC_SPACE_throttler {}
	Throttles messages about accessing space that was already removed, the warning ACCESSING_NONEXISTINC_SPACE. More...

Friends
class	Fil_shard

Detailed Description

The tablespace memory cache.

Member Typedef Documentation

◆ Fil_shards

using Fil_system::Fil_shards = std::vector<Fil_shard *>

Constructor & Destructor Documentation

◆ Fil_system() [1/3]

Fil_system::Fil_system	(	size_t	n_shards,
		size_t	max_open
	)

Constructor.

Parameters

[in]	n_shards	Number of shards to create
[in]	max_open	Maximum number of open files

◆ ~Fil_system()

Fil_system::~Fil_system ( )

Destructor.

◆ Fil_system() [2/3]

Fil_system::Fil_system ( Fil_system && )

privatedelete

◆ Fil_system() [3/3]

Fil_system::Fil_system ( const Fil_system & )

privatedelete

Member Function Documentation

◆ assign_new_space_id()

bool Fil_system::assign_new_space_id ( space_id_t * space_id )

Assigns a new space id for a new single-table tablespace.

This works simply by incrementing the global counter. If 4 billion ids is not enough, we may need to recycle ids.

Parameters

[out] space_id Set this to the new tablespace ID

Returns: true if assigned, false if not

◆ close_all_files()

void Fil_system::close_all_files ( )

Close all open files.

◆ close_file_in_all_LRU()

bool Fil_system::close_file_in_all_LRU ( )

Tries to close a file in all the LRU lists.

The caller must hold the mutex.

Returns: true if success, false if should retry later

◆ count_undo_deleted()

size_t Fil_system::count_undo_deleted ( space_id_t undo_num )

inline

Count how many truncated undo space IDs are still tracked in the buffer pool and the file_system cache.

Parameters

[in] undo_num undo tablespace number.

Returns: number of undo tablespaces that are still in memory.

◆ encryption_reencrypt()

void Fil_system::encryption_reencrypt ( const std::vector< space_id_t > & sid_vector )

Reencrypt tablespace keys by current master key.

◆ encryption_rotate()

size_t Fil_system::encryption_rotate ( )

Rotate the tablespace keys by new master key.

Returns: the number of tablespaces that failed to rotate.

◆ flush_file_spaces()

void Fil_system::flush_file_spaces ( )

Flush to disk the writes in file spaces possibly cached by the OS (note: spaces of type FIL_TYPE_TEMPORARY are skipped)

◆ get_limit_for_non_lru_files()

size_t Fil_system::get_limit_for_non_lru_files ( size_t open_files_limit )

static

Returns maximum number of allowed non-LRU files opened for a specified open files limit.

◆ get_minimum_limit_for_open_files()

size_t Fil_system::get_minimum_limit_for_open_files ( size_t n_files_not_belonging_in_lru ) const

Returns minimum open files limit to be set to allow the specified number of non-LRU files opened.

This is inverse function for the get_limit_for_non_lru_files.

◆ get_open_files_limit()

size_t Fil_system::get_open_files_limit ( ) const

inline

Returns maximum number of allowed opened files.

◆ get_space_by_name()

fil_space_t * Fil_system::get_space_by_name ( const char * name )

inline

Fetch the fil_space_t instance that maps to the name.

Parameters

[in] name Tablespace name to lookup

Returns: tablespace instance or nullptr if not found.

◆ ibd_open_for_recovery()

fil_load_status Fil_system::ibd_open_for_recovery	(	space_id_t	space_id,
		const std::string &	path,
		fil_space_t *&	space
	)

private

Open an ibd tablespace and add it to the InnoDB data structures.

This is similar to fil_ibd_open() except that it is used while processing the redo log, so the data dictionary is not available and very little validation is done. The tablespace name is extracted from the dbname/tablename.ibd portion of the filename, which assumes that the file is a file-per-table tablespace. Any name will do for now. General tablespace names will be read from the dictionary after it has been recovered. The tablespace flags are read at this time from the first page of the file in validate_for_recovery().

Parameters

[in]	space_id	tablespace ID
[in]	path	path/to/databasename/tablename.ibd
[out]	space	the tablespace, or nullptr on error

Returns: status of the operation

◆ is_deleted()

bool Fil_system::is_deleted ( space_id_t space_id )

inlinenoexcept

Check if a particular undo space_id for a page in the buffer pool has been deleted recently.

Its space_id will be found in Fil_shard::m_deleted_spaces until Fil:shard::checkpoint removes the fil_space_t from Fil_system.

Parameters

[in] space_id Tablespace ID to check.

Returns: true if this space_id is in the list of recently deleted spaces.

◆ is_greater_than_max_id()

bool Fil_system::is_greater_than_max_id ( space_id_t space_id ) const

inline

Check a space ID against the maximum known tablespace ID.

Parameters

[in] space_id Tablespace ID to check

Returns: true if it is > than maximum known tablespace ID.

◆ iterate()

dberr_t Fil_system::iterate ( Fil_iterator::Function & f )

Iterate through all persistent tablespace files (FIL_TYPE_TABLESPACE) returning the nodes via callback function cbk.

Parameters

[in] f Callback

Returns: any error returned by the callback function.

◆ lookup_for_recovery()

bool Fil_system::lookup_for_recovery ( space_id_t space_id )

Lookup the tablespace ID for recovery and DDL log apply.

Parameters

[in] space_id Tablespace ID to lookup

Returns: true if the space ID is known.

◆ moved()

void Fil_system::moved	(	dd::Object_id	object_id,
		space_id_t	space_id,
		const char *	space_name,
		const std::string &	old_path,
		const std::string &	new_path,
		bool	dd_flag_missing
	)

inline

Note that a file has been relocated.

Parameters

[in]	object_id	Server DD tablespace ID
[in]	space_id	InnoDB tablespace ID
[in]	space_name	Tablespace name
[in]	old_path	Path to the old location
[in]	new_path	Path scanned from disk
[in]	dd_flag_missing	This tablespace is outside default data directory but missing the DD_TABLE_DATA_DIRECTORY flag. This can happen before versions 8.0.37/8.4.1/9.0.0

◆ mutex_acquire_all()

void Fil_system::mutex_acquire_all ( ) const

inline

Acquire all the mutexes.

◆ mutex_owned_all()

bool Fil_system::mutex_owned_all ( ) const

inline

Check if all mutexes are owned.

Returns: true if all owned.

◆ mutex_release_all()

void Fil_system::mutex_release_all ( ) const

inline

Release all the mutexes.

◆ open_for_recovery()

dberr_t Fil_system::open_for_recovery ( space_id_t space_id )

Open a tablespace that has a redo log record to apply.

Parameters

[in] space_id Tablespace ID

Returns: DB_SUCCESS if the open was successful

◆ operator=()

Fil_system & Fil_system::operator= ( const Fil_system & )

privatedelete

◆ prepare_open_for_business()

dberr_t Fil_system::prepare_open_for_business ( bool read_only_mode )

Update the DD if any files were moved to a new location.

Free the Tablespace_files instance.

Parameters

[in] read_only_mode true if InnoDB is started in read only mode.

Returns: DB_SUCCESS if all OK

◆ purge()

void Fil_system::purge ( )

inline

Clean up the shards.

◆ rename_tablespace_name_in_cache()

dberr_t Fil_system::rename_tablespace_name_in_cache	(	space_id_t	space_id,
		const char *	old_name,
		const char *	new_name
	)

Rename a tablespace in tablespace cache.

Use the space_id to find the shard. This operation updates the tablespace name in cache and doesn't produce any persistent log (REDO/DDL_LOG etc.) and doesn't change any thing on disk or in DD.

Parameters

[in]	space_id	tablespace ID
[in]	old_name	old tablespace name
[in]	new_name	new tablespace name

Returns: DB_SUCCESS on success

◆ set_open_files_limit()

bool Fil_system::set_open_files_limit ( size_t & new_max_open_files )

Changes the maximum opened files limit.

Parameters

[in,out] new_max_open_files New value for the open files limit. If the limit cannot be changed, the value is changed to a minimum value recommended. If there are any concurrent calls to set_open_files_limit in progress, setting the limit will fail and the new_max_open_files will be set to 0.

Returns: true if the new limit was set.

◆ shard_by_id()

Fil_shard * Fil_system::shard_by_id ( space_id_t space_id ) const

inline

Fil_shard by space ID.

Parameters

[in] space_id Tablespace ID

Returns: reference to the shard

◆ space_acquire()

fil_space_t * Fil_system::space_acquire	(	space_id_t	space_id,
		bool	silent
	)

Acquire a tablespace when it could be dropped concurrently.

Used by background threads that do not necessarily hold proper locks for concurrency control.

Parameters

[in]	space_id	Tablespace ID
[in]	silent	Whether to silently ignore missing tablespaces

Returns: the tablespace, or nullptr if missing or being deleted

◆ space_belongs_in_LRU()

bool Fil_system::space_belongs_in_LRU ( const fil_space_t * space )

static

Determines if a file belongs to the least-recently-used list.

Parameters

[in] space Tablespace to check

Returns: true if the file belongs to fil_system->m_LRU mutex.

◆ update_maximum_space_id()

bool Fil_system::update_maximum_space_id ( space_id_t space_id )

inline

Update the maximum known space ID if it's smaller than max_id.

Parameters

[in] space_id Value to set if it's greater

Returns: true if the maximum space ID was changed by this call.

◆ validate()

bool Fil_system::validate ( ) const

Checks the consistency of the tablespace cache.

Returns: true if ok

◆ wait_while_ios_in_progress()

void Fil_system::wait_while_ios_in_progress ( ) const

inline

Allows other threads to advance work while we wait for I/Os to complete.

Friends And Related Function Documentation

◆ Fil_shard

friend class Fil_shard

friend

Member Data Documentation

◆ m_ACCESSING_NONEXISTINC_SPACE_throttler

ib::Throttler Fil_system::m_ACCESSING_NONEXISTINC_SPACE_throttler {}

private

Throttles messages about accessing space that was already removed, the warning ACCESSING_NONEXISTINC_SPACE.

◆ m_MANY_NON_LRU_FILES_OPENED_throttler

ib::Throttler Fil_system::m_MANY_NON_LRU_FILES_OPENED_throttler {}

private

Throttles messages about high files not belonging in LRU count, the warning ER_IB_WARN_MANY_NON_LRU_FILES_OPENED.

◆ m_max_assigned_id

std::atomic<space_id_t> Fil_system::m_max_assigned_id

private

Maximum space id in the existing tables, or assigned during the time mysqld has been up; at an InnoDB startup we scan the data dictionary and set here the maximum of the space id's of the tables there.

◆ m_moved

dd_fil::Tablespaces Fil_system::m_moved

private

List of tablespaces that have been relocated.

We need to update the DD when it is safe to do so.

◆ m_moved_mutex

std::mutex Fil_system::m_moved_mutex

private

Mutex protecting the list of relocated tablespaces.

◆ m_n_files_not_belonging_in_lru

std::atomic_size_t Fil_system::m_n_files_not_belonging_in_lru {}

private

Current number of files that are not belonging in LRU.

This includes redo and temporary tablespaces, but not files that were temporarily removed from the LRU for I/O.

◆ m_next_shard_to_close_from_LRU

std::atomic_size_t Fil_system::m_next_shard_to_close_from_LRU {}

private

Next index (modulo number of shards) to try to close a file from the LRU list to distribute closures evenly between the shards.

◆ m_open_files_limit

fil::detail::Open_files_limit Fil_system::m_open_files_limit

private

◆ m_shards

Fil_shards Fil_system::m_shards

private

Fil_shards managed.

◆ m_TRYING_TO_OPEN_FILE_FOR_LONG_TIME_throttler

ib::Throttler Fil_system::m_TRYING_TO_OPEN_FILE_FOR_LONG_TIME_throttler {}

private

Throttles messages about long waiting for opened files limit, the warning ER_IB_MSG_TRYING_TO_OPEN_FILE_FOR_LONG_TIME.

The documentation for this class was generated from the following file:

storage/innobase/fil/fil0fil.cc

Public Types

Public Member Functions

Static Public Member Functions

Private Member Functions

Private Attributes

Friends

Detailed Description

Member Typedef Documentation

◆ Fil_shards

Constructor & Destructor Documentation

◆ Fil_system() [1/3]

◆ ~Fil_system()

◆ Fil_system() [2/3]

◆ Fil_system() [3/3]

Member Function Documentation

◆ assign_new_space_id()

◆ close_all_files()

◆ close_file_in_all_LRU()

◆ count_undo_deleted()

◆ encryption_reencrypt()

◆ encryption_rotate()

◆ flush_file_spaces()

◆ get_limit_for_non_lru_files()

◆ get_minimum_limit_for_open_files()

◆ get_open_files_limit()

◆ get_space_by_name()

◆ ibd_open_for_recovery()

◆ is_deleted()

◆ is_greater_than_max_id()

◆ iterate()

◆ lookup_for_recovery()

◆ moved()

◆ mutex_acquire_all()

◆ mutex_owned_all()

◆ mutex_release_all()

◆ open_for_recovery()

◆ operator=()

◆ prepare_open_for_business()

◆ purge()

◆ rename_tablespace_name_in_cache()

◆ set_open_files_limit()

◆ shard_by_id()

◆ space_acquire()

◆ space_belongs_in_LRU()

◆ update_maximum_space_id()

◆ validate()

◆ wait_while_ios_in_progress()

Friends And Related Function Documentation

◆ Fil_shard

Member Data Documentation

◆ m_ACCESSING_NONEXISTINC_SPACE_throttler

◆ m_MANY_NON_LRU_FILES_OPENED_throttler

◆ m_max_assigned_id

◆ m_moved

◆ m_moved_mutex

◆ m_n_files_not_belonging_in_lru

◆ m_next_shard_to_close_from_LRU

◆ m_open_files_limit

◆ m_shards

◆ m_TRYING_TO_OPEN_FILE_FOR_LONG_TIME_throttler