WL#5457: Add an LRU list to the data dictionary
Status: Complete — Priority: Medium
Close and remove non-system tables from the InnoDB data dictionary. We also make an exception for tables that are either referenced by another table or refer to another table in a foreign key relationship. This should address BUG#20877.
For tables that have a reference count of zero, are not referenced by any foreign key and don't refer to any table (FK) are removed from the dictionary cache if it reaches an upper bound. The count is controlled by the parameter --table-cache-size We only make a best effort to remove, there is nothing stopping the data dictionary from going beyond the upper limit. Added functions: dict_check_sys_foreign_tables_exist(void) dict_index_remove_from_cache_low() dict_table_remove_from_cache_low() dict_lru_validate() dict_lru_find_table() dict_non_lru_find_table() dict_move_to_mru() dict_table_open_on_name_low() dict_table_open_on_name_no_stats() dict_table_can_be_evicted() dict_make_room_in_cache() dict_table_move_from_lru_to_non_lru() dict_table_move_from_non_lru_to_lru() dict_table_move_from_non_lru_to_lru() dict_table_remove_from_cache_low() dict_index_remove_from_cache_low() dict_lru_validate() dict_lru_find_table() dict_non_lru_find_table() innobase_add_index_cleanup() lock_table_has_locks() Rename dict_table_decrement_handle_count() to dict_table_close() Rename dict_table_get_on_id() to dict_table_open_on_id() Rename dict_table_get() to dict_table_open_on_name() Fields added to dict_sys_t: UT_LIST_BASE_NODE_T(dict_table_t) table_non_LRU /*!< List of tables that can't be evicted from the cache. */ Fields added to dict_table_t: unsigned can_be_evicted:1; /*!< TRUE if it's not an InnoDB system table or a table that has no FK relationships */ ulint n_rec_locks; /*!< Count of the number of record locks on this table. We use this to determine whether we can evict the table from the dictionary cache. It is protected by the lock mutex. */ ulint n_ref_count; /*!< count of how many handles are opened to this table; dropping of the table is NOT allowed until this count gets to zero; MySQL does NOT itself check the number of open handles at drop */ Added following enumerations to sym_tab_entry. SYM_UNSET, /*!< Unset entry. */ SYM_TABLE_REF_COUNTED, /*!< database table name, ref counted. Must be closed explicitly. */ Added new global variable srv_dict_table_cache_size. This is tied to the --table-cache-size parameter. With this change all InnoDB tables should be opened using the public interface of dict0dict.c and not the low-level equivalent functions, this applies to internal InnoDB calls too. I've made the low level variants static and private to the dict/ source files. The tables must be closed after use. We use this to reference count all opens including open requests from MySQL layer. Previously we only reference counted open requests from the MySQL layer. Several changes were made to existing code to conform to the new behavior. The changes are too numerous to list individually. When opening or closing a table the only important requirement is whether the caller has the dict_sys_t::mutex locked or not. This state is passed to the open/close functions using a flag. Which is set to TRUE if the caller has locked the dict mutex or FALSE if it is not locked. The internal SQL parser has been updated to use the new interfaces too. A new symbol type is introduced for reference counted tables. In fact we don't open the table multiple times as we did in the past. The master thread will attempt to keep the dictionary table cache within limits periodically. Because this check can be expensive we limit it to once every 60 seconds and scan only 50% of the LRU list when the server is active and 100% when the server is idle. Before a table is dropped we move it from the LRU list to the non-LRU list this is to ensure that the table is not asynchronously evicted from the cache. Added several debug assertions that check whether a table has any kind of lock on the table. There was a bug in existing code that resulted in a table being dropped while it had record locks on the table. The record locks were implicit locks of recovered transactions that had been converted to explicit locks. See rb://379.
Copyright (c) 2000, 2016, Oracle Corporation and/or its affiliates. All rights reserved.