WL#14628: Compatibility option to enable old replication terminology in P_S instrumentation
Affects: Server-8.0
—
Status: Complete
EXECUTIVE SUMMARY ================= We introduce a compatibility option to enable the provious terminology in strings shown in performance_schema, information_schema, SHOW PROCESSLIST, SHOW REPLICA STATUS, and slow query log. The strings were modified in 8.0.26 by WL#14194. In WL#14194, a number of performance_schema instrumentation names were changed. This includes locks, condition variables, memory allocations, thread names, thread stages, and thread commands. The result is that the observed contents of performance_schema tables have changed. This may impact monitoring tools that check these strings. Such monitoring tools can enable the option when upgrading to 8.0.26, to ensure that they keep functioning as expected. USER STORIES ============ U1. As a person that manages a MySQL deployment observed by monitoring tools, - I need to upgrade the server without changing instrumentation names, - so that upgrade does not have to wait for the monitoring tools to be adjusted to use the new names. U2. As a person that manages a MySQL deployment observed by multiple independent monitoring tools, - I need to enable the old instrumentation names only for some sessions, - so that I can fix each tool independently, and in the process disable the compatibility mode selectively for those tools that have been fixed. SCOPE ===== The option only impacts terminology visible in performance_schema and modified in WL#14194. The option impacts all places where changed instrumentation names occur: performance_schema and information_schema tables, SHOW PROCESSLIST, SHOW REPLICA STATUS, and slow query log. LIMITATIONS =========== There is no way to enable compatibility for each string individually. A monitoring tool needs to adjust all strings at once. There is no way to enable compatibility for each table, log file, or command individually. While performance_schema/information_schema tables and SHOW PROCESSLIST/SHOW REPLICA STATUS show the old names based on the session option; slow query log shows old names based on the global option. The reason is that the performance_schema/information_schema tables and SHOW PROCESSLIST/SHOW REPLICA STATUS compute the result sets dynamically, so this computation can be intercepted, whereas the logs exist in shared global storage - files or tables - which are accessed through generic methods such as SELECT, which cannot be intercepted.
1. FUNCTIONAL REQUIREMENTS
==========================
FR1. There shall be a configuration option that controls whether
instrumentation names existing prior to 8.0.26 are shown instead
of the newer names.
FR2. In commands and tables whose contents are only materialized by a
session that observes them, i.e.,
perforance_schema/information_schema tables, SHOW PROCESSLIST,
and SHOW REPLICA STATUS, the session value for the configuration
option shall decide whether the old or new names are used.
FR3. In logs whose contents are written to shared storage, i.e., slow
query log, the global value for the configuration option shall
decide whether the old or new names are used.
FR4. The configuration option shall be deprecated.
1. NEW OPTION ============= NAME: terminology_use_previous SCOPES: session, global, persisted, command-line DYNAMIC: yes PRIVILEGES: none TYPE: enumeration VALUES: NONE, BEFORE_8_0_26 DEFAULT: NONE PRIVILEGES: NONE REPLICATED: NO DEPRECATED: YES DESCRIPTION: Make monitoring tables and statements use the identifiers that were in use before they were changed in a given release. That includes names for mutexes, read/write locks, condition variables, memory allocations, thread names, thread stages, and thread commands. When the session option is set to BEFORE_8_0_26, the session uses the names that were in use until 8.0.25, when it selects from performance_schema tables, or selects from INFORMATION_SCHEMA.PROCESSLIST, or issues SHOW PROCESSLIST or SHOW REPLICA STATUS. When the global option is set to BEFORE_8_0_26, new sessions use BEFORE_8_0_26 as default for the session option, and in addition the thread commands that were in use until 8.0.25 are written to the slow query log. 2. IMPACT OF NEW OPTION ======================= 2.1. Categories of instrumentation names ---------------------------------------- The option affects the following categories of instrumentation names: - Mutexes. - Read/write locks. - Condition variables. - Memory allocations. - Thread names. - Thread stages. - Thread commands. 2.2. Full list of instrumentation names affected by the option -------------------------------------------------------------- In all the following cases, the words 'master', 'slave', and 'mts' have been replaced by 'source', 'replica', and 'mta'. Capitalization has been preserved. In two cases, other parts of the identifier were modified; each of them is marked with a footnote with a justification. Mutexes: Old: "Master_info::data_lock" New: "Source_info::data_lock" Old: "Master_info::run_lock" New: "Source_info::run_lock" Old: "Master_info::sleep_lock" New: "Source_info::sleep_lock" Old: "Master_info::info_thd_lock" New: "Source_info::info_thd_lock" Old: "Master_info::rotate_lock" New: "Source_info::rotate_lock" Old: "Slave_reporting_capability::err_lock" New: "Replica_reporting_capability::err_lock" Old: "key_mts_temp_table_LOCK" New: "key_mta_temp_table_LOCK" Old: "key_mts_gaq_LOCK" New: "key_mta_gaq_LOCK" Old: "Relay_log_info::slave_worker_hash_lock" New: "Relay_log_info::replica_worker_hash_lock" Old: "LOCK_slave_list" New: "LOCK_replica_list" Old: "LOCK_slave_net_timeout" New: "LOCK_replica_net_timeout" Old: "LOCK_sql_slave_skip_counter" New: "LOCK_sql_replica_skip_counter" Read/write locks: Old: "LOCK_sys_init_slave" New: "LOCK_sys_init_replica" Condition variables: Old: "Relay_log_info::slave_worker_hash_lock" New[1]: "Relay_log_info::replica_worker_hash_cond" Old: "Master_info::data_cond" New: "Source_info::data_cond" Old: "Master_info::start_cond" New: "Source_info::start_cond" Old: "Master_info::stop_cond" New: "Source_info::stop_cond" Old: "Master_info::sleep_cond" New: "Source_info::sleep_cond" Old: "Master_info::rotate_cond" New: "Source_info::rotate_cond" Old: "Relay_log_info::mts_gaq_cond" New: "Relay_log_info::mta_gaq_cond" Memory allocations: Old: "Slave_job_group::group_relay_log_name" New: "Replica_job_group::group_relay_log_name" Old: "rpl_slave::check_temp_dir" New: "rpl_replica::check_temp_dir" Old: "SLAVE_INFO" New: "REPLICA_INFO" Old: "show_slave_status_io_gtid_set" New: "show_replica_status_io_gtid_set" Old: "Relay_log_info::mts_coor" New: "Relay_log_info::mta_coor" Thread names: Old: "slave_io" New: "replica_io" Old: "slave_sql" New: "replica_sql" Old: "slave_worker" New: "replica_worker" Thread stages: Old: "Changing master" New: "Changing replication source" Old: "Checking master version" New: "Checking source version" Old: "Connecting to master" New: "Connecting to source" Old: "Flushing relay log and master info repository." New: "Flushing relay log and source info repository." Old: "Killing slave" New: "Killing replica" Old: "Master has sent all binlog to slave; waiting for more updates" New: "Source has sent all binlog to replica; waiting for more updates" Old: "Queueing master event to the relay log" New: "Queueing source event to the relay log" Old: "Reconnecting after a failed master event read" New[3]: "Reconnecting after a failed source event read" Old: "Reconnecting after a failed registration on master" New[3]: "Reconnecting after a failed registration on source" Old: "Registering slave on master" New: "Registering replica on source" Old: "Sending binlog event to slave" New: "Sending binlog event to replica" Old: "Slave has read all relay log; waiting for more updates" New: "Replica has read all relay log; waiting for more updates" Old: "Waiting for slave workers to process their queues" New: "Waiting for replica workers to process their queues" Old: "Waiting for Slave Worker queue" New: "Waiting for Replica Worker queue" Old: "Waiting for Slave Workers to free pending events" New: "Waiting for Replica Workers to free pending events" Old: "Waiting for Slave Worker to release partition" New: "Waiting for Replica Worker to release partition" Old: "Waiting for master to send event" New: "Waiting for source to send event" Old: "Waiting for master update" New: "Waiting for source update" Old: "Waiting for the slave SQL thread to free enough relay log space" New[2]: "Waiting for the replica SQL thread to free relay log space" Old: "Waiting for slave mutex on exit" New: "Waiting for replica mutex on exit" Old: "Waiting for slave thread to start" New: "Waiting for replica thread to start" Old: "Waiting for the slave SQL thread to advance position" New: "Waiting for the replica SQL thread to advance position" Old: "Waiting to reconnect after a failed master event read" New[3]: "Waiting to reconnect after a failed source event read" Old: "Waiting to reconnect after a failed registration on master" New[3]: "Waiting to reconnect after a failed registration on source" Old: "Waiting until MASTER_DELAY seconds after master executed event" New: "Waiting until SOURCE_DELAY seconds after source executed event" Thread commands: Old: "Register Slave" New: "Register Replica" Footnotes: [1] Also corrected 'lock' to 'cond'. [2] Also removed the word 'enough' to keep the text within the length limit. [3] These were missed in WL#14194, and this worklog renames them. 2.3. Monitoring tables, commands, and logs that expose the names ---------------------------------------------------------------- Mutexes are visible with a 'wait/synch/mutex/' prefix in: - the NAME column of performance_schema.mutex_instances - the EVENT_NAME column of all performance_schema.events_waits_* tables, i.e.: - events_waits_summary_by_account_by_event_name - events_waits_summary_by_host_by_event_name - events_waits_summary_by_instance - events_waits_summary_by_thread_by_event_name - events_waits_summary_by_user_by_event_name - events_waits_summary_global_by_event_name - events_waits_current - events_waits_history - events_waits_history_long Read/write locks are visible with a 'wait/synch/rwlock/' prefix in: - the NAME column of performance_schema.rwlock_instances - the EVENT_NAME column of all performance_schema.events_waits_* tables Condition variables are visible with a 'wait/synch/cond/' prefix in: - the NAME column of performance_schema.cond_instances - the EVENT_NAME column of all performance_schema.events_waits_* tables Memory allocations are visible with a 'memory/sql/' prefix in: - the EVENT_NAME column of all performance_schema.memory_summary_* tables, i.e.: - memory_summary_by_account_by_event_name - memory_summary_by_host_by_event_name - memory_summary_by_thread_by_event_name - memory_summary_by_user_by_event_name - memory_summary_global_by_event_name Thread names are visible with a 'thread/sql/' prefix in: - the NAME column of performance_schema.threads Thread stages are visible: - with a 'stage/sql/' prefix, in the EVENT_NAME column of all performance_schema.events_stages_* tables, i.e.: - events_stages_summary_by_account_by_event_name - events_stages_summary_by_host_by_event_name - events_stages_summary_by_thread_by_event_name - events_stages_summary_by_user_by_event_name - events_stages_summary_global_by_event_name - events_stages_current - events_stages_history - events_stages_history_long - without prefix, in: - the PROCESSLIST_STATE column of performance_schema.threads - the STATE column of performance_schema.processlist - the STATE column of SHOW PROCESSLIST - the STATE column of INFORMATION_SCHEMA.PROCESSLIST - the Replica_IO_State column of SHOW REPLICA STATUS - the Slave_IO_State column of SHOW SLAVE STATUS - the Replica_SQL_Running_State of SHOW REPLICA STATUS - the Slave_SQL_Running_State of SHOW SLAVE STATUS Thread commands are visible: - with a 'statement/com/' prefix, in the EVENT_NAME column of all performance_schema.events_statements_history* and performance_schema.events_statements_summary_*_by_event_name tables, i.e.: - events_statements_summary_by_account_by_event_name - events_statements_summary_by_host_by_event_name - events_statements_summary_by_thread_by_event_name - events_statements_summary_by_user_by_event_name - events_statements_summary_global_by_event_name - events_statements_history - events_statements_history_long - without a prefix, in: - the PROCESSLIST_COMMAND column of performance_schema.threads - the COMMAND column of performance_schema.processlist - the COMMAND column of SHOW PROCESSLIST - the COMMAND column of INFORMATION_SCHEMA.PROCESSLIST - the sql_text column of mysql.slow_log Note: in general, thread commands are also visible in the command_type column of mysql.general_log, and in the COMMAND_CLASS element of the audit log. However, this worklog only alters one thread command - Register Replica - and by chance that is excluded from both the general log and from the audit log. 2.4. How the global/session options affect tables, commands, and logs --------------------------------------------------------------------- Among all monitoring tables, commands, and logs listed in 2.3, some are computed/materialized only in the moment a session observes them, whereas others are sent to shared storage, such as a table or file: - The values in performance_schema and information_schema tables are computed by the client that selects from it. - The values in SHOW PROCESSLIST and SHOW REPLICA STATUS statements are computed by the client that executes SHOW. - The values in the slow log are stored in a table or file. For names computed by the client that show them, the name is only visible to that client. For names sent to shared storage, any client may observe them. Therefore: - the session option decides which name should be used in performance_schema and information_schema tables, and in SHOW PROCESSLIST and SHOW REPLICA STATUS; - the global option decides which names should be used in the slow log. 2.4.1. Slow log, User story U2, and overloading of global option ................................................................ It can be limiting that names in logs are only affected by the global option, for two reasons: 1. User Story U2, which justifies using a session option, may apply equally well to logs. I.e., there may be two different monitoring applications observing the slow log, for instance, and one of them may be fixed to understand the new terminology before the other. However, we do not see a better solution, since is not possible to translate the contents of a table or file on a per-session basis. The reason is that the string is in shared storage (not per-session), and will be processed using generic techniques that cannot be intercepted in order to translate a string. For instance, the file may be accessed directly by third party tools such as grep, i.e., not even through a client. So we do not address user story U2 for logs. 2. The global option is overloaded: one the one hand, like most other global options, it serves as the default for the session option in new sessions. On the other hand, it controls what is written to the slow log. An application where clients expect new terminology, but uses a monotoring application that expects old terminology in the slow log, cannot both satisfy the needs of the application and the expected default for the sessions. We overload the option in order to create too many options. While we do not have first hand information, we guess that the impact on applications of these limitations are likely to be small. The reason is that there is only one translated identifier that appears in a log, "Register Replica", and: - "Register Replica" is an internal command, used only by a replica when it initates the replication connection. So there is not much reason for users to watch it. - "Register Replica" is fast - just a few CPU instructions. So it is unlikely to ever end up in any slow log. The first limitation cannot be fixed; users will have the trouble of upgrading all slow log parsers at the same moment. In case the second limitation turns out to be an issue, we can introduce a separate option at that time, allowing slow log parsers to expect different terminology than the session default. We leave that as possible future work. 3. SECURITY =========== No impact on security. 4. UPGRADE, DOWNGRADE, AND CROSS-VERSION REPLICATION ==================================================== It does not harm upgrade, downgrade, or cross-version replication that this feature did not exist in 8.0.25 and was added in 8.0.26. The feature provides a means to upgrade to 8.0.26 without affecting applications.
We implement this worklog in several steps.
0. Move LOCK_replica_list to the global list
Context:
This is a simplification that was relevant in an earlier version of
the worklog. Now the worklog does not depend on it, but we keep the
refactoring since it improves the code.
Background:
There is an API for registering mutex names with performance_schema.
Other server modules than replication don't invoke the API from the
module, but add the mutex to the list in mysqld.cc and rely on the
call to the API in mysqld.cc. For other instrumentation classes than
mutexes - condition variables, memory allocations, thread stages,
thread names, etc - replication depends on the invocation in
mysqld.cc. So it is only replication mutexes that have a special
case.
Problems:
- Replication mutexes are initialized in a different way, compared
to other instrumentation, which is a bit inconsequent.
- There is a need for a specific call to a cleanup routine in
replication.
- There is an error condition where slave_list_inited has not been
initialized. By initializing the mutex together with other mutexes,
we ensure it happens very early during server start, so no code that
uses the mutex can execute before that.
Fix:
Move the initialization of replication mutex instrumentation to the
global initialization.
1. Encapsulate access to P_S instrumentation names
This is a refactoring only and has no observable consequences; hence
no test case.
We encapsulate all instrumentation names in performance_schema in a
class. This will allow us to change the implementation of the class
to use a backward compatible name, in the future.
2. New option that enables old terminology
Introduce the new configuration option
terminology_use_previous. When set to NONE, the server is
not in compatibility mode and uses the newest version of all names.
When set to BEFORE_8_0_26, it uses the names that were present up to
and including 8.0.26.
We introduce a framework that translates new names to old names. The
framework uses std::unordered_map objects where the keys are new names
and values are old names. There is such an object for each
instrumentation class, i.e., one for mutexes, one for thread names,
etc. All those maps are collected in a std::unordered_map where the
keys are instrumentation class identifiers and values are the
maps. Finally, there is a vector where each element contains the
translation maps for a specific version. This makes the framework
extensible, so in case we modify more names in the future, the
configuration option can be extended with a new enumeration value and
the vector with a new element.
We modify the class introduced in Step 1, so that:
- The setter for PFS_instr_name takes the instrumentation class as an
argument. For each version in the vector, it looks up the
instrumentation class to get the translation map, and looks up the
name in that map. If the name is found, the setter caches the
alternative name in member variables of PFS_instr_name. It also
caches the version in which the name appeared.
- The getter for PFS_instr_name checks if the session configuration
option is enabled. If it is, and the version specified in the option
is older than the version cached in the member variable, it returns
the alternative name cached in the member variable.
3. Fix tests after adding option
Fix tests that break whenever you add an option.
4. Encapsulate access to command_name array
This is a refactoring that has no observable consequences; hence no
test case.
We encapsulate all access to the 'command_name' array. This will
allow us to change the implementation of the class to use a compatible
name when the option is enabled.
5. Translate statement names
Intercept the Command_name class, so that it translates to old names
according to the @@terminology_use_previous option. When
the name is going to be returned in a result-set, it should use the
session option to determine to use the old or new name. When the name
is going to be written to a log, it should use the global option,
since the observer of the log is not the current session.
Since we only have one command name that needs translation, and don't
currently plan to have others, we simplify the implementation of
Command_class and allow only one name to be translated. The
limitation is only in the layout of private members, not in the public
API, and can be generalized later if needed.
6. Encapsulate THD::proc_info
This is a refactoring that has no observable consequences; hence no
test case.
We do the following small refactorings related to thread stage info:
- Encapsulate all access to THD::proc_info. This will allow us to
change the implementation of the class to use a compatible name when
the option is enabled. This is the primary objective of the patch,
and the majority of the changes.
- Remove a case where group replication checks the current thread
stage by comparing a string, and instead make it compare the thread
stage keys. This is more efficient, and not sensitive to changing the
string.
- Change type of THD::m_current_stage_key from unsigned int to
PSI_stage_key. PSI_stage_key is a type alias for unsigned int, and
describes better the meaning of this variable.
- Move the definition of THD::m_proc_info near where related members
are defined.
- Change name of the parameter to sql_show.cc:thread_state_info to a
more descriptive one. This also facilitates the next patch where
another parameter is added.
7. Translate thread stage stored in THD
Intercept access to THD::proc_info where it is necessary to convert
the name according to @@session.terminology_use_previous.
This makes INFORMATION_SCHEMA.PROCESSLIST and SHOW REPLICA STATUS show
results according to @@session.terminology_use_previous.
Many places in the code should not translate the string, because they
just save the old stage in order to revert it later. So we introduce
a new member function that translates the string, and uses it only
where needed.
The existing terminology_use_previous framework assumes that
translated strings have a prefix, corresponding to the
performance_schema instrumentation class, for instance, "stage/sql/".
However, this prefix is added only in performance_schema tables, and
not in INFORMATION_SCHEMA.PROCESSLIST or SHOW REPLICA STATUS.
Therefore, we add an argument to
terminology_use_previous::lookup that allows the caller to
specify that the given string does not have a prefix and the return
value should not have one.
8. Test case
Test cases to verify that the compatibility option works as expected.
9. Translate four more stage names
Background:
WL#14194 replaced outdated replicaion terminology in, among other
things, thread stages. This was implemented by inspecting all
PSI_stage_info definitions.
WL#14628 adds a compatibility option that enables the old names.
Problem:
1. The thread stage text can be set without going through the
PSI_stage_info framework. There were three such texts that
contained outdated terminology, which were missed in WL#14194.
2. The code for this was using arrays of char* where C++ classes are
more appropriate. And the last array element was never used.
3. There were three debug symbols that enabled big code blocks, which
were never used in any tests.
Fix:
1a. Use the PSI_stage_info framework. Change to new terminology.
1b. Add the strings to the terminology_use_previous
framework introduced in WL#14628.
2. Restructure the code to use a class instead of arrays of char*.
Do not re-introduce the last array element in the class.
3. Remove the debug symbols and the code blocks they enable.
Copyright (c) 2000, 2023, Oracle Corporation and/or its affiliates. All rights reserved.