WL#11879: Replicate original server version
Affects: Server-8.0 — Status: Complete
Executive Summary ================= Replication topologies may be composed of servers of different versions. These servers may have some features that may be incompatible between themselves. However, there is, currently, no way to extract the version of neither the original nor the immediate server version for a replicated transaction, which creates a lot of difficulties when implementing with cross-version replication approaches. This worklog implements two new session variables: - original_server_version, which will be used to tag each transaction with the version of the server where it was originally executed, more specifically, with the value of @@GLOBAL.version. This variable will be replicated, unchanged, to the servers in the replication topology. - immediate_server_version, used to tag each transaction with the version of the immediate server in the replication topology. Developer Stories ================= As a developer, I want to unequivocally know the original and immediate server version of each transaction so that I can implement adequate cross-version replication strategies without having to introduce new specific compatibility options. As a developer, I want to be able to know the version of the original server where a transaction was executed so I can correctly execute replicated statements which had semantic changes between versions. For instance, the default collation for utf8mb4 changed between 5.7 and 8.0, and the developer could have used original_server_version to determine that the replicated statement was originated in a 5.7 server and should be applied with the 5.7 default. As a developer, I want to be able to know the version of the immediate server in the replication topology so I know how to apply statements that changed syntax between versions. For instance, if 8.0 had changed the CREATE TABLE syntax from 'CREATE TABLE pet (name VARCHAR(20))' to 'CREATE TABLE NAME (pet) COLUMNS (name VARCHAR(20))', the 8.0 server would be able to rely on immediate_server_version to correctly apply the old syntax. User Story ============ As a user, I want to be able to check which are the MySQL versions where the transactions in my replication topology were originally executed to facilitate auditing and debugging possible replication issues. Glossary ======== Original server: the server where a transaction was originally executed. Immediate server: the immediate master in a replication topology. Considering the following replication topology A->B->C, for transactions originally executed by A and replicated to C, we can say that A has no immediate master because it is the transactions' original master, B's immediate master is A, and C's immediate master is B.
Functional Requirements ======================= FR1. There SHALL be a new variable, @@original_server_version, which will store the server version in which each transaction was originally executed. FR2. There SHALL be a new variable, @@immediate_server_version, which will store the version of the immediate server in the replication topology for each transaction. FR3. The new variables @@original_server_version and @@immediate_server_version SHALL be replicated in the Gtid_log_event, or Anonymous_gtid_log_event if GTID_MODE=OFF. FR4. mysqlbinlog SHALL print /*!80014 SET @@SESSION.original_server_version=XXXXXX*/ and /*!80014 SET @@SESSION.immediate_server_version=XXXXXX*/ statements. FR5. If all servers in the replication topology support this feature, the value of @@original_server_version SHALL be replicated unchanged through the topology. FR6. The value of the new variables SHALL be a five to six-digit numerical representation of @@GLOBAL.version, without the suffix indicating server build or configuration information, in the original server where the transaction was first executed and on the immediate server. Example: if the @@GLOBAL.version is "8.0.14-debug", the new variable @@original_server_version SHALL be '80014'. FR7. If the server is replicating from a master that does not support these variables, it SHALL store '0' in its binary log for original_server_version. FR8. When replicating to an older server that does not support this variable, the older server SHALL be able to apply a Gtid_log_event from the newer master. FR9. In Group Replication, a View Change transaction is a special transaction issued by all members while sharing the same GTID. In order to identify the original donor's server version, each member will tag these transactions with its own server version. So, exceptionally, there may be different values of original_server_version for the same GTID only for these transactions. Non-Functional Requirements =========================== NFR1. Adding the new original_server_version and immediate_server_version fields to the Gtid_log_event SHALL NOT introduce an overhead over 5%.
Interface Specification ======================= I1. Two system variables will be implemented: NAME: original_server_version SCOPE: Session DYNAMIC: Yes PERMISSIONS: SUPER TYPE: unsigned int DEFAULT: UNDEFINED_SERVER_VERSION (999999) DESCRIPTION: For internal use of replication. Specifies the server version of the server where the transaction was originally executed. It allows the original server version to be propagated throughout the replication topology. This option is primarily intended to support replication from a 8.0.13- or older master server into an 8.0.14+ slave server, or group replication with a 8.0.13- primary node and one or more 8.0.14+ secondaries. If the transaction was originally executed by an older server without support for this variable, the variable will store 0 (UNKNOWN_SERVER_VERSION). NAME: immediate_server_version SCOPE: Session DYNAMIC: Yes PERMISSIONS: SUPER TYPE: unsigned int DEFAULT: UNDEFINED_SERVER_VERSION (999999) DESCRIPTION: For internal use of replication. Specifies the version of the immediate server where the transaction was executed. It allows the immediate server version to be propagated to the next server in the replication topology. This option is primarily intended to support replication from a 8.0.13- or older master server into an 8.0.14+ slave server, or group replication with a 8.0.13- primary node and one or more 8.0.14+ secondaries. If the immediate master is an older server without support for this variable, the variable will store 0 (UNKNOWN_SERVER_VERSION). I2. The user will not need to set the variables, as the replication infrastructure will set them automatically, but the variables need to be settable so that the output of mysqlbinlog can be applied. High Level Specification ======================== H1. The Gtid_log_event will be extended to include the new original_server_version and immediate_server_version fields. These new fields will be determined when the Gtid is written to the binlog. This way we can distinguish between master, slave applier and server applying mysqlbinlog output and act accordingly. The determined values will be written to the binlog so that they can be propagated through the replication topology. H2. If original_server_version is the same as immediate_server_version, we will only write immediate_server_version to the binlog, using its highest bit to indicate this so that original_server_version will be set using the value of the retrieved immediate_server_version. H3. The mysqlbinlog client tool dumping a Gtid_log_event will print two new SET statements for each transaction, so that the variables are set by the server applying this output accordingly. Cross-Version Replication ========================= CVR1. new->old: - mysqlbinlog: old server will ignore SET statements because its version is older than 8.0.14 - replication: old server will ignore new fields because they are outside the boundaries it expects for Gtid_log_events CVR2. old->new: - mysqlbinlog: due to the absence of the SET statements, the new server will set the original_server_version to 0 (UNKNOWN_SERVER_VERSION), indicating that the original server does not support this feature and will set immediate_server_version to its own server version, as expected. - replication: by not reading it from the Gtid_log_event, the new server will set both values to 0.
L1. Add two new fields original_server_version and immediate_server_version to gtid_log_event. Since they are sent from master to slave, the variables will be added to the encoding/decoding functions. When GTID_MODE = OFF, there is no Gtid_log_event, so we store original_server_version and immediate_server_version in Anonymous_gtid_log_event. L2. [anonymous_]gtid_log_event will be extended by 4 to 8 bytes to accommodate the new fields. L3. Since we are using 4 bytes to store the server versions and their highest value is 999999, we can safely use the highest bit of immediate_server_version to indicate whether original_server_version is the same as immediate_server_version or not, and send only immediate_server_version if they are the same. If the original_server_version is equal to immediate_server_version, the highest bit of immediate_server_version will be set to 0 and we will write to the binlog only immediate_server_version, saving 4 bytes per [anonymous_]gtid_log_event. On the other hand, if original_server_version is different from immediate_server_version, the highest bit of immediate_server_version will be set to 1 and we will write to the binlog both values. L4. Both in Gtid_log_event and Anonymous_gtid_log_events, mysqlbinlog will write /*!80014 SET @@SESSION.original_server_version=XXXXXX*/ and /*!80014 SET @@SESSION.immediate_server_version=XXXXXX*/. L5. The heuristic to compute the original_server_version is the following: - When the original_server_version session variable is set to a value other than UNDEFINED_SERVER_VERSION (999999), it means that either the original server version is known ( != 0 ) or the server version is not known ( == 0 ). - When original_server_version == UNDEFINED_SERVER_VERSION, we assume that: a) if this thread is a slave applier or mysqlbinlog applier, then the server version is unknown and the original server did not support this feature ( = 0 ); b) else, this is considered a new transaction, original on this server ( = @@GLOBAL.version[minus suffix]);
Copyright (c) 2000, 2019, Oracle Corporation and/or its affiliates. All rights reserved.