WL#11879: Replicate original server version

Affects: Server-8.0   —   Status: Complete

Executive Summary
=================
Replication topologies may be composed of servers of different versions. These
servers may have some features that may be incompatible between themselves.
However, there is, currently, no way to extract the version of neither the
original nor the immediate server version for a replicated transaction, which
creates a lot of difficulties when implementing with cross-version replication
approaches.
This worklog implements two new session variables:
- original_server_version, which will be used to tag each transaction with the
version of the server where it was originally executed, more specifically, with
the value of @@GLOBAL.version. This variable will be replicated, unchanged, to
the servers in the replication topology.
- immediate_server_version, used to tag each transaction with the version of the
immediate server in the replication topology.

Developer Stories
=================
As a developer, I want to unequivocally know the original and immediate server
version of each transaction so that I can implement adequate cross-version
replication strategies without having to introduce new specific compatibility
options.

As a developer, I want to be able to know the version of the original server
where a transaction was executed so I can correctly execute replicated
statements which had semantic changes between versions. For instance, the
default collation for utf8mb4 changed between 5.7 and 8.0, and the developer
could have used original_server_version to determine that the replicated
statement was originated in a 5.7 server and should be applied with the 5.7
default.

As a developer, I want to be able to know the version of the immediate server in
the replication topology so I know how to apply statements that changed syntax
between versions. For instance, if 8.0 had changed the CREATE TABLE syntax from
'CREATE TABLE pet (name VARCHAR(20))' to
'CREATE TABLE NAME (pet) COLUMNS (name VARCHAR(20))', the 8.0 server would be
able to rely on immediate_server_version to correctly apply the old syntax.

User Story
============
As a user, I want to be able to check which are the MySQL versions where the
transactions in my replication topology were originally executed to facilitate
auditing and debugging possible replication issues.

Glossary
========
Original server: the server where a transaction was originally executed.
Immediate server: the immediate master in a replication topology. Considering
                  the following replication topology A->B->C, for transactions
                  originally executed by A and replicated to C, we can say that
                  A has no immediate master because it is the transactions'
                  original master, B's immediate master is A, and C's immediate
                  master is B.
Functional Requirements
=======================

FR1. There SHALL be a new variable, @@original_server_version, which will store
the server version in which each transaction was originally executed.

FR2. There SHALL be a new variable, @@immediate_server_version, which will store
the version of the immediate server in the replication topology for each
transaction.

FR3. The new variables @@original_server_version and @@immediate_server_version
SHALL be replicated in the Gtid_log_event, or Anonymous_gtid_log_event if
GTID_MODE=OFF.

FR4. mysqlbinlog SHALL print
/*!80014 SET @@SESSION.original_server_version=XXXXXX*/ and
/*!80014 SET @@SESSION.immediate_server_version=XXXXXX*/ statements.

FR5. If all servers in the replication topology support this feature, the value
of @@original_server_version SHALL be replicated unchanged through the topology.

FR6. The value of the new variables SHALL be a five to six-digit numerical
representation of @@GLOBAL.version, without the suffix indicating server build
or configuration information, in the original server where the transaction was
first executed and on the immediate server. Example: if the @@GLOBAL.version is
"8.0.14-debug", the new variable @@original_server_version SHALL be '80014'.

FR7. If the server is replicating from a master that does not support these
variables, it SHALL store '0' in its binary log for original_server_version.

FR8. When replicating to an older server that does not support this variable,
the older server SHALL be able to apply a Gtid_log_event from the newer master.

FR9. In Group Replication, a View Change transaction is a special transaction
issued by all members while sharing the same GTID. In order to identify the
original donor's server version, each member will tag these transactions with
its own server version. So, exceptionally, there may be different values of
original_server_version for the same GTID only for these transactions.

Non-Functional Requirements
===========================

NFR1. Adding the new original_server_version and immediate_server_version fields
to the Gtid_log_event SHALL NOT introduce an overhead over 5%.
Interface Specification
=======================
I1. Two system variables will be implemented:
NAME: original_server_version
SCOPE: Session
DYNAMIC: Yes
PERMISSIONS: SUPER
TYPE: unsigned int
DEFAULT: UNDEFINED_SERVER_VERSION (999999)
DESCRIPTION: For internal use of replication. Specifies the server version of
the server where the transaction was originally executed. It allows the original
server version to be propagated throughout the replication topology. This option
is primarily intended to support replication from a 8.0.13- or older master
server into an 8.0.14+ slave server, or group replication with a 8.0.13- primary
node and one or more 8.0.14+ secondaries. If the transaction was originally
executed by an older server without support for this variable, the variable will
store 0 (UNKNOWN_SERVER_VERSION).

NAME: immediate_server_version
SCOPE: Session
DYNAMIC: Yes
PERMISSIONS: SUPER
TYPE: unsigned int
DEFAULT: UNDEFINED_SERVER_VERSION (999999)
DESCRIPTION: For internal use of replication. Specifies the version of the
immediate server where the transaction was executed. It allows the immediate
server version to be propagated to the next server in the replication topology.
This option is primarily intended to support replication from a 8.0.13- or older
master server into an 8.0.14+ slave server, or group replication with a 8.0.13-
primary node and one or more 8.0.14+ secondaries. If the immediate master is an
older server without support for this variable, the variable will store
0 (UNKNOWN_SERVER_VERSION).

I2. The user will not need to set the variables, as the replication
infrastructure will set them automatically, but the variables need to be
settable so that the output of mysqlbinlog can be applied.

High Level Specification
========================

H1. The Gtid_log_event will be extended to include the new
original_server_version and immediate_server_version fields. These new fields
will be determined when the Gtid is written to the binlog. This way we can
distinguish between master, slave applier and server applying mysqlbinlog output
and act accordingly. The determined values will be written to the binlog so that
they can be propagated through the replication topology.

H2. If original_server_version is the same as immediate_server_version, we will
only write immediate_server_version to the binlog, using its highest bit to
indicate this so that original_server_version will be set using the value of the
retrieved immediate_server_version.

H3. The mysqlbinlog client tool dumping a Gtid_log_event will print two new SET
statements for each transaction, so that the variables are set by the server
applying this output accordingly.

Cross-Version Replication
=========================
CVR1. new->old:
 - mysqlbinlog: old server will ignore SET statements because its version is
   older than 8.0.14
 - replication: old server will ignore new fields because they are outside the
   boundaries it expects for Gtid_log_events
CVR2. old->new:
 - mysqlbinlog: due to the absence of the SET statements, the new server will
   set the original_server_version to 0 (UNKNOWN_SERVER_VERSION),
   indicating that the original server does not support this feature and will
   set immediate_server_version to its own server version, as expected.
 - replication: by not reading it from the Gtid_log_event, the new server will
   set both values to 0.
L1. Add two new fields original_server_version and immediate_server_version to
gtid_log_event.
Since they are sent from master to slave, the variables will be added to the
encoding/decoding functions.
When GTID_MODE = OFF, there is no Gtid_log_event, so we store
original_server_version and immediate_server_version in Anonymous_gtid_log_event.

L2. [anonymous_]gtid_log_event will be extended by 4 to 8 bytes to accommodate
the new fields.

L3. Since we are using 4 bytes to store the server versions and their highest
value is 999999, we can safely use the highest bit of immediate_server_version
to indicate whether original_server_version is the same as
immediate_server_version or not, and send only immediate_server_version if they
are the same.
If the original_server_version is equal to immediate_server_version, the highest
bit of immediate_server_version will be set to 0 and we will write to the binlog
only immediate_server_version, saving 4 bytes per [anonymous_]gtid_log_event.
On the other hand, if original_server_version is different from
immediate_server_version, the highest bit of immediate_server_version will be
set to 1 and we will write to the binlog both values.

L4. Both in Gtid_log_event and Anonymous_gtid_log_events, mysqlbinlog will write
/*!80014 SET @@SESSION.original_server_version=XXXXXX*/ and
/*!80014 SET @@SESSION.immediate_server_version=XXXXXX*/.

L5. The heuristic to compute the original_server_version is the following:
    - When the original_server_version session variable is set to a value other
    than UNDEFINED_SERVER_VERSION (999999), it means that either the
    original server version is known ( != 0 ) or the server version is
    not known ( == 0 ).
    - When original_server_version == UNDEFINED_SERVER_VERSION, we assume that:
      a) if this thread is a slave applier or mysqlbinlog applier, then the
      server version is unknown and the original server did not support this
      feature ( = 0 );
      b) else, this is considered a new transaction, original on this server
      ( = @@GLOBAL.version[minus suffix]);