WL#12475: Protocol Changes to specify compression configuration for connections

WL#12039 which is based on Facebook's contribution (BUG#88567) uses connection attributes to indicate the compression method and the corresponding compression level for the network connection with the server. The connection attributes are user over-writable and optional, so the relevant packets should be extended to specify the compression configuration as the server in it's handshake indicate support for specification of compression library.

This WL is created to remove the connection attributes added in WL#12039 and make the above relevant changes to the protocol. It is a sub WL of WL#12039 and will adhere to all the existing requirements in WL#12039. In addition, on the server end, the connection related tables should contain the compression configuration (method and level) used by a connection.

1. It should be possible to specify type of compression method to be used for compressing data transfer between client and server.
2. MYSQL_OPT_COMPRESSION_ALGORITHMS option should be used to set compression method.
3. Valid compression method is any combination of zlib,zstd and uncompressed.
4. It should be possible to tune the compression level using MYSQL_OPT_ZSTD_COMPRESSION_LEVEL option for zstd compression algorithm.
5. If compression is not enabled then compression_algorithm status variable will be set to uncompressed and compression_level set to 0.
6. If compression method is configured as 'zlib' or 'zstd' but not compression level then default value will be set.
7. If compression method is configured as 'zstd,uncompressed' or 'zstd' and compression level is set to an invalid value then connection will be refused.
8. If compression method is configured as 'zlib,zstd,uncompressed' then connection is established with zlib compression provided server supports. If server supports only zstd then zstd compression is enabled, else connection is established without any compression.
9. For zlib default compression level should be set to 6 and for zstd it is 3.
10. For zstd valid compression level range from 1 to 22.
11. It should be possible to view kind of compression used for a session using SHOW STATUS like '%compression%';
12. It should be possible to view kind of compression used in a replication setup using performance_schema.replication_connection_configuration table.
13. It should be possible to configure server to use any set of supported compression methods using protocol_compression_algorithms variable.
14. Default value for protocol_compression_algorithms should be "zlib,zstd,uncompressed"
15. It should be possible for old clients to connect to new server even if server is configured with compression algorithms.
16. It should be possible for new clients to connect to old server even if client is configured with new compression options.
17. It should be possible to configure mysql client to use specific compression method using --compression-algorithms command line option.
18. --compression-algorithms can be set to any combination of zlib, zstd, uncompressed values.
19. It should be possible to configure mysql client to use compression level for zstd compression using --zstd-compression-level command line option.
20. Invalid value set for --zstd-compression-level will result in connection failure provided server and client support zstd compression.
21. --compression-algorithms and --zstd-compression-level options should be available for mysql, mysqltest, mysqldump, mysqladmin, mysqlcheck, mysqlimport, mysqlshow, mysqlslap, mysqlpump, mysqlbinlog clients.
22. It should be possible to configure replication slave with compression method and level using CHANGE MASTER TO statement.
23. CHANGE MASTER TO statement can now accept compression method/level with 2 new options MASTER_COMPRESSION_ALGORITHMS and MASTER_ZSTD_COMPRESSION_LEVEL.
24. MASTER_COMPRESSION_ALGORITHMS accepts any combination of zlib, zstd, uncompressed as valid values.
25. CHANGE MASTER TO statement fails if MASTER_COMPRESSION_ALGORITHMS is set to invalid value.
26. MASTER_ZSTD_COMPRESSION_LEVEL accepts values from 1 to 22.
27. It should be possible to configure compression method/level in group replication using 2 new options group-replication-recovery-compression-algorithms and group-replication-recovery-zstd-compression-level.
28. group-replication-recovery-compression-algorithms will have same behavior of MASTER_COMPRESSION_ALGORITHMS and group-replication-recovery-compression-level will have same behavior of MASTER_ZSTD_COMPRESSION_LEVEL

NON FRs: Based on compression level used time taken to transfer data between client and server can get affected. If higher compression level is used more time will be spent in compressing the data thus resulting in delay of data transfers.

New mysql options:

MYSQL_OPT_COMPRESSION_ALGORITHMS (argument type: char *)
This option represents the type of compression algorithm to be used to
compress network packets sent between client and server.

MYSQL_OPT_ZSTD_COMPRESSION_LEVEL (argument type: unsigned int *)
This option represents the extent to which compression should be done on the
given buffer.

If compression method is set to "zlib" then CLIENT_COMPRESS capability flag
will be enabled else if set to "zstd" then new capability flag
CLIENT_ZSTD_COMPRESSION_ALGORITHM will be enabled.

Any invalid value set to compression level will result into a connection 
failure. If client is configured with both the compression algorithms, then 
connection is enabled with compression based on negotiated algorithm provided
server supports at least one of them.

For zlib compression method, the default compression level will be set to 6
and for zstd it is 3. Valid compression levels for zstd is between 1 to 22 
inclusive.

New command line options for client

--compression-algorithms
This command line option can be used to configure compression method. Valid 
values are any combination of zlib, zstd, uncompressed.
Presence of value 'uncompressed' will let client fallback to normal connection 
without compression enabled.
If value is set to 'zstd' or 'zlib' then connection is established with 
compression enabled if both client/server support it, else connection is 
refused.
If value is set to 'zstd,uncompressed' or 'zlib,uncompressed' then connection
is established with compression enabled if both client/server support it, else
connection is still established with no compression enabled provided server is 
configured with 'uncompressed' value.
If value is set to 'zstd,zlib,uncompressed' then connection is established based
on following preference order: 
1. If server supports zlib then enable zlib compression.
2. Else if server supports zstd then enable zstd compression.
3. Else if server supports uncompressed then connect without compression.

--zstd-compression-level
This command line option can be used to configure compression level for zstd
compression algorithm. Valid value are between 1 and 22 inclusive.

These command line options will be introduced for mysql, mysqltest, mysqldump, 
mysqladmin, mysqlcheck, mysqlimport, mysqlshow, mysqlslap, mysqlbinlog clients.

Note: When --compression-algorithms is set without --compress option then 
protocol is still enabled with compression.

Protocol changes:

Initial handshake packet (Protocol::HandshakeV10) from server will have a 
capability flag with CLIENT_ZSTD_COMPRESSION_ALGORITHM when global variable 
protocol-compression-algorithms contains zstd.

If client is configured with the new option MYSQL_OPT_COMPRESSION_ALGORITHMS as 
"zstd" and MYSQL_OPT_ZSTD_COMPRESSION_LEVEL, then client will send compression 
level in its response packet (Protocol::HandshakeResponse41). Below is the
additional payload to be added to Protocol::HandshakeResponse41 packet.

if capabilities & CLIENT_ZSTD_COMPRESSION_ALGORITHM {
  1              compression level
}

Server reads client response and checks the compression level. If its a valid 
value then connection will proceed and further data packets will be compressed 
using negotiated algorithm, else connection is declined.

Note: This extra bytes of information used for negotiation of compression
method will be sent in handshake packet if and only if
CLIENT_ZSTD_COMPRESSION_ALGORITHM capability flag is enabled on both client and
server side.

Backward Compatibility:

A new capability flag CLIENT_ZSTD_COMPRESSION_ALGORITHM will be introduced to 
ensure backward compatibility. On client side this capability flag is set when
MYSQL_OPT_COMPRESSION_ALGORITHMS option is set to "zstd".

New system variables:

This variable accepts compression algorithm name. Server can support any number 
of compression algorithms. This variable can be configured to set particular 
algorithm. If in future server supports many compression algorithms, then user 
can set this variable to an algorithm of his choice.

Value 'uncompressed' has a special meaning to this variable. This value means 
connection can be configured with compression enabled or disabled. If this value 
is not set in this variable then connection is expected to be compression 
enabled only. If client connects without --compression-algorithms or --compress
then connection is refused.

If this variable value is set to "zstd" then CLIENT_ZSTD_COMPRESSION_ALGORITHM 
capability flag will be enabled at server side.

Below table describes behavior of what compression algorithm/level is negotiated 
between server and client based on compression configurations:

NOTE: 
1. fail - means connection is refused
2. invalid level is checked only in case where both client and server support
zstd compression in combination with uncompressed only. In rest of the cases 
level is ignored.

Below table describes what compression algorithm is established between client and server based on existing --compress option.

Replication slave configuration

CHANGE MASTER TO syntax will be extended to accompany these new compression 
specific attributes. New extended syntax will be as below:

CHANGE MASTER TO option [, option] ... [ channel_option ]
option:
    MASTER_BIND = 'interface_name'
  | MASTER_HOST = 'host_name'
  | ...
  | IGNORE_SERVER_IDS = (server_id_list)
  | MASTER_COMPRESSION_ALGORITHMS = "String containing a comma-separated list 
consisting of one, two, or three strings from the set zlib, zstd, uncompressed,
in any order."
  | MASTER_ZSTD_COMPRESSION_LEVEL = [1-22] (integer specifying the compression 
ratio in case zstd is used)

MASTER_COMPRESSION_ALGORITHMS specifies what compression method to be used for 
slave/master protocol, provided both the slave and the master support it.
Valid values are any combination of zlib or zstd or uncompressed. 
START SLAVE will decide which compression method to use based on 
@@global.slave_compressed_protocol: if slave_compressed_protocol=1, it uses zlib 
with compression level set to 6 else use compression algorithm/level specified
as part of CHANGE MASTER statement.

Note: slave-compressed-protocol is an existing global variable.
CHANGE MASTER TO fails in following cases:
1. MASTER_COMPRESSION_ALGORITHMS is set to invalid value. That is any value other
than zlib,zstd,uncompressed.
2. MASTER_ZSTD_COMPRESSION_LEVEL is set to value < 1 or > 22.

During upgrade, the value for 
MASTER_COMPRESSION_ALGORITHMS/MASTER_ZSTD_COMPRESSION_LEVEL in an existing 
replication channel will be set to: uncompressed/3

When a new replication channel is created, the default for 
MASTER_COMPRESSION_ALGORITHMS/MASTER_ZSTD_COMPRESSION_LEVEL is: uncompressed/3

NOTE: If user never specifies MASTER_COMPRESSION_ALGORITHMS or 
MASTER_ZSTD_COMPRESSION_LEVEL, the channel will use uncompressed/3.

Master info repository

Master info repository which can be mysql.slave_master_info or master.info file 
will be populated with these new compression attributes.
During server startup, wrong values in master info repository are handled as 
follows:
1. MASTER_COMPRESSION_ALGORITHMS is invalid: reset it to uncompressed
2. MASTER_ZSTD_COMPRESSION_LEVEL is out of range for reset it to 3
if either of the above cases happens, a warning shall be reported in server log.

If master-info-repository=FILE then MASTER_COMPRESSION_ALGORITHMS=uncompressed 
and MASTER_ZSTD_COMPRESSION_LEVEL=3 are represented as string, in the same 
format as accepted by CHANGE MASTER.

Below new columns will be added to mysql.slave_master_info table
* Master_compression_algorithm CHAR(64) CHARACTER SET utf8 COLLATE utf8_bin NOT 
NULL COMMENT 'Compression algorithm used for data transfer between master and 
slave.'
* Master_zstd_compression_level INTEGER UNSIGNED NOT NULL COMMENT 'Compression 
level associated with zstd compression algorithm.'

Below new columns will be added to 
performance_schema.replication_connection_configuration table
* COMPRESSION_ALGORITHM CHAR(64) collate utf8mb4_bin not null COMMENT 
 'Compression algorithm used for data transfer between master and slave'
* ZSTD_COMPRESSION_LEVEL INTEGER not null COMMENT 'Compression level associated 
  with zstd compression algorithm'

Group Replication configuration

Two new options will be introduced to support configuration of compression 
method and level. Below is the details:

New status variables:

New status variables compression_algorithm and compression_level will be
introduced to track the negotiated algorithm between client and server.
These variables scope will be session only.

Existing option MYSQL_OPT_COMPRESS is no where effected with this WL.
Note: Configuring compression method and level will not be provided for 
federated storage engine.