WL#9252: X Protocol connection compression

Affects: Server-8.0   —   Status: Complete


Latency of requests on networks depends on:

  • networks RTT
  • utilization of the network

While the networks RTT can't be changed, the utilization of the network can be changed by compressing the packets to reduce the traffic.

In context of the xprotocol most of the data that's transferred are:

  • statement text (strings)
  • resultset data (strings, numbers, ...)

which both should be nicely compressible.

CPU latency vs. network latency trade-off

Compression uses CPU time to encode the data and to decode the data. Sending less data over the network allows to reduce network utilization and therefore the extra latency added due to overloaded network.

Compression algorithms

While zlib/deflate have been the de-facto compression algorithm in network protocols, in the recent years new algorithms where developed that compress faster, compress better and/or are more CPU efficient. To support add new algorithms over time, the server should announce which algorithms it can understand.

Clients may support zero or more of the servers compression algorithms and should select one that fits the users requirements best.

make compression algorithm negotiable

Impact on middleboxes

(middleboxes like MySQL Router, ...)

Middleboxes like Routers/Load-Balancers need to track the state of the protocol flow to make decisions to which backend should be sent and when the connection enters a bad state. That's done by tracking the message headers and error messages and notices.

If a middlebox must decompress all messages to be able to track state they may get CPU bound and overloaded and slow down the communication between client and server.

middleboxes MUST be able to read message headers without noticeable overhead if the data is compressed.

Security Considerations

To prevent DoS each side MUST apply limits on the uncompressed payload before decompression.

Functional requirements

a client MUST be able to query the compression algorithms a server supports
  1. an admin MUST be able to limit servers supported compression algorithms

  2. a client MUST be able to choose one compression algorithm of the supported algorithms in capability negotiation phase (excluding reseted sessions)

  3. server MUST enable compression after successful authentication in case when compression algorithm was set successful

  4. a server MUST return an error when client tries to set unsupported compression algorithm

  5. a server MUST allow multiple changes to compression algorithm in capability negotiation phase (last change is used)

a client MUST be able to control how and if uncompressed messages may be combined into a compressed message.
a client MUST be able to enable compression on connection after successful authentication in case when compression algorithm was set successful
  1. client and server MUST manage a client-incoming compression context

  2. client and server MUST manage a client-outgoing compression context

  3. a server MUST report an error and disconnect after decompression error

  4. a client MUST disconnect after it detect decompression error

  5. a server MUST decompress each compressed message to keep compression context in sync on both sides even if expectation block in X Protocol is skipped

  6. a server MAY consider to compress following X Protocol messages ColumnMetaData, Row, FetchDone, FetchSuspended, FetchMoreResultssets, FetchDoneMoreOutParams, Notice

a client MUST be able to query compression statistics
a client MUST be able to try to enable compression and accept a connection when server doesn't support compression.



Compression needs to be negotiated between client and server to agree:

  • compression algorithm
  • combining of messages
  • max uncompressed messages per compressed message

Compression must be configured before session setup, at capability negotiation phase and is enabled after successful authentication.

User may check which compression algorithms are available by sending:

client->server: Mysqlx.Connection.CapabilitiesGet()
client<-server: Mysqlx.Connection.Capabilities(....)

To enable compression a client must set the compression configuration:

client->server: Mysqlx.Connection.CapabilitiesSet(...new configuration...)
client<-server: Mysqlx.Ok()


  • algorithm
  • server_combine_mixed_messages
  • server_max_combine_messages

algorithm (GetCapability)

names of supported compression algorithms

algorithm (SetCapability)

compression algorithm to use after session is estabilished.

server_combine_mixed_messages (SetCapability)

if true, server is allowed to combine different message types into a compressed message

server_max_combine_messages (SetCapability)

if set, the server MUST not store more than N uncompressed messages into a compressed message
no limit

Example Message Flow

client->server: Mysqlx.Connection.CapabilitiesGet()
client<-server: Mysqlx.Connection.Capabilities(
    "compression":{"algorithm": [ "deflate_stream", "lz4_message"]}, ...)

Clients can ask the server to enable compression by using "CapabilitiesSet" with key "compression.algorithm" and a string value.

Connection.CapabilitiesSet({ "compression":{"algorithm": "deflate_stream",...}, ...)
returns Ok and use "Compression Algorithm" for Compressed message after authentication or Error

If a client does not set a "compression":{"algorithm":...}, then compression neither client nor server will use Compressed messages.

Compression options can be changed multiple times, where last data from the last Mysqlx.Connection.CapabilitiesSet will be used:

client->server: Mysqlx.Connection.CapabilitiesSet("compression":{"algorithm": "deflate_stream",..})
client<-server: Mysqlx.Ok
client->server: Mysqlx.Connection.CapabilitiesSet("compression":{"algorithm": "lz4_message",..})
client<-server: Mysqlx.Ok

Compressed Message

Once compression has been negotiated client and server can send compressed messages.

Compressed messages are announced in a xproto frame as:

// mysqlx.proto
message ClientMessages {
  enum Type {
    // ...
    COMPRESSED = 46;

message ServerMessages {
  enum Type {
    // ...

    COMPRESSED = 19;

Both client and server wrap a compressed payload in a Compressed message:

// mysqlx_connection.proto
message Compressed {
  optional uint64 uncompressed_size = 1;
  optional Mysqlx.ServerMessages.Type server_messages = 2;
  optional Mysqlx.ClientMessages.Type client_messages = 3;
  required bytes payload = 4;
size of the uncompressed payload that is passed to the compressor
set by server to message-type of the uncompressed message if only a single uncompressed message type is contained in payload.
set by client to message-type of the uncompressed message if only a single uncompressed message type is contained in payload.
output of the compression algorithm.

Optimized Encoding

Protobuf doesn't enforce a order of fields in serialized message.

But, a sender of compressed message SHOULD send uncompressed size before payload to allow the receiver to implement a streaming decode where uncompressed size and the initial part of the payload are passed to the decompressor before the whole payload is received.

Example Implementation using libprotobuf with C++

void send_compressed_payload(const Mysqlx::ServerMessages::Type msg_type,     
    const int uncompressed_size, const std::string &compressed_payload) {
  std::string output;
  google::protobuf::StringOutputStream string_zero_stream(&output)
  google::protobuf::CodedOutputStream ostream(&string_zero_stream);
  Mysqlx::Compression compression_first_fields;
  Mysqlx::Compression compression_payload;

  compression_first_fields.server_messages = msg_type;
  compression_first_fields.uncompressed_size = uncompressed_size;


  // use SerializePartial to encode the "first_fields" before the "payload"  

  return output;


The compressor receives up to server_max_combine_messages xproto frames:

length   = 4 byte;
msg_type = 1 byte;
payload  = *<length>;

msg      = length + msg_type + payload;
msgs     = msg msg*;
payload  = compressor(msgs);

Compression Algorithm: lz4_message

LZ4 is a fast compression algorithm.

The payload of the Compressed message consists of:

Compression Algorithm: deflate_stream

deflate is the compression algorithm from zlib, PNG, ... and is widely available.

The payload of the Compressed message is built in following way:

The deflate's compression context is kept alive over all messages to improve compression over time.

Note: each X Protocol compressed frame must be complete. Data from the zlib layer must be flushed before serializing them to X Protocol frame.

Maximum message size

The size of messages in xprotocol is limited to value of 'Mysqlx_max_allowed_packet' plugin variable.

The value applies to both:

  • Compressed messages
  • messages after uncompression

When a single message extracted from compressed message is larger than 'Mysqlx_max_allowed_packet' then decompression must be stopped and X Plugin must return ER_X_DECOMPRESSION_FAILED error and close the connection.

Compression and message type

The server will not compress control flow messages like:

  • Mysqlx.Error
  • Mysqlx.Sql.StmtExecuteOk
  • Mysqlx.Ok.
  • global Mysqlx.Notice.Frame

to make it easier for middleware to track the protocol without having to uncompress the whole stream.

On the other side:

  • Mysqlx.Resultset.ColumnMetaData
  • Mysqlx.Resultset.Row
  • Mysqlx.Resultset.FetchDone
  • Mysqlx.Resultset.FetchSuspended
  • Mysqlx.Resultset.FetchDoneMoreResultsets
  • Mysqlx.Resultset.FetchDoneMoreOutParams
  • Mysqlx.Notice.Frame (only local notices)

MAY be compressed.


Status variables

Monitoring of compressed data by the server

  • mysqlx_bytes_sent_compressed_payload - data after compression which were send to the wire, one thing needs to be taken into concern. mysqlx_bytes_sent consist of mysqlx_bytes_sent_compressed_payload and X Protocol data that were not compressed like X Protocol headers, X Protocol uncompressed messages.

    Property Value
    Name Mysqlx_bytes_sent_compressed_payload
    Scope SESSION and GLOBAL
    Default 0
  • mysqlx_bytes_sent_uncompressed_frame - data before compression which were compressed to mysqlx_bytes_sent_compressed_payload bytes.

    Efficiency of compression algorithm can be checked by looking at the compression ratio:

    mysqlx_bytes_sent_uncompressed_frame / mysqlx_bytes_sent_compressed_payload

    Efficiency of compression on X Protocol can be checked by looking at the compression following:

    (mysqlx_bytes_sent - mysqlx_bytes_sent_compressed_payload + mysqlx_bytes_sent_uncompressed_frame) / mysqlx_bytes_sent
    Property Value
    Name mysqlx_bytes_sent_uncompressed_frame
    Scope SESSION and GLOBAL
    Default 0

Monitoring of uncompressed data by the server

  • mysqlx_bytes_received_compressed_payload - compressed data received on the wire (is a component of mysqlx_bytes_received)

    Property Value
    Name mysqlx_bytes_received_compressed_payload
    Scope SESSION and GLOBAL
    Default 0
  • mysqlx_bytes_received_uncompressed_frame - compressed data (mysqlx_bytes_received_compressed_payload) were uncompressed into mysqlx_bytes_received_uncompressed_frame bytes

    Property Value
    Name mysqlx_bytes_received_uncompressed_frame
    Scope SESSION and GLOBAL
    Default 0

Errors and Warnings

  • When client sends a compressed frame, but no compression is negotiated with the xplugin, it returns:

    client->server: Compressed
    client<-server: Error(ER_X_FRAME_COMPRESSION_DISABLED)
    Property Value
    Error code 5170
    Error text Client didn't enable%s compression.
  • When client sends compressed a frame and decompression fails in the xplugin, it returns:

    Property Value
    Error code 5171
    Error text Payload decompression failed
  • When client sends a compressed frame and X Plugin can't read a proper compression header or there are more or less data in the frame, in comparison to sizes presented in headers, it returns:

    client->server: Compressed
    client<-server: Error(ER_X_BAD_COMPRESSED_FRAME)
    Property Value
    Error code 5174
    Error text Payload decompression failed
  • When client tries to negotiate compression and uses an unsupported or disabled compression algorithm:

    client->server: CapabilitySet("compression":{"algorithm":"unsupported"...})
    Property Value
    Error code 5175
    Error text Invalid or unsupported value for 'compression.algorithm'
  • When client tries to negotiate compression and uses a unsupported option:

    client->server: CapabilitySet("compression":{"unsupported_field":"..."...})
    Property Value
    Error code 5178
    Error text Invalid or unsupported option for 'compression'
  • When client tries to negotiate compression and uses compression without required fields. Required is "compression.algorithm":

    client->server: CapabilitySet("compression":{})
    Property Value
    Error code 5179
    Error text The algorithm is required for 'compression'


  • mysqlx_compression_algorithms - using this variable, admin may limit supported compression algorithms

    client->server: CapabilitiesGet
    client->server: Capabilities(compression:{algorithm:["deflate_message", "lz4_frame"]})
    ... admin executes SET @@mysqlx_compression_algorithms="deflate_message"...
    client->server: CapabilitiesGet
    client->server: Capabilities(compression:{algorithm=["deflate_message"]})
    Property Value
    Name mysqlx_compression_algorithms
    Type SET("deflate_message", "lz4_frame")
    Scope Global
    Default 'deflate_message,lz4_frame'