Description

X Plugin monitors not authenticated connections for their time of living. User is dropped after the time reaches the maximum allowed time. Authenticated connections are not monitored. When peer application is going to hang after authentication (doesn't generate queries) then it is going to take server resources forever. If fault application leaves a connection then after some time the number of maximum allowed connections could be reached easily ('mysqlx_max_connections' or 'max_connections'). It could have serious consequences for other users, like blocking access for new connections.

To secure the MySQL Servers and X Plugins resources the plugin needs to monitor time of all I/O operations and drop idle connections.

Notes

Same timeouts for MySQL Server:

• net_read_timeout
• net_write_timeout
• interactive_timeout
• wait_timeout
• Aborted_clients

Functional requirements

1. User should be able to set all Mysqlx io-timeouts system variables from MySQLs Server configuration file
2. User should be able to set all Mysqlx io-timeouts system variables from command line arguments passed to MySQL Server
3. Any user should be able to set all Mysqlx io-timeouts session system variables
4. User without super privilege or SYSTEM_VARIABLES_ADMIN privilege, shouldn't be able to set any of Mysqlx io-timeouts global system variables
5. User should be able to get values of all Mysqlx io-timeouts global system variables
6. Client should be able to inform the server that the session is interactive one
7. Connection must be dropped after "mysqlx_wait_timeout" time on noninteractive client when client doesn't send the message header
8. Connection must be dropped after "mysqlx_interactive_timeout" time on interactive client when client doesn't send the message header
9. Connection must be dropped after "mysqlx_read_timeout" time when client doesn't send the message payload
10. Connection must be dropped after "mysqlx_write_timeout" time when the send data weren't stored in TCP stack
11. Connection must be dropped after 'min(mysqlx_wait_timeout, mysqlx_connection_timeout)' time on non-authenticated connection when client doesn't send the message header
12. Connection must be droppped after 'min(mysqlx_read_timeout, mysqlx_connection_timeout)' time on non-authenticated connection when client doesn't send the message payload

Non-functional requirements

1. X Plugin performance should be at the same level
2. X Plugin percent coverage shouldn't be less then before the feature

Protocol

Each message in X Protocol begins with a header followed by protobuf encoded message. Currently the plugin assumes that the user is active after reception of the header. This is visible in performance schema in table "socket_instances", column "STATE". Current worklog should use same definition of active connection and differentiate timeout variables base on this.

I/O timeouts

There are three different scenarios which must be "secured":

1. X Plugin tries to send data to client. If the write operation isn't going to end in time defined by "mysqlx_write_timeout" variable (look at configuration section) then the connection must be aborted.
2. X Plugin tries to read message header from client and it isn't going to end in time defined by "mysqlx_wait_timeout" then the connection must be aborted.
3. X Plugin tries to read message payload from client and it isn't going to end in time defined by "mysqlx_read_timeout" then the connection must be aborted.

When aborting after read failure, the plugin needs to send an error message to the client. The "Mysqlx.Error" message can't be used because it is synchronous, it goes as an response in a concrete flow. X Protocol defines global notices which are asynchronous and they may be send at any moment. Before aborting the plugin must transmit a global notice with "frame" that contains a "Mysqlx.Notice.Warning" message with "level" field set to "ERROR", "code" field set to "ER_IO_READ_ERROR", "message" field set to information about the timeout. In case of write failure the plugin can't transmit anything other to client, notice isn't send in this case.

When X Plugin is going to send a notice, the information isn't guarantied to reach the client. It because of an issue with windows VIO implementation which doesn't takes into account the behavior of TCP stack. Windows TCP stack requires that the TCP buffers are empty when calling "close" on the socket. In case when TCP buffers contain data, "close" is going to send "RST" packet, breaking the connection (notice isn't delivered).

Time of living of unauthenticated client is already monitored and can't exceed "mysqlx_connection_timeout". X Plugin while doing I/O should get the timeout value from one of cases described above and compare it with "mysqlx_connection_timeout" to get lowest value. The chosen value must be considered as the timeout for the I/O operation. The time that client has for authentication is still the same.

Capabilities

"session.mysqlx_wait_timeout" must be used for every connection that is active. In case of non-interactive client the default value of "session.mysqlx_wait_timeout" is copied from "global.mysqlx_wait_timeout" variable, in other case it's copied from "global.mysqlx_interactive_timeout". To distinguish interactive and non-interactive client a new capability must be introduced. The new capabilities name should be "client.interactive" and it must accept "boolean", "integer", "float" values which must be interpreted as "boolean" value.

client -> server: Mysqlx.Connection.CapabilitySet(["client.interactive"=true])
client <- server: Mysqlx.Ok | Mysqlx.Error

Setting this capability on old MySQL Servers (X Plugins) is going to return an error because it doesn't exist there. If this capability is optional for the client then by ignoring the error, it accepts that X Plugin uses unlimited timeout values.

Configuration

All timeout values must be configurable using plugin system variables. X Plugin must support following:

• mysqlx_wait_timeout

Number or seconds that X Plugin must wait for activity on a connection. After this time the read-operation/connection is going to be aborted. If the client is non-interactive, the initial value of "session" variable is copied from "global.mysqlx_wait_timeout". For interactive client the initial value is copied from "global.mysqlx_interactive_timeout".

• mysqlx_interactive_timeout

Default value for "session.mysqlx_wait_timeout" for interactive client.

• mysqlx_read_timeout

Number or seconds that X Plugin must wait for blocking read operation to complete. After this time the operation/connection is going to be aborted.

• mysqlx_write_timeout

Number or seconds that X Plugin must wait for blocking write operation to complete. After this time the operation/connection is going to be aborted.

Error and Warnings

All errors generated by timeouts must be transmitted in global notice, in following flow:

client <- server: Mysqlx.Notice.Frame

Aborted read operations are going to send following error:

Instrumentation

Transmission of local notice containing a "Mysqlx.Notice.Warning" increments "Mysqlx_notice_warning_sent" status variable. The same status variable must be increment in case of global notice that contain "Mysqlx.Notice.Warning" message.

• Mysqlx_notice_warning_sent

Number of local and global warning notices sent back to client.

• Mysqlx_aborted_clients

Number of clients that were disconnected because of IO timeout.

VIO library already supports timeouts. After setting the "timeout" all I/O operations are switched from blocking "read/write" to nonblocking with additional call to "select".

The implementation can be summarized in several steps:

• Add plugin system variables
• Add storage for those variables in "xpl::Plugin_system_variables"
• Add capability "client.interactive"
• Add storage for the capability in "(ngs::|xpl::)Client"
• Choose a timeout variable and pass it to "vio_timeout"
• when "vio_read" fails, send a notice and disconnect
• when "vio_write" fails, disconnect