To authenticate a user against the server the client server protocol employs one of several authentication methods.

As of MySQL 5.5 the authentication method to be used to authenticate connections to a particular MySQL account is indicated in the mysql.user table. For earlier servers it's always mysql native authentication or old password authentication depending on the CLIENT_SECURE_CONNECTION flag.

Client and server negotiate what types of authentication they support as part of the Connection Phase and Determining Authentication Method.

Each authentication method consists of a client plugin name a server plugin name a specific exchange

The exchanged input and output data may either be sent as part of the Protocol::Handshake and the Protocol::HandshakeResponse: or as a part of the Protocol::AuthSwitchRequest: and following packets. The structure is usually the same.

Limitations

While the overall exchange of data is free-form there are some limitations in the initial handshake of the amount of data that can be exchanged without causing an extra round trip:

The auth_plugin_data field in Protocol::Handshake packet can only carry 255 bytes max (see CLIENT_SECURE_CONNECTION).
The auth_reponse_data field in Protocol::HandshakeResponse: packet can only carry 255 bytes max too if CLIENT_PLUGIN_AUTH_LENENC_CLIENT_DATA is not set.
The client-side plugin may not receive its initial data in the initial handshake

Old Password Authentication

Authentication::Old:

The server name is mysql_old_password
The client name is mysql_old_password
Client side requires an 8-byte random challenge from server
Client side sends a 8 byte response packet based on a proprietary algorithm.

Note: If the server announces Native Authentication in the Protocol::Handshake packet the client may use the first 8 bytes of its 20-byte auth_plugin_data as input.

Warning: The hashing algorithm used for this auth method is broken as shown in CVE-2000-0981.

Native Authentication Caching_sha2_password information Clear text client plugin Windows Native Authentication authentication_fido information