4.7.2 S3-compatible Storage

MySQL Shell S3 support has been tested against the following S3-compatible storage services:

• Amazon Web Services S3

  See Amazon Simple Storage Service Documentation for more information.

• Oracle Cloud Infrastructure Object Storage

  See Amazon S3 Compatibility API for more information.

MySQL Shell supports configuring AWS credentials in environment variables and in configuration files.

• Configuration Parameter Precedence

• Environment Variables

• Configuration Files

• Connection Retry Strategy

Configuration Parameter Precedence

Configuration parameters are used in order of precedence:

1. Option. For example, s3Profile takes precedence over the environment variable AWS_PROFILE, and the default profile in the configuration file.

2. Environment variable. For example, the environment variable AWS_SHARED_CREDENTIALS_FILE takes precedence over the default location of the credentials file, ~/.aws/credentials.

3. DEFAULT environment variable, if it exists. For example, the environment variable AWS_REGION and the environment variable AWS_DEFAULT_REGION.

4. Parameter defined in a configuration file.

5. AWS default values.

For example, for the AWS region, in order of precedence:

1. s3Region option.

2. AWS_REGION environment variable.

3. AWS_DEFAULT_REGION environment variable.

4. Region defined in the configuration file.

5. The default value of us-east-1.

AWS credentials are read in the following order of precedence:

1. Environment variables, if the s3Profile option is not defined.

2. Credentials file, for the specified profile.

3. credential_process, as defined in the config file. This configuration type contains an external command which retrieves or generates AWS authentication credentials and writes them to stdout.

4. Config file, for the specified profile.

Environment Variables

For information on AWS environment variables, see Environment variables to configure the AWS CLI

You can define the following AWS S3-specific environment variables:

• AWS_PROFILE: Specifies the name of the profile to use. This can be the name of a profile in a credentials or config file, or the value default to use the default profile. This environment variable overrides the [default] profile named in the configuration file. You can override this environment variable with the --s3Profile option.

• AWS_SHARED_CREDENTIALS_FILE: The location of the file used to store access keys. Such as ~/.aws/credentials.

• AWS_CONFIG_FILE: The location of the file used to store configuration profiles. Such as ~/.aws/config.

• AWS_REGION: Specifies the AWS Region to send the request to. This value overrides the AWS_DEFAULT_REGION environment variable and the profile defined in the configuration file.

• AWS_DEFAULT_REGION: Specifies the AWS Region to send the request to. This value is overridden by the --s3Region option and the AWS_REGION environment variable, if specified.

• AWS_ACCESS_KEY_ID: Specifies an AWS access key associated with an IAM user or role.

• AWS_SECRET_ACCESS_KEY: Specifies the secret key associated with the access key. This variable overrides the aws_secret_access_key defined in the profile.

• AWS_SESSION_TOKEN: Specifies the session token value required if you are using temporary security credentials. This variable overrides the aws_session_token defined in the profile.

Configuration Files

MySQL Shell requires the following parameters in one or more configuration files:

• aws_access_key_id: specifies the access key associated with the user. If not present, an exception is thrown.

• aws_secret_access_key: specifies the secret key associated with the access key. If not present, an exception is thrown.

• aws_session_token: specifies the session token required if you are using temporary security credentials. If not present, it is not used to authenticate the user.

• region: specifies the cloud service region. If not present, the default value of us-east-1 is used.

• credential_process:This configuration type contains an external command which retrieves or generates AWS authentication credentials and writes them to stdout.

  Note

  credential_process can only be defined in the config file.

  For more information, see the following:

  • AWS Credentials Process

  • AWS Configuration and Credential Options

  • AWS Sourcing credentials with an external process

If you have installed and configured the AWS CLI, MySQL Shell reads the connection parameters from the default location, .aws/, automatically. To use an alternate configuration, do so from the command line, using the relevant override options.

If you are using Oracle Cloud Infrastructure's S3 compatibility API, see Creating a Customer Secret Key for information on creating the credentials required.

Note

By default, the AWS CLI creates two configuration files, config, which stores parameters such as region and output format, and credentials, which stores access keys and session tokens. It is also possible to place all configuration parameters in a single file, config. For more information, see AWS Configuration and authentication settings . However, if you have defined access key and secret access keys in both files, those defined in the credentials file take precedence.

The following example shows a default pair of AWS CLI configuration files:

        /home/.aws/credentials
        
        [default]
        aws_access_key_id = AKIA6AV......
        aws_secret_access_key = XHRY579I.....
        
        /home/.aws/config
        
        [default]
        region = us-west-1
        output = json

Connection Retry Strategy

All failed connections to AWS S3 are retried three times, with a 1 second delay between retries.

If a failure occurs 10 minutes after the connection was created, the delay is changed to an exponential back-off strategy:

• First delay: 3-6 seconds

• Second delay: 18-36 seconds

• Third delay: 40-80 seconds