Changes in MySQL Shell 8.0.11 (2018-04-19, General Availability)

• AdminAPI Added or Changed Functionality

• AdminAPI Bugs Fixed

• Functionality Added or Changed

• Bugs Fixed

AdminAPI Added or Changed Functionality

• InnoDB Cluster instances running MySQL 8.0.11 and higher can now be configured for InnoDB Cluster usage remotely, without the need to log in to the instance and run dba.configureLocalInstance() locally. Use the new dba.configureInstance() operation, which enables remote automatic configuration of compatible instances for InnoDB Cluster usage and if necessary also supports remote restart of compatible instances after configuration.

  Similarly, compatible instances support having any changes to their settings automatically persisted by AdminAPI after any cluster topology changes. Now, when working with remote instances if you create a cluster, add an instance to a cluster, or remove an instance from a cluster supporting this functionality, there is no need to log in to the instance and run dba.configureLocalInstance() to persist the changes to the instance's option file. This makes working with production clusters, which are built on networked instances, much easier. This also ensures that instances automatically rejoin a cluster in the case of a restart.

  As part of this feature the following improvements were made:

  • Improved display of messages.

  • Improved detection of loopback hostnames on Debian and Ubuntu, and local instances based on the hostname.

  • Improved display of required configuration changes.

  • Replaced automatic configuration of instances after issuing dba.createCluster and dba.addInstance with the configuration check dba.checkInstance(), aborting the command if the check fails.

  • When specifying an account in interactive mode, if the user's hostname is local, for example localhost, you are given the option to recreate the account configured for remote usage by using the hostname %, or to create a new InnoDB Cluster administrator account.

  (WL #10434, WL #11344)

  References: See also: Bug #27608299, Bug #27112727, Bug #27629803.

AdminAPI Bugs Fixed

• After creating a cluster administrative user using dba.createCluster(), MySQL Shell attempted to reconnect using the new user but still using localhost, which failed causing the whole configuration to fail. Now MySQL Shell no longer switches accounts between administrative account creation and instance configuration. (Bug #27673816)

• The changes introduced by Bug#27545850 mean that now it is not possible to modify Group Replication related options while the plugin is starting or stopping. The AdminAPI operations have been modified to ensure that they respect this change, and attempting to issue a statement which would change a Group Replication operation while the plugin is starting results in an error. There is no check for when the plugin is stopping and so you should ensure you do not attempt to configure an instance if there is a chance the plugin is stopping. (Bug #27545850)

• When rejoining an instance to a cluster the displayed output has been improved. (Bug #27437389)

• The dba.createCluster() function fails if used on an instance with innodb_page_size=4k, because the instance_name column of the instances table in the InnoDB Cluster metadata uses a VARCHAR of size 256, which is too large for the innodb_page_size=4k. However, the size of this column cannot be changed because it is used to hold hostnames of the cluster members which can have a length up to 255 bytes. This limitation is now validated when using dba.createCluster(), dba.checkInstanceConfiguration(), dba.configureInstance(), and dba.configureLocalInstance(). (Bug #27329079)

• The AdminAPI commands which manage sandboxes failed with an error when using a sandbox directory with non-ASCII characters in the path. This occurred when the user name had a non-ASCII character because the default sandbox directory is a subdirectory of the $HOME or %userprofile% path, which is usually based on the current user name. The fix adds Unicode support to the internal provision tool used by AdminAPI. (Bug #27181177)

• MySQL Shell now sets the value of group_replication_local_address based on port by calculating the result of ((port * 10) + 1). The port variable defaults to 3306, in which case MySQL Shell sets group_replication_local_address to ((3306 * 10) +1), resulting in port 33061 instead of the previous default of 13306. (Bug #27146799)

• The online help for all AdminAPI commands which require a connection displayed detailed information on connection data, which cluttered up the details of each command. Now, the help displays an overview of connection details and information about where else to get more help. (Bug #27146290)

• When creating a cluster on a Debian type platform with the default system configuration, the` cluster.addInstance() command failed with an error if the instance hostname was used in the connection parameter. This happened because on these platforms the hostname is resolved to the IP address 127.0.1.1 by default, but this IP address is not supported by the Group Replication Group Communication Service (GCS). The fix adds a validation to the dba.createCluster() and cluster.addInstance() functions to verify if the connection hostname resolves to 127.0.1.1 and issue an error in that case. (Bug #27095984)

• The Cluster.forceQuorumUsingPartitionOf() operation was not working with a non-root user, even if that user had all the required privileges. The fix ensures that the specified user is used to connect to all instances, instead of the root user. (Bug #27089930)

• Issuing dba.createCluster() in interactive mode as a user which did not have all the required privileges resulted in an unexpected halt. The error message generated when issuing dba.configureLocalInstance() and dba.checkInstanceConfiguration() if the account being used did not have the required privileges has also been improved. (Bug #27076753, Bug #27324699)

• When using adoptFromGR to convert a Group Replication group into an InnoDB Cluster, the output recommended adding instances. This message has now been improved to show that the instances were already added. (Bug #27061615)

• In interactive mode the cluster.removeInstance() AdminAPI function was not accepting the second parameter with the options dictionary. This prevented use of the force option, which failed with an error. (Bug #26986141)

  References: See also: Bug #27572618.

• The cluster.addInstance() function was not checking the validity of the server_uuid being added to the cluster. Now, the server_uuid of an instance joining the cluster is checked to be unique, and if it is not an error is generated and the instance is prevented from joining the cluster. (Bug #26962715)

• When setting up a cluster a replication user is created to enable distributed recovery. If MySQL Shell logging was enabled, the CREATE USER statements were not being correctly added to the log. (Bug #26938488)

• The dba.checkInstanceConfiguration() and dba.configureLocalInstance() operations were not correctly reporting any errors related to incorrect configuration of the instance's server_id. Now, when an error is reported it is displayed in the list of variables not meeting the InnoDB Cluster requirements.

  In addition, the X Plugin load has been removed from the my.cnf created by AdminAPI for sandbox instances running MySQL version 8.0.11 and later because X Plugin is installed by default for those versions. (Bug #26836230)

• When using dba.configureLocalInstance() with the clusterAdmin option to create a user which can administer the cluster, the created account had too many privileges. (Bug #26737608)

• When you create a cluster and add instances to it internal users are created, which are required by Group Replication for distributed recovery when instances join the cluster. These automatically generated replication users were not being removed from instances after removing them from the cluster or after dissolving the cluster. (Bug #26395608)

• If an instance contained InnoDB Cluster metadata but was stand alone, in other words it did not belong to a cluster, it was not possible to use dba.dropMetadataSchema(). (Bug #26315635)

• If you issued dba.configureLocalInstance() against an instance which was already valid for InnoDB Cluster usage and specified the myCnf option, but the my.cnf file was not readable, the operation would print that there are issues that need to be fixed. Now when you set the myCnf option, using either dba.configureInstance() and dba.configureLocalInstance(), the operations only use it if necessary. In other words, if the target instance is not valid for InnoDB Cluster usage and settings must be changed. (Bug #25702994)

• The cluster.describe() and cluster.status() AdminAPI methods return JSON objects containing the same information but some fields were identified by different tags. To make the tags consistent, the object returned from cluster.describe() has changed so that instances has been replaced with topology, and host with address. (Bug #25247515)

• Creating a cluster from an existing Group Replication group by using the adoptFromGR:true option on an instance which already belonged to cluster was not being correctly detected. Now such a situation is detected and generates an error. (Bug #25061891, Bug #25664766)

• MySQL Shell is able to create a session to an IPv6 address, but it failed to create an InnoDB Cluster using such connections, reporting an URI-related error. This was related to the encoding of the URI containing an IPv6 address, which has been fixed. The core issue, IPv6 support for AdminAPI is a known issue as Group replication requires an IPv4 network, see Group Replication Requirements. The implementation of InnoDB Cluster related operations has been modified in order to check if operations are executed using an IPv4 based session and IPv4 connection data. An exception is generated if these conditions are not met. (Bug #25042407)

• Attempting to add two different instances with the same label to an InnoDB Cluster correctly resulted in an error, however the instance with the duplicated label was being left with a partially initialized configuration. (Bug #24761416)

Functionality Added or Changed

• Important Change; Microsoft Windows: Before installing MySQL Shell, make sure you have the Visual C++ Redistributable for Visual Studio 2015 (available at the Microsoft Download Center) installed on your Windows system. This now applies for both Community and Commercial versions of MySQL Shell.

• Important Change; X DevAPI: The way which document IDs are generated has been changed. Now document IDs are generated by the server, rather than by the client. As a result the getLastDocumentID(), getDocumentId and getDocumentIDs() methods have been removed. To get a list of the document IDs automatically generated by an 8.0.11 and later server, use Result.getGeneratedIDs(). The type of column generated for the document ID in a collection has changed from VARCHAR(32) to VARBINARY(32). The generated document ID can be overridden manually by including an ID but you must respect the server generated IDs to avoid conflicts. If you are using InnoDB Cluster, use the mysqlx_document_id_unique_prefix variable to ensure your documents can be moved between ReplicaSets.

  Now, if you are adding documents to a collection on a server running a version of MySQL earlier than 8.0.11, you must manually include a document ID as these versions do not add the ID automatically. (WL #10955, WL #11390)

• X DevAPI: Support for the NOWAIT and SKIP LOCKED InnoDB locking modes has been added to the lock operations. Now you can use these locking modes with the lockShared() and lockExclusive() methods, for example:

  • Table.select().lockShared([LockContention])

  • Table.select().lockExclusive([LockContention])

  • Collection.find().lockExclusive([LockContention])

  • Collection.find().lockExclusive([LockContention])

  where LockContention can be one of:

  • DEFAULT - if the function encounters a row lock it waits until there is no lock

  • NOWAIT - if the function encounters a row lock it aborts and generates an ER_LOCK_NOWAIT error

  • SKIP_LOCKED - if the function encounters a row lock it skips the row and continues

  For more information see Locking Read Concurrency with NOWAIT and SKIP LOCKED. (WL #11327, WL #11417, WL #11242)

• The indexing of collections has been improved to make large collections of documents more efficient to navigate. Now, you can create an index based on one or more fields found in the documents in the collection, using a JSON document which maps fields from the collection's documents to MySQL types. The majority of MySQL types are supported, in addition to spatial indexes and GeoJSON data. (WL #9876, WL #10858)

• MySQL Shell can now connect to a MySQL server with an account that uses the caching_sha2_password authentication plugin. Assuming the server is configured for encrypted connections, you can use such accounts over both X Protocol and the classic MySQL protocol. See Using Encrypted Connections.

  Important

  If you are not using encrypted connections, to connect over X Protocol with an account that uses the caching_sha2_password authentication plugin, the user's password must be stored in the cache. Currently there is no way to store the password over X Protocol if not using encrypted connections.

  When using the classic MySQL protocol for connections with such an account and unencrypted connections, you can configure MySQL for password exchange using an RSA key pair. MySQL Shell supports such connections and the following command options have been added:

  • Use the --server-public-key-path option to specify the RSA public key file.

  • Use the --get-server-public-key option to request the public key from the server.

  For more information, see Caching SHA-2 Pluggable Authentication. (WL #11591, WL #11555)

• MySQL Shell now has a configuration file which stores configuration changes across sessions. Use the new \option MySQL Shell command for querying and changing configuration options. Alternatively use the following methods with the shell.options object:

  shell.options.set_persist(optionName, value)
  shell.options.unset_persist(optionName, value)

  In addition the new defaultMode option has been added, which enables you to configure the programming language which MySQL Shell starts up in. You can override the default mode using the command options. (WL #10783)

• The util.checkForServerUpgrade([uri]) operation has been extended to check for the following incompatible features:

  • obsolete sql_mode

  • partitioned tables in shared tablespaces

  • removed functions

  (WL #11622)

Bugs Fixed

• X DevAPI: The handling of SQL wildcard characters was corrected for the X DevAPI Schema.getTable(), Schema.getCollection() and Session.getSchema() functions. (Bug #26392984)

• When the \quit command was used to exit MySQL Shell, this event could be noted in the error log as a lost connection. The issue has now been fixed. (Bug #27821045, Bug #90281)

• When the MySQL Shell command-line option --json=raw was used, the output was actually provided in pretty-printed format, and an empty string was displayed in place of error messages. These issues have now been corrected. (Bug #27733996, Bug #26737357)

• When MySQL Shell commands were executed from a script, interactive prompting for passwords and confirmations was not available. Now, interactive prompting is enabled by default when the commands are used in a script, as it is when they are used on the command line. The --no-wizard command line option disables interactive prompting for MySQL Shell commands used in both ways. (Bug #27702250)

• The util.checkForServerUpgrade() operation now prompts the user for a password interactively if the required password is not provided along with the command. If the --no-wizard option has been used to disable the connection wizard, missing credentials instead result in an error and the function is not executed. (Bug #27514395)

• The util.checkForServerUpgrade() operation was requiring the wrong privileges for the user passed to the function. The user now requires ALL privileges, and does not require the GRANT OPTION privilege. (Bug #27506702)

• The util.checkForServerUpgrade() operation was rejecting host names that included the % (percent) symbol or were specified as a numeric IP address. (Bug #27506079, Bug #27513260)

• By default, MySQL Shell connections are assumed to require a password, which is requested at the login prompt. A new MySQL Shell command-line option --no-password is provided to explicitly specify that no password is used, and to disable the password prompt. The --no-password option can be used if socket peer-credential authentication is in use (for Unix socket connections), or for any authentication method where the user has a passwordless account (though note that this situation is insecure and not recommended).

  The methods previously provided by MySQL Shell for specifying that no password is used for the connection are still valid, and can be used instead of the --no-password option. These methods are as follows:

  • If you are connecting using a URI type string, place a : after the user name in the URI type string but do not specify a password after it.

  • If you are connecting using individual parameters, specify the --password= option with an empty value.

  (Bug #26986360)

• When an error occurred while returning a result set, the fetch operation was interrupted but the associated error was not reported. The error is now reported correctly. (Bug #26906527)

• Support for microseconds has been added to the mysqlx.dateValue() function and the Date object. (Bug #26429497)

• Argument validation and error messages were improved for the mysqlx.dateValue() function. (Bug #26429426, Bug #26429377)

• The db global object is not available for connections in SQL mode. This reference has been removed from the message returned by the \connect command in that situation. (Bug #26428665)

• A shortcut to uninstall MySQL Shell is no longer provided in Microsoft Windows menus in the group of installed MySQL programs. Uninstallation of MySQL Shell should be handled through MySQL Installer. (Bug #26317449)

• The runtime timer for MySQL Shell, which reports the time taken for each query execution, has been refactored to provide increased precision of 4 digits for fractional seconds. (Bug #25976636, Bug #86135)

• In the event of a disconnection, MySQL Shell did not reconnect to the schema that was in use before the connection was lost. The last active schema set by the user is now restored during the automatic reconnection process, and during a manually triggered reconnection using the \reconnect command. (Bug #25974003, Bug #86115)

• MySQL Shell no longer attempts to reconnect automatically when the connection to the server is lost. A new MySQL Shell command \reconnect is provided, which makes MySQL Shell try several reconnection attempts for the current global session with the existing connection parameters. If those attempts are unsuccessful, you can make a fresh connection using the \connect command and specifying the connection parameters. (Bug #25105307)

• A memory leak was fixed that occurred if MySQL Shell was started and a connection was made to the MySQL server, then the user exited MySQL Shell without executing any commands. (Bug #24794589)