Changes in MySQL Shell 8.0.29 (2022-04-26, General Availability)

• AdminAPI Bugs Fixed

• Functionality Added or Changed

• Bugs Fixed

AdminAPI Bugs Fixed

• If MySQL Router was upgraded as part of an InnoDB Cluster upgrade process, the dba.upgradeMetadata() command continued to report that MySQL Router was outdated on the first try. This was because the command gave the MySQL Router account required privilege access, but MySQL Router required a metadata cache reload (TTL) to implement this and update the metadata information. Retrying the command one or more times would work when the TTL took place, which by default happens every 0.5 seconds. The command now attempts to retrieve MySQL Router’s metadata information for up to two seconds before returning the results to the user, and adds a note to explain a high TTL as a possible cause of MySQL Router being reported as outdated. (Bug #33914146)

• The Cluster.rebootClusterFromCompleteOutage() command did not implement a changed setting for group_replication_view_change_uuid that was generated and set by a Cluster.rescan() command. The operation now checks if a new value for group_replication_view_change_uuid has been persisted and uses it when rebooting the cluster. (Bug #33850528)

• For MySQL Server instances at release 8.0.27 and above, for an InnoDB Cluster that is part of an InnoDB ClusterSet, the group_replication_view_change_uuid system variable is required and must be set to the same value on all member servers in the cluster. Previously, the Cluster.rescan() command generated and set a value for the system variable on the cluster’s member instances if it was not found or did not match. Following this, a full reboot of the InnoDB Cluster was required to implement the change, which AdminAPI informed the administrator about but did not perform automatically. If the reboot was not performed, members with a pending group_replication_view_change_uuid correction were subsequently unable to rejoin the group using Group Replication’s auto-rejoin function if they were restarted.

  Now, the Cluster.rescan() command does not set the system variable value automatically by default. The administrator may specify the new Cluster.rescan() option updateViewChangeUuid to generate and set a value for the system variable in the same way as the previous behavior. If the new option is not specified, the command makes no changes to the system variable, and just issues a warning to the administrator that the system variable value must be set and the cluster must be rebooted. (Bug #33748812)

• AdminAPI’s Cluster.setPrimaryInstance(instance) command overrides the election process and forces a specific server instance in the underlying Group Replication group to become the new primary. It uses the group_replication_set_as_primary function. Previously, the function waited for all active transactions on the existing primary to end, including incoming transactions after the function was used, before making the current primary read only and changing to the new primary. There was no upper limit to the wait time. From MySQL Shell 8.0.29, you can use the new runningTransactionsTimeout option to add a timeout between 0 and 3600 seconds for transactions that are running when you use the function. When you set a timeout, incoming transactions after the command is issued are rejected. (Bug #33538559)

• If the host name and port reported by a server instance (report_host and report_port) were changed after the instance joined an InnoDB Cluster, the cluster metadata was not changed correspondingly, causing issues with operations such as setting the instance as the primary instance. The Cluster.status() command now reports any mismatch as an instance error, and the Cluster.rescan() command now detects if these items have changed and automatically updates the metadata to match the values taken from the instance. (Bug #33016337)

• When the Cluster.addInstance operation is used to add a server instance to an InnoDB Cluster, MySQL Shell checks for tables on the server that do not have primary keys, and fails the operation if any are present. If the user selected MySQL Clone as the provisioning method, which clears all the data on the server anyway, that check was still carried out although it was unnecessary. The check is now ignored when cloning is specified for the Cluster.addInstance operation. (Bug #32992693)

• The dba.configureInstance() function, which pre-checks instance configuration for InnoDB Cluster usage, did not handle the log_bin system variable correctly for MySQL 5.7 instances, where its behavior is different to MySQL 8.0 instances. The handling and output for the log_bin system variable have now been corrected. (Bug #31964706)

• MySQL Shell's Cluster.setPrimaryInstance() command uses a call to a server function to specify the instance that becomes the new primary server. However, if an error was returned from the server function, such as the specified server instance being unable to act as the primary due to its MySQL Server version being higher than others in the group, MySQL Shell did not check the error and reported the action as successful. MySQL Shell now checks for errors returned by server functions and displays them to the user. (Bug #31775698)

• When the adoptFromGR option of the dba.createCluster command was set to false, the behavior was the same as when the option was omitted, with a confirmation prompt provided and the operation proceeding if the user confirmed it. Specifying adoptFromGR: false now prevents cluster creation if the instance belongs to a Group Replication group, without presenting a prompt to the user. (Bug #30548447)

• AdminAPI’s validation of the expel timeout value for group members (group_replication_member_expel_timeout), which can be set using the expelTimeout option when creating a cluster, has been corrected to take into account changes to the default and maximum values since the option was introduced in MySQL 8.0.13 and to make the check release specific. (Bug #29337169)

• If an instance was added to an InnoDB Cluster using MySQL Clone, and Ctrl+C or the waitRecovery: 0 option was used to let recovery take place in the background, the recovery account from the donor instance was used instead of the recovery account created for the new instance. If the new instance was later removed with a Cluster.removeInstance command, the recovery account created for it was not dropped, since AdminAPI was only checking for the recovery account that was actually in use on the instance (the donor’s account). The account removal logic now considers the account created for the instance and cleans it up even if it was not used. (Bug #105776, Bug #33630808)

• If MySQL Router was bootstrapped to use Unix sockets or to disable TCP ports, AdminAPI did not handle the resulting endpoint values correctly, resulting in errors during AdminAPI operations relating to MySQL Router management such as Cluster.listRouters(). (Bug #105649, Bug #33602309)

Functionality Added or Changed

• MySQL Shell's new diagnostics utility, util.debug.collectDiagnostics, enables you to gather diagnostic information from the connected MySQL server, generate reports in TSV and YAML formats, and present them in a zip archive in the location of your choice.

  This utility enables you to retrieve diagnostic information from standalone servers, members of replication topologies, MySQL Database Service DB Systems, and so on.

  Information retrieved includes the contents of the performance_schema.error_log and processlist, lists of global and persisted variables, replication group information, and a list of tables without primary keys.

  See collectDiagnostics Utility. (WL #14863)

• MySQL Shell now supports the --fido-register-factor connection option to register a FIDO device for authentication to the server. You use the option with mysqlsh in the same way as with the mysql client program, and MySQL Shell passes on any errors raised by the MySQL client library or the client side authentication plugin. (WL #14769)

• In the event of a failure, MySQL Shell's instance dump utility util.dumpInstance(), schema dump utility util.dumpSchemas(), table dump utility util.dumpTables(), and dump loading utility util.loadDump() now report an error code and user-friendly message for recognizable errors. The messages provide context about the stage in the process where the error occurred, and the names of the objects involved. Error codes for the dump operations are in the range 52000-52999, and errors for the load operation are in the range 53000-53999. Error codes for connection and network errors that can be experienced by any of the utilities are in the range 54000-54999, and the error code matches the HTTP error involved. For example, error 54404 occurs when the target of a URL is not found. (WL #14841)

• MySQL Shell’s shell.prompt() function, part of the MySQL Shell API, enables developers to create scripts for external applications to interact with a user through MySQL Shell to gather data. The function has new options to support additional prompt types besides a plain string value and a password prompt, comprising: a question with predefined answers, selecting from a list of options, a directory path, a file path for a new file to be saved, and a file path for an existing file to be opened. You can also supply a title, which MySQL Shell does not show to the user but applications can use to identify the prompt, and a description for the prompt, which MySQL Shell shows to the user as separate paragraphs before the prompt message. You can specify a default value for any of the prompt types to be returned to the external application if the user provides no data. (WL #14872)

Bugs Fixed

• When uninstalling MySQL Shell from the Debian or Ubuntu platforms, a number of files and directories were left behind. The uninstaller now clears these up. (Bug #33934625)

• MySQL Shell has a new API function Column.getFlags() that makes the list of enabled column flags available to the scripting interface. Reporting of column flags was standardized between X Protocol and classic MySQL protocol. (Bug #33916064)

• Some issues with MySQL Shell’s handling of binary data have been corrected. BINARY, VARBINARY, and BLOB data types are now correctly reported as bytes, and TEXT, CHAR, and VARCHAR data types are reported as strings. For printed results, binary columns are encoded using Base64 for JSON output format, and as hex strings for other output formats. In the scripting layer, for Python, binary data is represented as binary strings, and for JavaScript, it uses ArrayBuffers; binary data can be inserted into the database using either of those formats. (Bug #33891697, Bug #32789317, Bug #32108582)

• When MySQL Shell's instance dump utility util.dumpInstance(), schema dump utility util.dumpSchemas(), and table dump utility util.dumpTables() add an invisible primary key column to a table with the compatibility option’s create_invisible_pks modification, it is now added as the first column rather than the last column. (Bug #33880935)

• Throughput for MySQL Shell's JSON import utility util.importJSON() slowed greatly when the option convertBsonTypes: true or convertBsonOid: true was used. The cause of the slowdown (a JSON document parser method) has now been fixed. (Bug #33799202)

• To tolerate recent and future changes to the list of reserved keywords, MySQL Shell's instance dump utility util.dumpInstance(), schema dump utility util.dumpSchemas(), table dump utility util.dumpTables(), and dump loading utility util.loadDump() now quote all identifiers. (Bug #33744583)

• MySQL Shell's instance dump utility util.dumpInstance(), schema dump utility util.dumpSchemas(), and table dump utility util.dumpTables(), and dump loading utility util.loadDump() could not be used by a user account that contained an @ (at sign) character in the username. The issue could also occur with AdminAPI commands such as dba.checkInstanceConfiguration which retrieved information about the currently active user account. MySQL Shell now permits the @ (at sign) character in a user name when the account details have been returned by MySQL’s CURRENT_USER() function. (Bug #33743612)

• The BACKUP_ADMIN privilege is no longer required to perform a consistent dump of a MySQL Server instance (with the consistent option). MySQL Shell's instance dump utility util.dumpInstance(), schema dump utility util.dumpSchemas(), and table dump utility util.dumpTables() now carry out a consistency check. If the user account does not have the BACKUP_ADMIN privilege and LOCK INSTANCE FOR BACKUP cannot be executed, the utilities make an extra consistency check during the dump. If this check fails, an instance dump is stopped, but a schema dump or a table dump continues and returns an error message to alert the user that the consistency check failed. (Bug #33699844, Bug #33697289)

• When setting up an SSH tunnel, MySQL Shell now selects from a list of approved SSH algorithms and parameters which excludes algorithms and parameters that do not meet acceptable security standards. (Bug #33670923)

• MySQL Shell's dump loading utility util.loadDump() could stop unexpectedly if two or more threads loading data from an OCI Object Storage bucket (using the osBucketName option and the local OCI profile) performed an operation that triggered the headers cache cleaning operation. The cache cleaning class is no longer shared between threads. (Bug #33654278)

• From MySQL Shell 8.0.27, the instance dump utility util.dumpInstance(), schema dump utility util.dumpSchemas(), and table dump utility util.dumpTables() returned an error when attempting to dump an NDB table. This was because the utility’s partition awareness feature used an operation that is disabled for a storage engine that supports automatic partitioning. Partition awareness is now disabled for tables that use the NDB storage engine. (Bug #33643395)

• MySQL Shell's plugin manager now validates the MySQL Shell version and returns an error if a plugin cannot be installed on the current version. The URL of the official MySQL Shell plugin repository (https://cdn.mysql.com/mysql_shell/plugins_manifest.zip) has also been updated. (Bug #33642210)

• MySQL Shell's dump loading utility util.loadDump() could fail with the error “Invalid character in identifier” when loading a routine that was created when the ANSI_QUOTES SQL mode was in effect. This was caused by incorrect handling of identifiers quoted with double quotes. The issue has now been fixed, along with an issue in the dump utilities where identifiers in triggers were not quoted when required. (Bug #33640887)

• MySQL Shell's dump loading utility util.loadDump() now ensures that when the deferTableIndexes option is used to defer the creation of secondary indexes until after the table load, a full-text index is added first. Adding the first full-text index rebuilds the table, so doing this first ensures the other indexes do not need to be re-created. (Bug #33624751)

• Following a regression in MySQL Shell 8.0.28, zero dates returned by MySQL resulted in an exception in Python mode. The issue was fixed along with a number of other date and time value handling issues. (Bug #33621406)

• The online help for autorejointries listed the default value as 0. This was correct up to MySQL Shell 8.0.20. The default value since MySQL Shell 8.0.21 is 3. (Bug #31356257)