Changes in MySQL Shell 8.0.21 (2020-07-13, General Availability)

• AdminAPI Added or Changed Functionality

• AdminAPI Bugs Fixed

• Functionality Added or Changed

• Bugs Fixed

AdminAPI Added or Changed Functionality

• A new user configurable tag framework has been added to the metadata, to allow specific instances of a cluster or ReplicaSet to be marked with additional information. Tags can be any ASCII character and provide a namespace. You set tags for an instance using the setInstanceOption() operation. In addition, AdminAPI and MySQL Router 8.0.21 support specific tags, which enable you to mark instances as hidden and remove them from routing. MySQL Router then excludes such tagged instances from the routing destination candidates list. This enables you to safely take a server instance offline, so that applications and MySQL Router ignore it, for example while you perform maintenance tasks, such as server upgrade or configuration changes. To bring the instance back online, use the setInstanceOption() operation to remove the tags and MySQL Router adds the instance back to the routing destination candidates list, and it becomes online for applications. For more information, see Tagging Metadata. (WL #13788)

AdminAPI Bugs Fixed

• Important Change: Previously, Group Replication did not support binary log checksums, and therefore one of the requirements for instances in InnoDB Cluster was that binary log checksums were disabled by having the binlog_checksum system variable set to NONE. AdminAPI verified the value of binlog_checksum during the dba.checkInstanceConfiguration() operation and disallowed creating a cluster or adding an instance to a cluster that did not have binary log checksums disabled. In version 8.0.21, Group Replication has lifted this restriction, therefore InnoDB Cluster now permits instances to use binary log checksums, with binlog_checksum set to CRC32. The setting for binlog_checksum does not have to be the same for all instances. In addition, sandboxes deployed with version 8.0.21 and later do not set the binlog_checksum variable, which defaults to CRC32. (Bug #31329024)

• Adopting a Group Replication setup as a cluster can be performed when connected to any member of the group, regardless of whether it is a primary or a secondary. However, when a secondary member was used, super_read_only was being incorrectly disabled on that instance. Now, all operations performed during an adoption are done using the primary member of the group. This ensures that no GTID inconsistencies occur and that super_read_only is not incorrectly disabled on secondary members. (Bug #31238233)

• Using the clusterAdmin option to create a user which had a netmask as part of the host resulted in an error when this user was passed to the dba.createCluster() operation. Now, accounts that specify a netmask are treated as accounts with wildcards, meaning that further checks to verify if the account accepts remote connections from all instances are skipped. (Bug #31018091)

• The check for instance read-only compatibility was using a wrong MySQL version as the base version. The cross-version policies were added to Group Replication in version 8.0.17, but the check was considering instances running 8.0.16. This resulted in a misleading warning message indicating that the added instance was read-only compatible with the cluster, when this was not true (only for instances 8.0.16). The fix ensures that the check to verify if an instance is read-compatible or not with a cluster is only performed if the target instance is running version 8.0.17 or later. (Bug #30896344)

• The maximum number of instances in an InnoDB Cluster is 9, but AdminAPI was not preventing you from trying to add more instances to a cluster and the resulting error message was not clear. Now, if a cluster has 9 instances, Cluster.addInstance prevents you adding more instances. (Bug #30885157)

• Adding an instance with a compatible GTID set to a InnoDB Cluster or InnoDB ReplicaSet on which provisioning is required should not require any interaction, because this is considered a safe operation. Previously, in such a scenario, when MySQL Clone was supported MySQL Shell still prompted to choose between cloning or aborting the operation. Now, the operation proceeds with cloning, because this is the only way to provision the instance.

  Note

  instances with an empty GTID set are not considered to have a compatible GTID set when compared with the InnoDB Cluster or InnoDB ReplicaSet. Such scenarios are considered to be unknown, therefore MySQL Shell prompts to confirm which action should be taken.

  (Bug #30884590)

• The Group Replication system variables (prefixed with group_replication) do not exist if the plugin has not been loaded. Even if the system variables are persisted to the instance's option file, they are not loaded unless the Group Replication plugin is also loaded when the server starts. If the Group Replication plugin is installed after the server starts, the option file is not reloaded, so all system variables have default values. Instances running MySQL 8.0 do not have a problem because SET PERSIST is used. However, on instances running version MySQL 5.7, the dba.rebootCluster() operation could not restore some system variables if the Group Replication plugin was uninstalled. Now, the dba.configureInstance() operation persists the Group Replication system variables to configuration files with the loose_ prefix. As a result, once the Group Replication plugin is installed, on instances running 5.7 the persisted values are used instead of the default values. (Bug #30768504)

• The updateTopologyMode option has been deprecated and the behavior of Cluster.rescan() has been changed to always update the topology mode in the Metadata when a change is detected. MySQL Shell now displays a message whenever such a change is detected. (Bug #29330769)

• The cluster.addInstance() and cluster.rejoinInstance() operations were not checking for the full range of settings which are required for an instance to be valid for adding to the cluster. This resulted in attempts to use instances which run on different operating systems to fail. For example, a cluster running on two instances that were hosted on a Linux based operating system would block the addition of an instance running Microsoft Windows. Now, cluster.addInstance() and cluster.rejoinInstance() operations validate the instance and prevent adding or rejoining an instance to the cluster if the value of the lower_case_table_names, group_replication_gtid_assignment_block_size or default_table_encryption of the instance are different from the ones on the cluster. (Bug #29255212)

Functionality Added or Changed

• MySQL Shell now has an instance dump utility, dumpInstance(), and schema dump utility, dumpSchemas(). The new utilities support the export of all schemas or a selected schema from an on-premise MySQL server instance into an Oracle Cloud Infrastructure Object Storage bucket or a set of local files. The schemas can then be imported into a MySQL Database Service DB System using MySQL Shell's new dump loading utility. The new utilities provide Oracle Cloud Infrastructure Object Storage streaming, MySQL Database Service compatibility checks and modifications, parallel dumping with multiple threads, and file compression. See Instance Dump Utility, Schema Dump Utility, and Table Dump Utility. (WL #13807)

• MySQL Shell's new dump loading utility, loadDump(), supports the import of schemas dumped using MySQL Shell's new instance dump utility and schema dump utility into a MySQL Database Service DB System. The dump loading utility provides data streaming from remote storage, parallel loading of tables or table chunks, progress state tracking, resume and reset capability, and the option of concurrent loading while the dump is taking place. See Dump Loading Utility. (WL #13808)

• The X DevAPI implementation now supports JSON schema validation, which enables you to ensure that your documents have a certain structure before they can be inserted or updated in a collection. To enable or modify JSON schema validation you pass in a JSON object like:

  {
      validation: {
        level: "off|strict",
        schema: "json-schema"
      }
  }

  Here, validation is JSON object which contains the keys you can use to configure JSON schema validation. The first key is level, which can take the value strict or off. The second key, schema, is a JSON schema, as defined at http://json-schema.org. If the level key is set to strict, documents are validated against the json-schema when they are added to the collection, or when an operation updates the document. If the document does not validate, the server generates an error and the operation fails. If the level key is set to off, documents are not validated against the json-schema.

  You can pass a validation JSON object to the schema.createCollection() operation, to enable JSON schema validation, and schema.modifyCollection() operation, to change the current JSON schema validation, for example to disable validation. For more information, see JSON Schema Validation. (WL #13019)

Bugs Fixed

• MySQL Shell plugins now support the use of the **kwargs syntax in functions defined in Python that are made available by the plugin. Using **kwargs in a function definition lets you call the function using a variable-length list of keyword arguments with arbitrary names. If the function is called from MySQL Shell's JavaScript mode, MySQL Shell passes the named arguments and their values into a dictionary object for the Python function. MySQL Shell first tries to associate a keyword argument passed to a function with any corresponding keyword parameter that the function defines, and if there is none, the keyword argument is automatically included in the **kwargs list. As a side effect of this support, any API function called from Python in MySQL Shell that has a dictionary of options as the last parameter supports defining these options using named arguments. (Bug #31495448)

• When switching to SQL mode, MySQL Shell queries the SQL mode of the connected server to establish whether the ANSI_QUOTES mode is enabled. Previously, MySQL Shell could not proceed if it did not receive a result set in response to the query. The absence of a result is now handled appropriately. (Bug #31418783, Bug #99728)

• In SQL mode, when the results of a query are to be printed in table format, MySQL Shell buffers the result set before printing, in order to identify the correct column widths for the table. With very large result sets, it was possible for this practice to cause an out of memory error. MySQL Shell now buffers a maximum of 1000 rows for a result set before proceeding to format and print the table. Note that if a field in a row after the first 1000 rows contains a longer value than previously seen in that column in the result set, the table formatting will be misaligned for that row. (Bug #31304711)

• Context switching in MySQL Shell's SQL mode has been refactored and simplified to remove SQL syntax errors that could be returned when running script files using the source command. (Bug #31175790, Bug #31197312, Bug #99303)

• The user account that is used to run MySQL Shell's upgrade checker utility checkForServerUpgrade() previously required ALL privileges. The user account now requires only the RELOAD, PROCESS, and SELECT privileges. (Bug #31085098)

• In Python mode, MySQL Shell did not handle invalid UTF-8 sequences in strings returned by queries. (Bug #31083617)

• MySQL Shell's parallel table import utility importTable() has a new option characterSet, which specifies a character set encoding with which the input data file is interpreted during the import. Setting the option to binary means that no conversion is done during the import. When you omit this option, the import uses the character set specified by the character_set_database system variable to interpret the input data file. (Bug #31057707)

• On Windows, if the MySQL Shell package was extracted to and used from a directory whose name contained multi-byte characters, MySQL Shell was unable to start. MySQL Shell now handles directory names with multi-byte characters correctly, including when setting up Python, loading prompt themes, and accessing credential helpers. (Bug #31056783)

• MySQL Shell's JSON import utility importJSON() now handles UTF-8 encoded files that include a BOM (byte mark order) at the start, which is the sequence 0xEF 0xBB 0xBF. As a workaround in earlier releases, remove this byte sequence, which is not needed. (Bug #30993547, Bug #98836)

• When the output format was set to JSON, MySQL Shell's upgrade checker utility checkForServerUpgrade() included a description and documentation link for a check even if no issues were found. These are now omitted from the output, as they are with the text output format. (Bug #30950035)