Dear MySQL users,
MySQL Shell 8.0.24 is a maintenance release of MySQL Shell 8.0 Series (a
component of the MySQL Server). The MySQL Shell is provided under Oracle’s
dual-license.
MySQL Shell 8.0 is highly recommended for use with MySQL Server 8.0 and 5.7.
Please upgrade to MySQL Shell 8.0.24.
MySQL Shell is an interactive JavaScript, Python and SQL console interface,
supporting development and administration for the MySQL Server. It provides
APIs implemented in JavaScript and Python that enable you to work with MySQL
InnoDB Cluster and use MySQL as a document store.
The AdminAPI enables you to work with MySQL InnoDB Cluster and InnoDB
ReplicaSet, providing integrated solutions for high availability and
scalability using InnoDB based MySQL databases, without requiring advanced
MySQL expertise. For more information about how to configure and work with
MySQL InnoDB Cluster and MySQL InnoDB ReplicaSet see
https://dev.mysql.com/doc/mysql-shell/8.0/en/admin-api-userguide.html
The X DevAPI enables you to create “schema-less” JSON document collections and
perform Create, Update, Read, Delete (CRUD) operations on those collections
from your favorite scripting language. For more information about how to use
MySQL Shell and the MySQL Document Store support see
https://dev.mysql.com/doc/refman/en/document-store.html
For more information about the X DevAPI see
https://dev.mysql.com/doc/x-devapi-userguide/en/
If you want to write applications that use the the CRUD based X DevAPI you can
also use the latest MySQL Connectors for your language of choice. For more
information about Connectors see
https://dev.mysql.com/doc/index-connectors.html
For more information on the APIs provided with MySQL Shell see
https://dev.mysql.com/doc/dev/mysqlsh-api-javascript/8.0/
and
https://dev.mysql.com/doc/dev/mysqlsh-api-python/8.0/
Using MySQL Shell’s SQL mode you can communicate with servers using the legacy
MySQL protocol. Additionally, MySQL Shell provides partial compatibility with
the mysql client by supporting many of the same command line options.
For full documentation on MySQL Server, MySQL Shell and related topics, see
https://dev.mysql.com/doc/mysql-shell/8.0/en/
For more information about how to download MySQL Shell 8.0.24, see the “General
Availability (GA) Releases” tab at
http://dev.mysql.com/downloads/shell/
We welcome and appreciate your feedback and bug reports, see
http://bugs.mysql.com/
Enjoy!
Changes in MySQL Shell 8.0.24 (2021-04-20, General Availability)
* AdminAPI Added or Changed Functionality
* AdminAPI Bugs Fixed
* Functionality Added or Changed
* Bugs Fixed
AdminAPI Added or Changed Functionality
* The API command-line integration, used for scripting
MySQL Shell, has been improved to use the function and
option metadata to properly parse arguments and associate
them to the corresponding parameter in the API call.
Previously, the command-line integration was processing
all the arguments as strings, causing errors using
operations such as:
mysqlsh user@hostname:3306 — cluster setOption “autoRejoinTries” 5
The command-line integration now interprets the data
being passed in based on what the API functions expect to
receive. This enhancement also introduces support for
lists in command-line integration calls.
Additionally, you can now access the MySQL Shell online
help from the command-line integration. For example, to
get help on the shell.options functions, issue:
$ mysqlsh — shell options –help
For more information, see API Command Line Integration
(https://dev.mysql.com/doc/mysql-shell/8.0/en/mysql-shell-command-line-integration.html).
In addition, when registering a new MySQL Shell extension
function, the new boolean cli option is supported by the
shell.addExtensionObjectMember() operation. When an
operation is registered with the cli option set to true,
the object and the functions are made available for the
command-line integration. This enables you to extend the
scripting possibilities of MySQL Shell. (Bug #31186637)
AdminAPI Bugs Fixed
* Most of the AdminAPI operations contain metadata
preconditions to determine if it is valid or not to
execute them. When this check was being done, it was
adding two entries to the log, one to indicate the check
was about to be done, and another to indicate the
metadata state. This meant that operations such as
monitoring a cluster, which implies executing regular
status requests, resulted in a large number of log
entries. Now, these two log messages have been merged
into a single entry that gets logged at the info level
when the metadata state is not correct, and as debug info
when the state is correct. In other words, if the
metadata state is correct, the message is only logged
when log_level=debug. (Bug #32582745)
* The memberRole is now included in the default output of
Cluster.status(). Previously, this information was only
included when the extended option had a value of 1 or
higher. This makes it easier to know what an instance’s
role is in the cluster, regardless of whether its
operating mode is R/W or R/O. (Bug #32381513)
* dba.checkInstanceConfiguration() was performing an
incorrect validation regarding the required privileges.
This resulted in an endless loop where the operation
detected missing privileges, you would give the grants as
specified by the interactive help, but the operation
would fail, indicating that the grants were missing. Now,
the verification checks the correct list. Additionally,
the internal list of required grants included SUPER,
which is deprecated in 8.0. The fix replaces the SUPER
grant with the fine-grained grants. (Bug #32287986)
* The memberSslMode option did not support the VERIFY_CA
and VERIFY_IDENTITY modes for the following operations:
+ dba.createCluster()
+ Cluster.addInstance()
+ Cluster.rejoinInstance()
Now, the memberSslMode option supports these modes, and
when they are used there is a validation to ensure that
the CA certificates are supplied. If you choose to use
the VERIFY_CA or VERIFY_IDENTITY mode, on each cluster
instance you must manually supply the CA certificates
using the ssl_ca and/or ssl_capath option. For more
information, see Securing your Cluster
(https://dev.mysql.com/doc/mysql-shell/8.0/en/configuring-innodb-cluster.html#mysql-innodb-cluster-securing).
Thanks to Daniël van Eeden for the contribution. (Bug
#32247631, Bug #32241000)
* In version 8.0.23, a check was added to verify if the
server_id of all cluster instances is registered in the
metadata as an instance attribute, and if not then the
metadata is updated accordingly. This check is executed
on add, rejoin and rescan operations. However, when
upgrading a cluster from a version earlier than 8.0.23 to
version 8.0.23 and higher, the server_id was not
registered in the metadata unless you performed a manual
rejoin. This was being silently ignored because it was
not included in any diagnostic messages. Now, a new
verification checks if the server_id of cluster instances
is missing from the metadata and includes a note message
in the instanceErrors attribute in the output of
Cluster.status() indicating to use Cluster.rescan() to
fix it. (Bug #32226871)
* From MySQL Shell version 8.0.17, AdminAPI stores the
replication or recovery accounts used for each added
instance in the metadata schema, in the instances table.
However, the specific transaction could fail or that
entry might have been manually removed from the metadata
schema, resulting in failures when trying to add other
instances to the cluster, and there was no way to resolve
this using AdminAPI. In such a situation, if you tried to
add an instance the operation failed with an error. Now,
AdminAPI tries to detect problems in the metadata related
to these accounts for an InnoDB Cluster or InnoDB
ReplicaSet. The status() operation prints a message if
the required account is missing in the metadata schema,
and also if it is not the one actually used. In such
situations, the MySQL Shell help instructs you to either
re-add the instance or run rescan() based on the detected
problem. The addInstance() operation also prints a hint
to call rescan() if any missing recovery users are found
in the metadata. (Bug #32157182)
* Cluster.addInstance() was permitting usage of the
expelTimeout and consistency options when it should not.
These options are cluster level settings that can only be
set using dba.createCluster() and Cluster.setOption().
(Bug #29779995)
* The dba.checkInstanceConfiguration() operation detects if
the instance has any tables that do not have a primary
key. Group Replication requires every table that is to be
replicated by the group to have a defined primary key.
However, this does not mean that having a table without a
primary key causes Group Replication to block or fail.
Rather, the outcome is that changes to that table are not
replicated but the group continues operating. Previously,
if the dba.checkInstanceConfiguration() operation
detected a table without a primary key, the operation
returned with a status of ok and only mentioned tables
with a missing primary key and unsupported engines. Now,
if the operation detects such a table, it returns with a
status of error. As part of this work, the
dba.createCluster() operation has been changed to fail if
it finds such tables. (Bug #29771457)
* As part of the fix for Bug#28701263, AdminAPI started
setting and persisting a default value of READ_ONLY for
the group_replication_exit_state_action system variable.
The default used by Group Replication was ABORT_SERVER.
However, in MySQL Server 8.0.16 the default value of
group_replication_exit_state_action became READ_ONLY so
AdminAPI should not change and persist it. Now, on
instances running 8.0.16 and later, the value of
group_replication_exit_state_action is not modified. (Bug
#29037274)
* The exception information listed in the online help had
become outdated and unwieldy for the interactive MySQL
Shell, so it has been removed. (Bug #28542904)
References: See also: Bug #29853828, Bug #32426083, Bug
#32157120, Bug #28825389.
* Using the allowRootFrom option with the
dba.deploySandboxInstance() operation was creating a
different remote root account depending on whether MySQL
Shell was running in interactive mode or not. Now, the
default value of allowRootFrom is consistent between both
modes, and the account is created as root@% in both
interactive and non-interactive mode. (Bug #27369121)
* When you issue dba.createCluster() and
dba.createReplica(), tables are created to store the
metadata. If the default storage engine was not InnoDB,
these operations could fail. Now, metadata creation
operations always use the InnoDB storage engine. (Bug
#101446, Bug #32110085)
Functionality Added or Changed
* From MySQL 8.0.24, SQL statements that you issue in MySQL
Shell’s SQL mode can be sent to the operating system’s
system logging facility (syslog on Unix, or the Windows
Event Log). You can select this option by specifying the
–syslog command-line option when starting MySQL Shell,
or by setting the history.sql.syslog MySQL Shell
configuration option. SQL statements that would be
excluded from the MySQL Shell code history are also
excluded from the system logging facility. (Bug
#31995742, Bug #31514599)
* MySQL Shell’s instance dump utility util.dumpInstance(),
schema dump utility util.dumpSchemas(), and table dump
utility util.dumpTables() can now check for tables that
do not contain primary keys. The check is carried out
when the ocimds option is enabled for checking
compatibility with MySQL Database Service, and an error
is reported for every table included in the dump that
does not have a primary key. The compatibility option,
which implements appropriate measures for compatibility,
has two new modification choices to notify MySQL Shell’s
dump loading utility to create primary keys in invisible
columns for tables that do not have them, or to ignore
the missing primary keys. Primary keys are required for
MySQL Database Service High Availability, which uses
Group Replication.
Bugs Fixed
* Previously, MySQL Shell retried requests to Oracle Cloud
Infrastructure Object Storage a maximum of 5 times, with
a 30 second wait in between retries, and a maximum
overall wait of 5 minutes. The retry strategy has now
been changed to increase the wait window and reduce the
possibility of a dump or load operation failing. MySQL
Shell now retries a maximum of 10 times, with a 1 minute
wait in between retries, and a maximum overall wait of 10
minutes. (Bug #32592962)
* MySQL Shell’s instance dump utility util.dumpInstance()
stopped with an error if the last schema to be dumped was
a schema that contained no tables. The issue has now been
fixed. (Bug #32540460)
* MySQL Shell’s instance dump utility util.dumpInstance()
has been optimized so that it can still be used
successfully if there are limitations on the server’s
resources such as disk space or the thread stack. To
handle such situations, the queries from the utility can
be repeated to retrieve smaller chunks of data if
required, and file sorting is avoided. (Bug #32528186)
* MySQL Shell’s instance dump utility util.dumpInstance()
incorrectly removed grants of all privileges to users.
The utility now expands GRANT ALL statements in the dump
to list all privileges granted on all schemas and tables
(*.*), and to list allowed privileges for system schemas.
The dump loading utility util.loadDump() now extracts the
lists of allowed and revoked global privileges during
loading, and strips these from GRANT statements relating
to system schemas and to all schemas and tables. (Bug
#32526567)
* MySQL Shell’s dump loading utility util.loadDump() now
grants privileges after all the data is loaded.
Previously, an error could occur if the utility tried to
grant a privilege on a routine that did not yet exist.
(Bug #32526496)
* MySQL Shell’s instance dump utility util.dumpInstance(),
schema dump utility util.dumpSchemas(), and table dump
utility util.dumpTables() could not complete the dump if
the gtid_executed system variable or the Information
Schema’s COLUMN_STATISTICS table was unavailable. The
utilities now display a warning message and log a
detailed error message in this situation. These items are
not required for a successful dump. (Bug #32515696)
* MySQL Shell’s handling and formatting has been improved
for the help text that you provide for dictionary
parameters and their options when you register a Python
plugin. (Bug #32509309)
* MySQL Shell’s instance dump utility util.dumpInstance(),
schema dump utility util.dumpSchemas(), and table dump
utility util.dumpTables() no longer write a FLUSH TABLES
statement to the binary log, as this can interfere with
replication. (Bug #32490714)
* From MySQL 8.0.23, MySQL Server supports replication from
a source server that does not have GTIDs enabled and does
not use GTID-based replication, to a replica that has
GTIDs enabled, using the
ASSIGN_GTIDS_TO_ANONYMOUS_TRANSACTIONS option of the
CHANGE REPLICATION SOURCE TO statement. MySQL Shell’s
instance dump utility util.dumpInstance(), schema dump
utility util.dumpSchemas(), and table dump utility
util.dumpTables() now support this functionality by
storing the binary log file name and position in the dump
metadata, in addition to the gtid_executed GTID set. The
additional privilege REPLICATION CLIENT is required in
order for the utilities to be able to collect this
information, although if the user ID does not have that
privilege, the dump continues but without the binary log
information.
The binary log information can be used after loading the
dumped data into the replica server to set up replication
with a non-GTID source server. MySQL Shell’s dump loading
utility util.loadDump() prints the binary log and GTID
set information from the dump metadata (in YAML format)
when you specify the new option showMetadata: true. (Bug
#32430402)
* MySQL Shell did not correctly handle an empty array that
was added to a collection. The result set normally
returned from the server is now skipped in this
situation. (Bug #32377134)
* MySQL Shell’s instance dump utility util.dumpInstance(),
schema dump utility util.dumpSchemas(), and table dump
utility util.dumpTables() were unable to chunk table data
from a MySQL 5.6 server instance, due to differences in
the output of the EXPLAIN SELECT statement for that
version. The utilities now account for the difference,
and also cache the server version information for faster
access. (Bug #32376447)
* MySQL Shell’s parallel table import utility
util.importTable() set a zero exit code if a non-critical
error occurred that did not interrupt the import, such as
a directory or file not being found. The utility now sets
a non-zero error code instead when the first non-critical
error is observed. (Bug #32286186)
* MySQL Shell’s upgrade checker utility
util.checkForServerUpgrade() now checks for spatial data
columns that were originally created in MySQL 5.6. The
underlying data type for such columns in MySQL 5.6 does
not match their underlying data type in MySQL 8.0, so
upgrade of the table is prohibited, and it must be
recreated. (Bug #32257211, Bug #101944)
* When MySQL Shell casts a string to a boolean value, the
operation is now case insensitive. Previously, the
results could differ between platforms. (Bug #32217910)
* When MySQL Shell’s \warnings command was used to show
warnings after each statement, warnings were not
displayed for a classic MySQL protocol connection. (Bug
#32151137)
* MySQL Shell’s parallel table import utility
util.importTable() now checks whether an uploaded object
is a directory, and excludes these from wildcard matching
that was specified for files. (Bug #31991122)
* MySQL Shell’s dump loading utility util.loadDump() can
split oversized chunks of data into smaller chunks for
upload. Previously, if loading was stopped then resumed
partway through this stage, the rows in the smaller
chunks that were already loaded were not taken into
account and skipped, which could lead to deadlocks. The
utility’s progress file now records the smaller chunks
individually so that they can be skipped if the load is
stopped and resumed. (Bug #31961688)
* An event that contained a sequence of two semi-colons
caused MySQL Shell’s instance dump utility
util.dumpInstance(), schema dump utility
util.dumpSchemas(), and table dump utility
util.dumpTables() to enter an infinite loop looking for
delimiters. (Bug #31820571)
* The decodeColumns option for MySQL Shell’s parallel table
import utility util.importTable() could be specified
without an accompanying columns option, resulting in the
import stopping with an error. (Bug #31407058)
* If a script that was run interactively in MySQL Shell’s
Python mode did not have a newline character at the end,
and the script ended with a multiline command, MySQL
Shell waited for input instead of processing the command.
The user had to press Enter to finish running the script,
and the last line of the script was incorrectly saved in
MySQL Shell’s code history. MySQL Shell now adds an empty
line after processing a script input stream, to ensure
that this situation does not occur. (Bug #30765725)
* MySQL Shell used a different character set for collations
depending on whether X Protocol or classic MySQL protocol
was used to connect to the MySQL server instance, leading
to inconsistency and in some situations, errors. For
MySQL 5.7 instances, MySQL Shell now uses a SET NAMES
statement to set all the relevant session system
variables to the utf8mb4 character set. For MySQL 8.0
instances, MySQL Shell now sets the collation_connection
system variable to the utf8mb4_0900_ai_ci character set.
(Bug #30516645)
Enjoy and thanks for the support!
On Behalf of the MySQL Engineering Team,
Nawaz Nazeer Ahamed