Changes in MySQL 5.7.17 (2016-12-12, General Availability)

• Compilation Notes

• Connection Management Notes

• MySQL Enterprise Notes

• Packaging Notes

• Security Notes

• Test Suite Notes

• X Plugin Notes

• Functionality Added or Changed

• Bugs Fixed

Compilation Notes

• For GCC versions higher than 4.4, -fno-expensive-optimizations was replaced with -ffp-contract=off, which has the effect of enabling more optimizations. Thanks to Alexey Kopytov for the patch. (Bug #24571672, Bug #82760)

Connection Management Notes

• MySQL Server now includes a plugin library that enables administrators to introduce an increasing delay in server response to clients after a certain number of consecutive failed connection attempts. This capability provides a deterrent that slows down brute force attacks that attempt to access MySQL user accounts. For more information, see The Connection-Control Plugins. (WL #8885)

MySQL Enterprise Notes

• Enterprise Encryption for MySQL Enterprise Edition now enables server administrators to impose limits on maximum key length by setting environment variables. These can be used to prevent clients from using excessive CPU resources by passing very long key lengths to key-generation operations. For more information, see MySQL Enterprise Encryption Usage and Examples. (Bug #19687742)

Packaging Notes

• RPM packages now are built with -DWITH_NUMA=ON for platforms with NUMA support: OEL higher than EL5, Fedora, SLES, Docker. (Bug #24689078)

Security Notes

• Incompatible Change: These changes were made to mysqld_safe:

  • Unsafe use of rm and chown in mysqld_safe could result in privilege escalation. chown now can be used only when the target directory is /var/log. An incompatible change is that if the directory for the Unix socket file is missing, it is no longer created; instead, an error occurs. Due to these changes, /bin/bash is required to run mysqld_safe on Solaris. /bin/sh is still used on other Unix/Linux platforms.

  • The --ledir option now is accepted only on the command line, not in option files.

  • mysqld_safe ignores the current working directory.

  Other related changes:

  • Initialization scripts that invoke mysqld_safe pass --basedir explicitly.

  • Initialization scripts create the error log file only if the base directory is /var/log or /var/lib.

  • Unused systemd files for SLES were removed.

  (Bug #24483092, Bug #25088048, Bug #25378439, Bug #25378565)

  References: See also: Bug #24464380, Bug #24388753, Bug #24619033, Bug #82920.

• OpenSSL is ending support for version 1.0.1 in December 2016; see https://www.openssl.org/policies/releasestrat.html. Consequently, MySQL Commercial Server builds now use version 1.0.2 rather than version 1.0.1, and the linked OpenSSL library for the MySQL Commercial Server has been updated from version 1.0.1 to version 1.0.2j. For a description of issues fixed in this version, see https://www.openssl.org/news/vulnerabilities.html.

  This change does not affect the Oracle-produced MySQL Community build of MySQL Server, which uses the yaSSL library instead. (WL #9136)

Test Suite Notes

• mysql-test-run.pl could not be run with --valgrind-option=--tool=custom_tool, for values of custom_tool such as massif or helgrind, because it added the options for memcheck that might not be understood by other tools. Also, the mysql-test-run.pl --callgrind option did not work because it supplied an invalid --base option to callgrind. Thanks to Daniel Black for the patch on which the fixes were based. (Bug #23713613, Bug #82039)

X Plugin Notes

• A failure occurred when X Plugin called a hostname resolution function. (Bug #25652096, Bug #85154)

• The stmt field was marked as required in the message:

  message Mysqlx.Crud.ModifyView {
     required Collection collection = 1;
  
     optional string definer = 2;
     optional ViewAlgorithm algorithm = 3;
     optional ViewSqlSecurity security = 4;
     optional ViewCheckOption check = 5;
  
     repeated string column = 6;
     required Mysqlx.Crud.Find stmt = 7;
  }

  (Bug #24968735, Bug #83595)

• For a table created with CREATE TABLE foo (doc JSON), the statement list_objects incorrectly reported the table as a collection. (Bug #24963952)

• When two X Plugin instances were started on the same port, the resulting error message was split over two lines and contained duplicate information. (Bug #24679018, Bug #83008)

• The performance_schema.threads table did not show processlist statistics for the X Plugin. (Bug #24638038)

• If the client sent an unknown message type to the server, the server would send back an error message and then disconnect the client. (Bug #24611754, Bug #82868)

• When the client sent a zero length message, the X Plugin did not send a response. For a subsequent message the client received an error E_X_BAD_MESSAGE and was disconnected, depending on the header content. (Bug #24595459, Bug #82862)

• X Plugin displayed its version to any connected user, including those not authenticated. (Bug #24562707, Bug #82784)

• When X Plugin failed to bind to @@GLOBAL.mysqlx_port it did not indicate this failure in the global status variables. (Bug #24554351, Bug #82761)

• The statement list_objects returned incorrect information for some views in sys database. (Bug #24472325)

• X Plugin was trying to send result sets to a client, even where the connection had been closed. The plugin now stops execution of a stored procedure if the connection is reset. (Bug #24440344)

Functionality Added or Changed

• Incompatible Change; Partitioning: The generic partitioning handler in the MySQL server is deprecated, and will be removed in MySQL 8.0. As part of this change, the mysqld --partition and --skip-partition options as well as the -DWITH_PARTITION_STORAGE_ENGINE build option are also deprecated, and will later be removed; partitioning will no longer be shown in the INFORMATION_SCHEMA.PLUGINS table or in the output of SHOW PLUGINS.

  Following the removal of the generic partitioning handler, the storage engine used for a given table will be expected to provide its own (“native”) partitioning handler as the InnoDB and NDB storage engines currently do. Currently, no other MySQL storage engines provide native partitioning support, nor is any planned for any other storage engines in current or development versions of MySQL.

  Use of tables with nonnative partitioning now results in an ER_WARN_DEPRECATED_SYNTAX warning. Also, the server performs a check at startup to identify tables that use nonnative partitioning; for any found, the server writes a message to its error log. To disable this check, use the --disable-partition-engine-check option.

  To prepare for migration to MySQL 8.0, any table with nonnative partitioning should be changed to use an engine that provides native partitioning, or be made nonpartitioned. For example, to change a table to InnoDB, execute this statement:

  ALTER TABLE table_name ENGINE = INNODB;

  (WL #8971, WL #9457)

• InnoDB: By default, InnoDB reads uncommitted data when calculating statistics. In the case of an uncommitted transaction that deletes rows from a table, InnoDB excludes records that are delete-marked when calculating row estimates and index statistics, which can lead to non-optimal execution plans for other transactions that are operating on the table concurrently using a transaction isolation level other than READ UNCOMMITTED. To avoid this scenario, a new configuration option, innodb_stats_include_delete_marked, can be enabled to ensure that InnoDB includes delete-marked records when calculating persistent optimizer statistics. (Bug #23333990)

• Group Replication: MySQL Group Replication is a new MySQL plugin that enables you to create a highly available distributed MySQL service across a group of MySQL server instances, with data consistency, conflict detection and resolution, and group membership services all built-in. By using a powerful new group communication service, which provides an implementation of the popular Paxos algorithm, the group of MySQL Server instances automatically coordinates on data replication, consistency, and membership. This provides all of the built-in mechanisms necessary for making your MySQL databases highly available.

  By default Group Replication operates in single-primary mode where a single server instance, called the primary, accepts write requests. The remaining server instances in the group, called secondaries, function as replicas of the primary. In the event of an unexpected failure of the primary, an automatic primary election process takes place and one of the secondaries is elected as the new primary. Group Replication also supports virtually synchronous multi-primary replication, with certain considerations and restrictions, which offers update everywhere functionality. In this mode all members are equal and you can distribute your reads and writes across all MySQL Server instances in the group.

  Regardless of the operating mode, Group Replication provides a dynamic membership service that relies on distributed failure detection. Server instances can join and leave the group dynamically, and you can query the group's membership list at any point through Performance Schema tables. Server instances that join the group automatically synchronize their state with the group by doing an automatic point-in-time recovery which ensures that they reach synchrony with the group.

  MySQL Group Replication's virtually synchronous replication is also a fully integrated part of MySQL, using the InnoDB storage engine, the Performance Schema tables, standard GTIDs and the well known replication infrastructure (binary and relay logs, multi-source replication, multithreaded slave execution, etc.), which makes it a familiar and intuitive experience for existing MySQL users and makes it very easy to integrate with MySQL's standard asynchronous and semisynchronous replication, allowing you to mix and match as needed to create varied and complex replication topologies. (WL #9503)

• The systemd service file for mysqld now includes a Documentation value in the [Unit] section to provide a link to the systemd documentation in the MySQL Reference Manual. (Bug #24735762)

• Unit testing now uses Google Mock 1.8. (Bug #24572381, Bug #82823)

• If mysqld is invoked with --daemonize, stdout and stderr are redirected to /dev/null if connected to a terminal type device, so that mysqld can behave as a true daemon. (Bug #21627629)

Bugs Fixed

• Incompatible Change: A change made in MySQL 5.7.8 for handling of multibyte character sets by LOAD DATA was reverted due to the replication incompatibility (Bug #24487120, Bug #82641)

  References: See also: Bug #23080148.

• NDB Cluster: MySQL NDB Cluster encountered race conditions compiling lex_hash.h. (Bug #24931655, Bug #83477, Bug #27470071, Bug #89482)

• InnoDB; Microsoft Windows: On a MySQL 64-bit build on Windows, a file I/O retry result was misinterpreted due to a missing cast necessary for the correct operation of the retry path, resulting in a failing assertion and operating system error. (Bug #24711351)

• InnoDB: The INFORMATION_SCHEMA.REFERENTIAL_CONSTRAINTS table reported NULL for a foreign key constraint name (UNIQUE_CONSTRAINT_NAME) after restarting the server. (Bug #25126722)

• InnoDB: A prepared XA transaction was rolled back by a high priority transaction. The high priority transaction should wait if the blocking transaction is in a prepared state. (Bug #25032066)

• InnoDB: InnoDB passed an invalid argument to syscall(SYS_futex). (Bug #24923840, Bug #83375)

• InnoDB: The GCC mach_parse_compressed function should load one to five bytes depending on the value of the first byte. Due to a GCC bug, GCC 5 and 6 emit code to load four bytes before the first byte value is checked (GCC Bug #77673). A workaround prevents this behavior. Thanks to Laurynas Biveinis for the patch. (Bug #24707869, Bug #83073)

• InnoDB: Due to a glibc bug, short-lived detached threads could exit before the caller had returned from pthread_create(), causing a server exit. Thanks to Laurynas Biveinis for the patch. (Bug #24605956, Bug #82886)

• InnoDB: After increasing the value of innodb_undo_logs and restarting the server, the number of active undo tablespaces was not increased when assigning undo tablespaces to newly allocated rollback segments. (Bug #24488141)

• InnoDB: InnoDB incorrectly reported an error about missing encryption when restoring pages from the doublewrite buffer during recovery. (Bug #24471076)

• InnoDB: A cached undo segment was not removed from the rollback segment history during a slow shutdown. (Bug #24450908)

• InnoDB: An error during a table-rebuilding operation on a table with only a generated clustered index (GEN_CLUST_INDEX) raised and assertion due to an error called with an invalid key name. (Bug #24444831)

• InnoDB: Rotating the tablespace encryption master key while the server is in read-only mode raised an assertion instead of displaying an error message. (Bug #24404091)

• InnoDB: On a table without an explicitly defined primary key, InnoDB did not replace the implicit clustered index (GEN_CLUST_INDEX) when a unique key was defined on a NOT NULL column. (Bug #24397406)

• InnoDB: A high priority transaction involving a foreign key constraint check was not able to kill a lower priority blocking transaction. (Bug #24347476)

• InnoDB: Page cleaner threads asserted due to a regression related to the adaptive hash index feature. (Bug #24346574)

  References: This issue is a regression of: Bug #21407023.

• InnoDB: InnoDB failed to free memory used by the full-text optimizer thread. (Bug #24331265)

• InnoDB: When adding a new index, the server dropped an internally defined foreign key index and attempted to use a secondary index defined on a virtual generated column as the foreign key index, causing a server exit. InnoDB now permits a foreign key constraint to reference a secondary index defined on a virtual generated column. (Bug #23533396)

• InnoDB: An INFORMATION_SCHEMA.FILES query resulted in a server exit due to a race condition with a concurrent tablespace creation operation. (Bug #23477214, Bug #81614)

• InnoDB: A table-copying online ALTER TABLE operation on a ROW_FORMAT=REDUNDANT table with indexed virtual columns raised an assertion. (Bug #22018745)

• InnoDB: SHOW ENGINE INNODB STATUS output showed a “cleaning up” state for an idle thread. Thread state information was not reset after statement execution. (Bug #21974225, Bug #78777)

• InnoDB: After a server restart, concurrent INSERT operations a table with an auto-increment primary key resulted in a duplicate entry error. The current auto-increment value was not changed after auto_increment_increment and auto_increment_offset settings were modified. (Bug #20989615, Bug #76872)

• Replication: When using XA transactions, if a lock wait timeout or deadlock occurred for the applier (SQL) thread on a replication slave, the automatic retry did not work. The cause was that while the SQL thread would do a rollback, it would not roll the XA transaction back. This meant that when the transaction was retried, the first event was XA START which was invalid as the XA transaction was already in progress, leading to an XAER_RMFAIL error. (Bug #24764800)

  References: See also: Bug #83588, Bug #24923091, Bug #24966941.

• Replication: The group commit update of GTIDs has been refactored to improve performance on workloads with many small transactions. (Bug #24398760)

• Replication: If the relay_log option was not specified in a configuration file, the relay_log_basename variable was being internally constructed on the fly using hostname but the relay_log_basename variable was not set. When a slave tried to access this uninitialized variable it resulted in an unexpected halt of the server. (Bug #24352667)

• Replication: Tables with special DEFAULT columns, such as DEFAULT CURRENT_TIMESTAMP, that existed only on a slave were not being updated when using row-based replication (binlog_format=ROW). (Bug #22916743)

• Replication: An XA PREPARE statement that failed during the intermediate steps could lead to an inconsistent XA transaction state, where ID = -1 but the binlogged flag was set to true. This caused asserts while executing XA COMMIT and XA ROLLBACK queries. (Bug #22915670)

• Replication: Enabling semisynchronous replication when a server was during the commit stage could cause the master to stop unexpectedly. This was related to the patch for Bug# 75570. (Bug #22202516)

• Replication: The number of generated unwanted fseeks into the binary log file being replicated to a slave has been reduced. (Bug #83226, Bug #24763579)

• Replication: The fix for Bug #81657 was not correctly merged into MySQL 8.0. Thanks to Laurynas Biveinis for alerting us. (Bug #83124, Bug #24715790)

• Replication: The rpl.rpl_binlog_errors test was failing sporadically on Windows. (Bug #82302, Bug #24330138)

• Replication: When binlog_group_commit_sync_delay was set to a value between 1 and 9, if binlog_group_commit_sync_no_delay_count was set to a value greater than 1, and the number of transaction commits was less than binlog_group_commit_sync_no_delay_count, these commits hung forever if no more commits were received; and if binlog_group_commit_sync_no_delay_count was set to 0, all transaction commits hung forever. (Bug #80652, Bug #22891628)

• Replication: When starting semisynchronous replication, if more than 1024 file descriptors existed, the semisynchronous socket was not created correctly. This prevented semisynchronous replication from functioning correctly. (Bug #79865, Bug #23581389)

• Group Replication: For servers built with yaSSL, using group replication with encrypted connections could result in timeout failures waiting for view delivery. (Bug #23592214)

• JSON: When a JSON value consisted of a large sub-document wrapped in many levels of JSON arrays, objects, or both, serialization of the JSON value sometimes required an excessive amount time to complete. (Bug #23031146)

• Some Linux startup scripts did not process the datadir setting correctly. (Bug #25159791)

• CREATE TABLE now requires the FILE privilege if DATA DIRECTORY or INDEX DIRECTORY is specified explicitly as a table or partition option. ALTER TABLE requires the FILE privilege if either option is specified explicitly as a partition option (it ignores them if specified as table options). (Bug #25092566)

• CMake now avoids configuring the -fexpensive-optimizations option for GCC versions for which the option triggers faulty shift-or optimizations. (Bug #24947597, Bug #83517)

• OEL RPM packages now better detect which platforms have multilib support (for which 32-bit and 64-bit libraries can be installed). Thanks to Alexey Kopytov for the patch. (Bug #24925181, Bug #83457)

• OEL RPM packages now better detect which platforms do not have multilib support (for which 32-bit and 64-bit libraries can be installed). Thanks to Alexey Kopytov for the patch. (Bug #24916428, Bug #83428)

• Information about building MySQL 5.6 compatibility libraries in the MySQL 5.7 and higher .spec file is needed only for building libmysqlclient and libmysqld. Information about building the InnoDB memcached plugin was removed. (Bug #24908345, Bug #83409)

• To better provide atomic file creation, Debian packaging scripts now use the coreutils install command rather than touch, chmod, and chown. (Bug #24688682)

• For SLES packages, a typo in the installation script postamble prevented some cleanup from occurring. (Bug #24605300, Bug #82389)

• Warnings occurring during CREATE TABLE ... SELECT could cause a server exit. (Bug #24595992)

• For LOAD DATA statements, input data with too many column values produced only a warning, rather than an error as in MySQL 5.6. An error now occurs. (Bug #24577194, Bug #82830)

• For segmentation faults on FreeBSD, the server did not generate a stack trace. (Bug #24566529, Bug #23575445, Bug #81827)

• The .mylogin.cnf option file is intended for use by client programs, but the server was reading it as well. The server no longer reads it. (Bug #24557925)

• The X Plugin was built with compilation options different from other plugins. (Bug #24555770, Bug #82777)

• If mysqladmin shutdown encountered an error determining the server process ID file, it displayed an error message that did not clearly indicate the error was nonfatal. It now indicates that execution continues. (Bug #24496214)

• The data structure used for ZEROFILL columns could experience memory corruption, leading eventually to a server exit. (Bug #24489302)

• Operation of the mysql-multi.server.sh script was based on my.cnf in the data directory. That option file is no longer used, so mysql-multi.server.sh has been removed. (Bug #24487870)

• Use of very long subpartition names could result in a server exit. Now partition or subpartition names larger than 64 characters produce an ER_TOO_LONG_IDENT error. (Bug #24400628, Bug #82429)

• The Gis_wkb_vector<Gis_point> copy constructor was not explicitly instantiated, causing build problems for the Intel compiler. (Bug #24397833, Bug #82358)

• Upgrading from MySQL 5.6 to 5.7.13 and then to 5.7.14 resulted in an incorrect column order in the mysql.slave_master_info system table. (Bug #24384561, Bug #82384)

• The AppArmor profile installed by Ubuntu packages was missing an entry permitting libnuma to read a /sys hierarchy path, resulting in server startup failure. (Bug #23854929)

• For an INSERT statement for which the VALUES list produced values for the second or later row using a subquery containing a join, the server could exit after failing to resolve the required privileges. (Bug #23762382)

• Infinite recursion could occur if the audit_log plugin signalled an error while handling an error. (Bug #23717558, Bug #82052)

• MySQL now uses readdir() rather than readdir_r(). The latter has been deprecated since glibc 2.24 and caused debug builds of MySQL and builds using GCC 6.1 to fail.

  Additionally, several problems resulting in GCC 6.1 compiler warnings were corrected. (Bug #23708395, Bug #24437737, Bug #82515, Bug #24459890, Bug #82583, Bug #25103242)

• For audit log events in the connection class, the connection_type value was available only for connect events. The value is now available in connect, disconnect, and change-user events. (Bug #23541550)

• On Solaris, gettimeofday() could return an invalid value and cause a server shutdown. (Bug #23499695)

• The keyring_file plugin could attempt to write keys to its storage file when the file did not exist. To ensure that keys are flushed only when the correct storage file exists, keyring_file now stores a SHA-256 checksum of the keyring in the file. Before updating the file, the plugin verifies that it contains the expected checksum. (Bug #23498254)

• A union query resulting in tuples larger than max_join_size could result in a server exit. (Bug #23303485)

• For debug builds, failure to prepare a branch of an XA transaction could lead to a server exit. (Bug #23264552, Bug #81375)

• The optimizer could choose ref access on a secondary index rather than range access on the primary key, even when the cost was higher. (Bug #23259872, Bug #81341)

• For a query with ORDER BY and LIMIT, an optimizer trace did not record the optimizer's switch to a different index. (Bug #23227428, Bug #81250)

• For some deeply nested expressions, the optimizer failed to detect stack overflow, resulting in a server exit. (Bug #23135667)

• The sys schema ps_truncate_all_tables() function did not work with read_only enabled or for users with the SUPER privilege with super_read_only enabled, due to errors attempting to truncate Performance Schema tables. The server now skips the read_only/super_read_only check for Performance Schema tables, with the result that ps_truncate_all_tables() will work under such configurations. (Bug #23103937, Bug #81009)

• For sessions created through the X Plugin, incorrect thread attachment/detachment could cause a server exit. (Bug #23057045)

• The OS X DMG installer did not properly set up keyring plugin installation. (Bug #22991650)

• A binary (in-place) upgrade from MySQL 5.6 to 5.7 followed by a data export performed using mysqlpump resulted in an Invalid default value for date_column error for attempts to reload the dump file. (Bug #22919028, Bug #80706)

• SQL statements executed through the X Plugin were not instrumented in the Performance Schema. (Bug #22859462)

• DROP INDEX operations could fail due to inconsistent handling of index prefix lengths for TEXT-type columns (TINYTEXT and so forth).

  A consequence of this fix is more restrictive behavior for CREATE TABLE and CREATE INDEX statements for which a specified index prefix exceeds the maximum column data type size:

  • For a nonunique index, either an error occurs (if strict SQL mode is enabled), or the index length is reduced to lie within the maximum column data type size and a warning is produced (if strict mode is not enabled).

  • For a unique index, an error occurs regardless of SQL mode because reducing the index length might enable insertion of nonunique entries that do not meet the specified uniqueness requirement.

  (Bug #22740093, Bug #80392)

• The innodb_numa_interleave system variable was erroneously available on some systems that were not NUMA-enabled. Thanks to Tomislav Plavcic for the patch.

  CMake now sets the default WITH_NUMA value based on whether the current platform has NUMA support. For platforms without NUMA support, CMake behaves as follows:

  • With no NUMA option (the normal case), CMake continues normally, producing only this warning: NUMA library missing or required version not available

  • With -DWITH_NUMA=ON, CMake aborts with this error: NUMA library missing or required version not available

  (Bug #22678436, Bug #80288)

• When taking the server offline, a race condition within the Performance Schema could lead to a server exit. (Bug #22551677)

• On macOS, if a table with an associated trigger was renamed to a new name containing both lowercase and uppercase characters, DROP TRIGGER for the trigger resulted in an ER_NO_SUCH_TABLE error for the table. (Bug #22512899, Bug #79873)

• In the MYSQL_FIELD C API structure, the org_table value for derived tables was *, which could cause failure for queries that depend on this value. The org_table value for views and derived tables now is set as follows: If the column is selected from a view, org_table names the view. If the column is selected from a derived table, org_table names the base table. If a derived table wraps a view, org_table still names the base table. If the column is an expression, org_table is the empty string. (Bug #22364401, Bug #79641)

• The Performance Schema events_statements_summary_by_digest table could contain multiple rows for the same statement digest and schema combination, rather than the expected single (unique) row. (Bug #22320066, Bug #79533)

• For Performance Schema system and status variable tables, variable values expressed in a character set different from utf8 could be truncated or incorrect. (Bug #22313205)

• Queries that were grouped on a column of a BLOB-based type, and that were ordered on the result of the AVG(), VAR_POP(), or STDDEV_POP() aggregate function, returned results in the wrong order if InnoDB temporary tables were used. (Bug #22275357, Bug #79366)

• On Ubuntu, error messages were displayed during upgrades from Community to Commercial packages that made it appear as though mysqld and my_print_defaults had not been installed. Those messages were spurious and have been silenced. (Bug #21807248)

• An invalid string value in the WHERE clause of an UPDATE statement, caused an index scan rather than a range scan to be used. For values not present in the index, this could be much slower. Now the optimizer determines this to be an “impossible WHERE” condition. (Bug #21032418, Bug #76933)

• The return value from an fread() call was not checked. (Bug #20671150)

• An in-place ALTER TABLE operation failed to report an error when adding a DATE or DATETIME column under these conditions: a) the column was NOT NULL and no default value was supplied; b) strict and NO_ZERO_DATE SQL modes were enabled; c) the table was not empty.

  An ALTER TABLE operation failed with an error rather than a warning when adding a DATE or DATETIME column under these conditions: a) the column was NOT NULL and no default value was supplied; b) strict SQL mode was enabled and NO_ZERO_DATE SQL mode was not enabled; c) the table was not empty. (Bug #16888677)