MySQL Cluster on Windows (alpha).  MySQL Cluster NDB 7.0 is available on an experimental basis for Windows operating systems (for production use on Windows, you should use MySQL Cluster NDB 7.1.3 or later). Features and behavior comparable to those found on platforms that are already supported—such as Linux and Solaris—are planned for MySQL Cluster on Windows. In MySQL Cluster NDB 7.0, you must build from source (Windows binaries are available for MySQL Cluster NDB 7.1 releases). To enable MySQL Cluster support on Windows when building from source, you must configure the build using the WITH_NDBCLUSTER_STORAGE_ENGINE option. For more information, see Installing MySQL from Source on Windows.

Ability to add nodes and node groups online.  Beginning with MySQL Cluster NDB 6.4.0, it is possible to add new node groups (and thus new data nodes) to a running MySQL Cluster without shutting down and reloading the cluster. As part of enabling this feature, a new command CREATE NODEGROUP has been added to the cluster management client and the functionality of the ALTER ONLINE TABLE ... REORGANIZE PARTITION SQL statement has been extended. For more information, see Section 7.13, “Adding MySQL Cluster Data Nodes Online”.

Data node multithreading support.  Beginning with MySQL Cluster NDB 6.4.0, a multithreaded version of the data node daemon, named ndbmtd, is available for use on data node hosts with multiple CPU cores. This binary is built automatically when compiling with MySQL Cluster support; no additional options other than those needed to provide MySQL Cluster support are needed when configuring the build. In most respects, ndbmtd functions in the same way as ndbd, and can use the same command-line options and configuration parameters. In addition, the new MaxNoOfExecutionThreads configuration parameter can be used to determine the number of data node process threads for ndbmtd. For more information, see Section 6.3, “ndbmtd — The MySQL Cluster Data Node Daemon (Multi-Threaded)”.

Configuration cache.  Formerly, MySQL Cluster configuration was stateless—that is, configuration information was reloaded from the cluster's global configuration file (usually config.ini) each time ndb_mgmd was started. Beginning with MySQL Cluster NDB 6.4.0, the cluster's configuration is cached internally, and the global configuration file is no longer automatically re-read when the management server is restarted. This behavior can be controlled using the management server options --configdir, --initial, and --reload. In MySQL Cluster NDB 7.0.15 and later, the configuration cache can be disabled using the --config-cache option. For more information about these changes, see Section 5.2, “MySQL Cluster Configuration Files”. For more information about the new management server options, see Section 6.4, “ndb_mgmd — The MySQL Cluster Management Server Daemon”.

Detection of NDB API client connection errors.  In MySQL Cluster NDB 7.0 (6.4.0 and later releases), the NDB API's Ndb_cluster_connection class adds the get_latest_error() and get_latest_error_msg() methods for catching and diagnosing problems with NDB API client connections.

Snapshot options for backups.  Beginning with MySQL Cluster NDB 6.4.0, you can determine when performing a cluster backup whether the backup matches the state of the data when the backup was started or when it was completed, using the new options SNAPSHOTSTART and SNAPSHOTEND for the management client's START BACKUP command. See Section 7.3.2, “Using The MySQL Cluster Management Client to Create a Backup”, for more information.

Dynamic NDB transporter send buffer memory allocation.  Previously, the NDB kernel used a fixed-size send buffer for every data node in the cluster, which was allocated when the node started. Because the size of this buffer could not be changed after the cluster was started, it was necessary to make it large enough in advance to accommodate the maximum possible load on any transporter socket. However, this was an inefficient use of memory, since much of it often went unused. Beginning with MySQL Cluster NDB 6.4.0, send buffer memory is allocated dynamically from a memory pool shared between all transporters, which means that the size of the send buffer can be adjusted as necessary. This change is reflected by the addition of the configuration parameters TotalSendBufferMemory, ReservedSendBufferMemory, and OverLoadLimit, as well as a change in how the existing SendBufferMemory configuration parameter is used. For more information, see Section 5.2.13, “Configuring MySQL Cluster Send Buffer Parameters”.

Robust DDL operations.  Beginning with MySQL Cluster NDB 6.4.0, DDL operations (such as CREATE TABLE or ALTER TABLE) are protected from data node failures; in the event of a data node failure, such operations are now rolled back gracefully. Previously, if a data node failed while trying to perform a DDL operation, the MySQL Cluster data dictionary became locked and no further DDL statements could be executed without restarting the cluster.

IPv6 support in MySQL Cluster Replication.  Beginning with MySQL Cluster NDB 6.4.1, IPv6 networking is supported between MySQL Cluster SQL nodes, which makes it possible to replicate between instances of MySQL Cluster using IPv6 addresses. However, IPv6 is supported only for direct connections between MySQL servers; all connections within an individual MySQL Cluster must use IPv4. For more information, see Section 8.3, “Known Issues in MySQL Cluster Replication”.

Restoring specific databases, tables, or columns from a MySQL Cluster backup.  It is now possible to exercise more fine-grained control when restoring a MySQL Cluster from backup using ndb_restore. Beginning with MySQL Cluster NDB 6.4.3, you can choose to restore only specified tables or databases, or exclude specific tables or databases from being restored, using the new ndb_restore options --include-tables, --include-databases, --exclude-tables, and --exclude-databases. Beginning with MySQL Cluster NDB 7.0.7, it is also possible to restore to a table having fewer columns than the original using the --exclude-missing-columns option. For more information about all of these options, see Section 6.18, “ndb_restore — Restore a MySQL Cluster Backup”.

Improved Disk Data file system configuration.  As of MySQL Cluster NDB 6.4.3, you can specify default locations for MySQL Cluster Disk Data data files and undo log files using the data node configuration parameters FileSystemPathDD, FileSystemPathDataFiles, and FileSystemPathUndoFiles. This eliminates the need to use symbolic links to place Disk Data files separately from other files in data node file systems to improve Disk Data performance. For more information, see Disk Data file system parameters.

Automatic creation of Disk Data log file groups and tablespaces.  Beginning with MySQL Cluster NDB 6.4.3, using the data node configuration parameters InitialLogFileGroup and InitialTablespace, you can cause the creation of a MySQL Cluster Disk Data log file group, tablespace, or both, when the cluster is first started. When using these parameters, no SQL statements are required to create these Disk Data objects. For more information, see Disk Data object creation parameters.

Improved internal message passing and record handling.  MySQL Cluster NDB 7.0 contains 2 changes that optimize the use of network connections by addressing the size and number of messages passed between data nodes, and between data nodes and API nodes, which can increase MySQL Cluster and application performance:

Both of these optimizations are internal to the NDB API, and so is transparent to applications; this is true whether an application uses the NDB API directly or does so indirectly through an SQL node.

Configuration parameter data dumps.  Starting with MySQL Cluster NDB 7.0.6, the ndb_config utility supports a --configinfo option that causes it to dump a list of all configuration parameters supported by the cluster, along with brief descriptions, information about the parameters' default and permitted values, and the sections of the config.ini file in which the parameters apply. An additional --xml switch causes ndb_config to use XML rather than plaintext output. Using ndb_config --configinfo or ndb_config --configinfo --xml requires no access to a running MySQL Cluster, any other programs, or any files. For more information and examples, see Section 6.6, “ndb_config — Extract MySQL Cluster Configuration Information”.

Per-table reporting of free space on disk.  The INFORMATION_SCHEMA.FILES table shows information about used and free space in MySQL Cluster Disk Data data files, but this information is not applicable to individual tables. In MySQL Cluster NDB 7.0.8 and later, the ndb_desc utility provides two additional columns in its output that show the amount of space allocated on disk for a given NDB table as well the amount of space that remains available for additional storage of disk-based column data for that table. For more information, see Section 6.9, “ndb_desc — Describe NDB Tables”.

Improved restart times.  Optimizations in redo log handling and other file system operations introduced in MySQL Cluster NDB 7.0.9 have the potential to reduce considerably the time required for restarts. While actual performance benefits observed in production setups will naturally vary depending on database size, hardware, and other conditions, our own preliminary testing has shown that these improvements can yield startup times that are faster than those typical of previous MySQL Cluster NDB 7.0 releases by a factor of 50 or more.

Native support for default column values.  Starting with MySQL Cluster NDB 7.0.15, default values for table columns are stored in the NDB kernel rather than by the MySQL server as was done previously. This means that inserts on tables having column value defaults can be smaller and faster than before, because less data must be sent from SQL nodes to NDBCLUSTER.

Tables created using previous MySQL Cluster releases can still be used in MySQL Cluster 7.0.15 and later; however, they do not support native default values until they are upgraded. You can upgrade a table with non-native default values to support native default values using an offline ALTER TABLE statement.

--nowait-nodes option for management servers.  Starting with MySQL Cluster NDB 7.0.10, it is possible to configure a cluster with two management servers, but to start the cluster using only one of them by starting the management node daemon with the --nowait-nodes option. The other management server can then be started at a later time to join the running MySQL Cluster.

Increased flexibility in online upgrade procedure.  Previously, when performing an upgrade of a running MySQL cluster, the order in which the types of cluster nodes had to be upgraded was very strict. However, beginning with MySQL Cluster NDB 7.0.10, MySQL Cluster supports online upgrading of API nodes (including MySQL servers running as SQL nodes) online upgrading management nodes, data nodes, or both.

New CHANGE MASTER TO option for circular replication.  Beginning with MySQL Cluster NDB 7.0.11, the CHANGE MASTER TO statement supports an IGNORE_SERVER_IDS option which takes a comma-separated list of server IDs and causes events originating from the corresponding servers to be ignored. (Log rotation and log deletion events are preserved.)

See CHANGE MASTER TO Syntax, as well as SHOW SLAVE STATUS Syntax, for more information.

New replication conflict resolution strategy.  Beginning with MySQL Cluster NDB 7.0.11, the function NDB$MAX_DELETE_WIN() is available to implement “greatest timestamp, delete wins” conflict resolution. See NDB$MAX_DELETE_WIN(column_name), for more information.

Improved lock handling for primary key lookups on BLOB tables.  A MySQL Cluster table stores all but the first 256 bytes of any BLOB or TEXT column values in a separate BLOB table; when executing queries against such tables, a shared lock is obtained. Prior to MySQL Cluster NDB 7.0.12, when the query used a primary key lookup and took place within a transaction, the lock was held for the duration of the transaction, even after no more data was being read from the NDB table. Now in such cases, the lock is released when all BLOB data associated with the table has been read. (Bug #49190)

Heartbeat thread policy and priority.  Beginning with MySQL Cluster NDB 7.0.13, a new configuration parameter HeartbeatThreadPriority makes it possible to set the policy and the priority for the heartbeat thread on management and API nodes.

Improved access to partitioning information.  The ndb_desc utility now provides additional information about the partitioning of data stored in MySQL Cluster. Beginning with MySQL Cluster NDB 7.0.13, the --blob-info option causes this program to include partition information for BLOB tables in its output. Beginning with MySQL Cluster NDB 7.0.14, the --extra-node-info option causes ndb_desc to include information about data distribution (that is, which table fragments are stored on which data nodes). Each of these options also requires the use of the --extra-partition-info option.

Information about partition-to-node mappings can also be obtained using the Table::getFragmentNodes() method, also added in MySQL Cluster NDB 7.0.14.

Replication attribute promotion and demotion.  Beginning with MySQL Cluster NDB 7.0.14, MySQL Cluster Replication supports attribute promotion and demotion when replicating between columns of different but similar types on the master and the slave. For example, it is possible to promote an INT column on the master to a BIGINT column on the slave, and to demote a TEXT column to a VARCHAR column.

The implementation of type demotion distinguishes between lossy and non-lossy type conversions, and their use on the slave can be controlled by setting the slave_type_conversions global server system variable.

For more information, see Attribute promotion and demotion (MySQL Cluster).

Incompatible change in NDB API event reporting.  Beginning with MySQL Cluster NDB 7.0.15, DDL events are no longer reported on Event objects by default. Instead such event reporting must be enabled explicitly using the Event::setReport() method. For more information, see Event::setReport(), and The Event::EventReport Type.

Number of table attributes.  Beginning with MySQL Cluster NDB 7.0.15, the maximum number of attributes (columns plus indexes) per table has been increased from 128 to 512.

Heartbeat ordering.  Beginning with MySQL Cluster NDB 7.0.16, it is possible to set a specific order for transmission of heartbeats between data nodes, using the HeartbeatOrder data node configuration parameter introduced in this version. This parameter can be useful in situations where multiple data nodes are running on the same host and a temporary disruption in connectivity between hosts would otherwise cause the loss of a node group (and thus failure of the cluster).

Relaxed ndb_restore column comparison rules.  When restoring data, ndb_restore compares the attributes of a column for equality with the definition of the column in the target table. However, not all of these attributes need to be the same for ndb_restore to be meaningful, safe and useful. Beginning with MySQL Cluster NDB 7.0.16, ndb_restore automatically ignores differences in certain column attributes which do not necessarily have to match between the version of the column in a backup and the version of that column in the MySQL Cluster to which the column data is being restored. These attributes include the following:

In such cases, ndb_restore reports any such differences to minimize any chance of user error.

Storage of user data in anyValue.  When writing NDB events to the binary log, MySQL Cluster uses OperationOptions::anyValue to store the server ID. Beginning with MySQL Cluster NDB 7.0.17, it is possible to store user data from an NDB API application in part of the anyValue when mysqld has been started with the --server-id-bits option set to a nondefault value. Also beginning with MySQL Cluster NDB 7.0.17, it is possible to view this data in the output of mysqlbinlog, for which its own --server-id-bits option is added.

--add-drop-trigger option for mysqldump.  Beginning with MySQL Cluster NDB 7.0.19, this option can be used to force all CREATE TRIGGER statements in mysqldump output to be preceded by a DROP TRIGGER IF EXISTS statement.

Forcing node shutdown and restart.  In MySQL Cluster NDB 7.0.19 and later, it is possible using the ndb_mgm management client or the MGM API to force a data node shutdown or restart even if this would force the shutdown or restart of the entire cluster. In the management client, this is implemented through the addition of the -f (force) option to the STOP and RESTART commands. For more information, see Section 7.2, “Commands in the MySQL Cluster Management Client”. The MGM API also adds two new methods for forcing such a node shutdown or restart; see ndb_mgm_stop4(), and ndb_mgm_restart4(), for more information about these methods.

TimeBetweenEpochsTimeout and GCP stop control.  Beginning with MySQL Cluster NDB 7.0.21, it is possible to disable GCP stops by setting TimeBetweenEpochsTimeout to 0. In addition, a warning is written to the cluster log whenever the time required for a GCP save exceeds 60 seconds or the time required for a GCP commit exceeds 10 seconds. This warning includes a report of the current value of TimeBetweenEpochsTimeout. For more information, see Disk Data and GCP Stop errors.

Skipping corrupted tables in NDB native backups.  Beginning with MySQL Cluster NDB 7.0.21, you can cause ndb_restore to ignore tables that are corrupted due to missing blob parts tables by using the the --skip-broken-objects option. When this option is used, such tables are skipped, and the restoration of any remaining uncorrupted tables in the backup continues.

BLOB read and write batching.  Beginning with MySQL Cluster NDB 7.0.21, it possible to control batching of BLOB read and write operations. For SQL nodes, this can be done using the --ndb-blob-read-batch-bytes and --ndb-blob-write-batch-bytes options for mysqld. In NDB API applications, you can control batching of BLOB reads and writes using the NdbTransaction methods setMaxPendingBlobReadBytes(), getMaxPendingBlobReadBytes(), setMaxPendingBlobWriteBytes(), and getMaxPendingBlobWriteBytes().

Restoring from a NDB native backup to a differently-named database.  MySQL Cluster NDB 7.0.22 adds a new --rewrite-database option to ndb_restore, which makes it possible to restore to a database having a different name from that of the database in the backup. The option can be used multiple times, and it is possible to restore from more than one source database in the backup to a single target database (although no protection against table or other object name collision is provided).

See Section 6.18, “ndb_restore — Restore a MySQL Cluster Backup”, for more information.

INFORMATION_SCHEMA inprovements.  Beginning with MySQL Cluster NDB 7.0.22, INFORMATION_SCHEMA provides disk usage information for MySQL Cluster Disk Data tables. Previously, INFORMATION_SCHEMA.TABLES showed only the space usage for the in-memory data part of the table. Now, it also shows the space allocated for and used by the disk_part data of that table as well.

In addition, the INFORMATION_SCHEMA.PARTITIONS table (which did not show any statistics for NDB tables) now shows correct values in this table's TABLE_ROWS, AVG_ROW_LENGTH, DATA_LENGTH, MAX_DATA_LENGTH, and DATA_FREE columns, for each partition.

Configuration version information in ndbinfo.nodes.  You can see which version or versions of the MySQL Cluster configuration file are in effect on the data nodes by checking the config_generation column which is added to the nodes table in MySQL Cluster NDB 7.0.24.

Improvements in adding data nodes online.  Beginning with MySQL Cluster NDB 7.0.24, it is possible to add data nodes online to a running MySQL Cluster without performing a rolling restart of the cluster or starting data node processes with the --nowait-nodes option. This can be done by setting Nodegroup = 65536 in the config.ini file for any data nodes that should be started at a later time, when first starting the cluster. The amount of time the cluster waits before doing this can be controlled using the StartNoNodeGroupTimeout data node configuration parameter.

Unique key updates in replication.  It is possible in MySQL Cluster NDB 7.0.25 and later to employ operations that update unique keys when replicating NDB tables. Previously this could lead to duplicate key errors when trying to execute the binary log (due to the fact that row events in the binary log were ordered according to the partitioning of the base table, and could differ in order within the epoch for that in which they were executed).

Starting with MySQL Cluster NDB 7.0.22, the NDB kernel implements a number of statistical counters relating to actions performed by or affecting Ndb objects. Such events include starting, closing, and aborting transactions; operations using primary keys or unique keys; table, range, and pruned scans; blocking of threads by incompleted operations; and data and events sent and received by MySQL Cluster nodes. NDB API statistics counters are incremented inside the NDB kernel whenever NDB API calls are made or data is sent to or received by the data nodes. A MySQL Server running as an SQL node in the cluster can access the values of these counters as system status variables, as seen in the output of SHOW STATUS, or in the results of queries against the SESSION_STATUS or GLOBAL_STATUS table in the INFORMATION_SCHEMA database. By comparing the values of these status variables before and after the execution of statements affecting NDB tables, you can observe the corresponding actions taken on the NDB API level. This can be highly useful when monitoring and tuning MySQL Cluster.

For more information, see Section 7.15, “NDB API Statistics Counters and Variables”. See also Section 5.4.4, “MySQL Cluster Status Variables”, for information about the individual status variables.

Version 2 binary log row events.  New versions of the WRITE_ROW, UPDATE_ROW, and DELETE_ROW events have been implemented in MySQL Cluster NDB 7.0.27, extending them with additional information intended to support future enhancements; these are referred to as “Version 2” binary log events.

Version 2 log events are not backward compatible, and cannot be read by older slaves. A new mysqld option --log-bin-use-v1-row-events can be employed to force use of Version 1 events when writing the binary log. This can be used during upgrades to make a newer mysqld generate Version 1 binary log row events that can be read by older slaves.

Fail-fast data node capability.  Beginning with MySQL Cluster NDB 7.0.28, it is possible to enable fail-fast behavior for data nodes by enabling the CrashOnCorruptedTuple configuration parameter introduced in this version (disabled by default). Doing so causes data nodes to fail whenever they detect a corrupted tuple.

Rows per partition limit removed.  Previously it was possible to store a maximum of 46137488 rows in a single MySQL Cluster partition—that is, per data node. Beginning with MySQL Cluster NDB 7.0.36, this limitation has been lifted, and there is no longer any practical upper limit to this number.