18.2.4.2 What is New in NDB Cluster 7.4

The following improvements to NDB Cluster have been made in NDB Cluster 7.4:

• Conflict detection and resolution enhancements.  A reserved column name namespace NDB$ is now employed for exceptions table metacolumns, allowing an arbitrary subset of main table columns to be recorded, even if they are not part of the original table's primary key.

  Recording the complete original primary key is no longer required, due to the fact that matching against exceptions table columns is now done by name and type only. It is now also possible for you to record values of columns which not are part of the main table's primary key in the exceptions table.

  Read conflict detection is now possible. All rows read by the conflicting transaction are flagged, and logged in the exceptions table. Rows inserted in the same transaction are not included among the rows read or logged. This read tracking depends on the fragment replica having an exclusive read lock which requires setting ndb_log_exclusive_reads in advance. See Read conflict detection and resolution, for more information and examples.

  Existing exceptions tables remain supported. For more information, see Section 18.7.11, “NDB Cluster Replication Conflict Resolution”.

• Circular (“active-active”) replication improvements.  When using a circular or “active-active” NDB Cluster Replication topology, you can assign one of the roles of primary of secondary to a given NDB Cluster using the ndb_slave_conflict_role server system variable, which can be employed when failing over from an NDB Cluster acting as primary, or when using conflict detection and resolution with NDB$EPOCH2() and NDB$EPOCH2_TRANS() (NDB 7.4.2 and later), which support delete-delete conflict handling.

  See the description of the ndb_slave_conflict_role variable, as well as NDB$EPOCH2(), for more information. See also Section 18.7.11, “NDB Cluster Replication Conflict Resolution”.

• Per-fragment memory usage reporting.  You can now obtain data about memory usage by individual NDB Cluster fragments from the memory_per_fragment view, added in NDB 7.4.1 to the ndbinfo information database. For more information, see Section 18.6.14.17, “The ndbinfo memory_per_fragment Table”.

• Node restart improvements.  NDB Cluster 7.4 includes a number of improvements which decrease the time needed for data nodes to be restarted. These are described in the following list:

  • Memory allocated that is allocated on node startup cannot be used until it has been touched, which causes the operating system to set aside the actual physical memory required. In previous versions of NDB Cluster, the process of touching each page of memory that was allocated was singlethreaded, which made it relatively time-consuming. This process has now been reimplimented with multithreading. In tests with 16 threads, touch times on the order of 3 times shorter than with a single thread were observed.

  • Increased parallelization of local checkpoints; in NDB Cluster 7.4, LCPs now support 32 fragments rather than 2 as before. This greatly increases utilization of CPU power that would otherwise go unused, and can make LCPs faster by up to a factor of 10; this speedup in turn can greatly improve node restart times.

    The degree of parallelization used for the node copy phase during node and system restarts can be controlled in NDB 7.4.3 and later by setting the MaxParallelCopyInstances data node configuration parameter to a nonzero value.

  • Reporting on disk writes is provided by new ndbinfo tables disk_write_speed_base, disk_write_speed_aggregate, and disk_write_speed_aggregate_node, which provide information about the speed of disk writes for each LDM thread that is in use.

    This release also adds the data node configuration parameters MinDiskWriteSpeed, MaxDiskWriteSpeed, MaxDiskWriteSpeedOtherNodeRestart, and MaxDiskWriteSpeedOwnRestart to control write speeds for LCPs and backups when the present node, another node, or no node is currently restarting.

    These changes are intended to supersede configuration of disk writes using the DiskCheckpointSpeed and DiskCheckpointSpeedInRestart configuration parameters. These 2 parameters have now been deprecated, and are subject to removal in a future NDB Cluster release.

  • Faster times for restoring an NDB Cluster from backup have been obtained by replacing delayed signals found at a point which was found to be critical to performance with normal (undelayed) signals. The elimination or replacement of these unnecessary delayed signals should noticeably reduce the amount of time required to back up an NDB Cluster, or to restore an NDB Cluster from backup.

  • Several internal methods relating to the NDB receive thread have been optimized, to increase the efficiency of SQL processing by NDB. The receiver thread at time may have to process several million received records per second, so it is critical that it not perform unnecessary work or waste resources when retrieving records from NDB Cluster data nodes.

• Improved reporting of NDB Cluster restarts and start phases.  The restart_info table (included in the ndbinfo information database beginning with NDB 7.4.2) provides current status and timing information about node and system restarts.

  Reporting and logging of NDB Cluster start phases also provides more frequent and specific printouts during startup than previously. See Section 18.6.4, “Summary of NDB Cluster Start Phases”, for more information.

• NDB API: new Event API.  NDB 7.4.3 introduces an epoch-driven Event API that supercedes the earlier GCI-based model. The new version of the API also simplifies error detection and handling. These changes are realized in the NDB API by implementing a number of new methods for Ndb and NdbEventOperation, deprecating several other methods of both classes, and adding new type values to Event::TableEvent.

  The event handling methods added to Ndb in NDB 7.4.3 are pollEvents2(), nextEvent2(), getHighestQueuedEpoch(), and getNextEventOpInEpoch2(). The Ndb methods pollEvents(), nextEvent(), getLatestGCI(), getGCIEventOperations(), isConsistent(), and isConsistentGCI() are deprecated beginning with the same release.

  NDB 7.4.3 adds the NdbEventOperation event handling methods getEventType2(), getEpoch(), isEmptyEpoch(), and isErrorEpoch; it obsoletes getEventType(), getGCI(), getLatestGCI(), isOverrun(), hasError(), and clearError().

  While some (but not all) of the new methods are direct replacements for deprecated methods, not all of the deprecated methods map to new ones. The Event Class, provides information as to which old methods correspond to new ones.

  Error handling using the new API is no longer handled using dedicated hasError() and clearError() methods, which are now deprecated (and thus subject to removal in a future release of NDB Cluster). To support this change, the list of TableEvent types now includes the values TE_EMPTY (empty epoch), TE_INCONSISTENT (inconsistent epoch), and TE_OUT_OF_MEMORY (inconsistent data).

  Improvements in event buffer management have also been made by implementing new get_eventbuffer_free_percent(), set_eventbuffer_free_percent(), and get_event_buffer_memory_usage() methods. Memory buffer usage can now be represented in application code using Ndb::EventBufferMemoryUsage. The ndb_eventbuffer_free_percent system variable, also implemented in NDB Cluster 7.4, makes it possible for event buffer memory usage to be checked from MySQL client applications.

  For more information, see the detailed descriptions for the Ndb and NdbEventOperation methods listed. See also Event::TableEvent, as well as Ndb::EventBufferMemoryUsage.

• Per-fragment operations information.  In NDB 7.4.3 and later, counts of various types of operations on a given fragment or fragment replica can obtained easily using the operations_per_fragment table in the ndbinfo information database. This includes read, write, update, and delete operations, as well as scan and index operations performed by these. Information about operations refused, and about rows scanned and returned from a given fragment replica, is also shown in operations_per_fragment. This table also provides information about interpreted programs used as attribute values, and values returned by them.

• --ndb-log-fail-terminate option.  Beginning with NDB 7.4.28, you can cause the SQL node to terminate whenever it is unable to log all row events fully. This can be done by starting mysqld with the --ndb-log-fail-terminate option.

• NDB programs—NDBT dependency removal.  The dependency of a number of NDB utility programs on the NDBT library has been removed. This library is used internally for development, and is not required for normal use; its inclusion in these programs could lead to unwanted issues when testing.

  Affected programs are listed here, along with the NDB versions in which the dependency was removed:

  • ndb_restore, in NDB 7.4.24

  • ndb_show_tables, in NDB 7.4.28

  • ndb_waiter, in NDB 7.4.28

  The principal effect of this change for users is that these programs no longer print NDBT_ProgramExit - status following completion of a run. Applications that depend upon such behavior should be updated to reflect the change when upgrading to the indicated versions.

• End of ClusterJPA support.  ClusterJPA is no longer supported beginning with NDB 7.4.16; its source code and binary have been removed from the NDB Cluster distribution.

• Auto-Installer deprecation and removal.  The MySQL NDB Cluster Auto-Installer web-based installation tool (ndb_setup.py) is deprecated in NDB 7.4.30, and is removed in NDB 7.4.31 and later. It is no longer supported.

• ndbmemcache deprecation and removal.  ndbmemcache is no longer supported. ndbmemcache has been deprecated beginning with NDB 7.4.3, and is scheduled for removal in NDB 7.4.31.

• Node.js support removed.  Beginning with the NDB Cluster 7.4.30 release, support for Node.js by NDB 7.4 has been removed.

  Support for Node.js by NDB Cluster is maintained in NDB 8.0 only.

NDB Cluster 7.4 is also supported by MySQL Cluster Manager, which provides an advanced command-line interface that can simplify many complex NDB Cluster management tasks. See MySQL Cluster Manager 1.4.8 User Manual, for more information.