25.2.4 What is New in MySQL NDB Cluster 8.4

• What is New in NDB Cluster 8.4

• Changes in NDB 8.x Innovation Releases

The following sections describe changes in the implementation of MySQL NDB Cluster in NDB Cluster 8.4.3, as compared to earlier release series. NDB Cluster 8.4 is available as a Development release for preview and testing of new features currently under development. For production, please use NDB 8.0; for more information, see MySQL NDB Cluster 8.0. NDB Cluster 7.6 and 7.5 are previous GA releases still supported in production, although we recommend that new deployments for production use MySQL NDB Cluster 8.0.

NDB Cluster 7.4 and 7.3 were previous GA releases which have reached their end of life; they are no longer supported or maintained.

What is New in NDB Cluster 8.4

Major changes and new features in NDB Cluster 8.4 which are likely to be of interest are listed here:

• ndbinfo transporter_details table.  The transporter_details table provides information about individual transporters used in an NDB cluster. It is otherwise similar to the ndbinfo transporters table, which provides such information in aggregate form.

  NDB 8.4.0 provides additional columns as compared to the version introduced in NDB 8.0. These new columns, along with brief dscriptions of each, are listed here:

  • sendbuffer_used_bytes: Number of bytes of signal data currently stored pending send using this transporter.

  • sendbuffer_max_used_bytes: Historical maximum number of bytes of signal data stored pending send using this transporter. Reset when the transporter connects.

  • sendbuffer_alloc_bytes: Number of bytes of send buffer currently allocated to store pending send bytes for this transporter. Send buffer memory is allocated in large blocks which may be sparsely used.

  • sendbuffer_max_alloc_bytes: Historical maximum number of bytes of send buffer allocated to store pending send bytes for this transporter.

  NDB 8.4.1 adds a type column, which displays the transport's connection type (TCP or SHM).

  See Section 25.6.17.65, “The ndbinfo transporter_details Table”, for more information.

• NDB Replication: Filtering of unused updates.  Previously, when SQL nodes performing binary logging used log_replica_updates=OFF, any replicated updates which were applied on a replica NDB cluster were sent on to the SQL nodes performing binary logging. These updates were not actually applied or used for any other purpose; this entailed unnecessary network traffic and consumption of resources.

  In NDB 8.4.0 and later, updates applied on the replica SQL node are filtered out on this node, and are no longer sent onward to any other SQL nodes. Updates that do not trigger any logging are also no longer sent by the replica.

• Per-session binary log transaction cache sizing.  NDB 8.4.3 adds the ndb_log_cache_size server system variable, which makes it possible to set the size of the transaction cache used for writing the binary log. This enables use of a large cache for logging NDB transactions, and (using binlog_cache_size) a smaller cache for logging other transactions, thus making more efficient use of resources.

• Ndb.cfg file deprecation.  Use of an Ndb.cfg file for setting the connection string for an NDB process was not well documented or supported. As of NDB 8.4.3, use of this file is now formally deprecated; you should expect support for it to be removed in a future release of MySQL Cluster.

MySQL Cluster Manager has an advanced command-line interface that can simplify many complex NDB Cluster management tasks. See MySQL Cluster Manager 8.4.3 User Manual, for more information.

Changes in NDB 8.x Innovation Releases

New features and major changes in NDB Cluster Innovation releases (8.1, 8.2, 8.3) compared with NDB 8.0 which are likely to be of interest are listed here:

• TLS for cluster node communications.  NDB Cluster 8.3 and later provides support for network communications secured by Transport Layer Security (TLS) and Internet Public Key Infrastructure (PKI) to authenticate and encrypt connections between NDB nodes, and between the NDB management server and its clients; TLS is applied to the NDB Transporter Protocol, and to the NDB Management Protocol.

  This feature uses TLS mutual authentication, in which a node's own certificate file contains the chain of trust which the node uses to validate the certificates of its peers. When TLS is enabled on the cluster, data and management nodes use TLS to perform the following tasks:

  • Mutually authenticate NDB clients and servers at the network level, preventing unprivileged access as a client or server

  • Encrypt data transfer, avoiding data eavesdropping, modification, and man-in-the-middle attacks

  Connections that use the MySQL client protocol employ MySQL user authentication, and may use TLS (including optional mutual TLS) as described elsewhere in this Manual; see Section 8.3, “Using Encrypted Connections”, for more information.

  NDB implements a new tool ndb_sign_keys which can be used to create and manage CA, certificate files, and keys. You can generate a set of keys and certificates for all nodes in a cluster with a given configuration file using ndb_sign_keys --create-key.

  Using ndb_sign_keys, a node certificate can be bound to a particular hostname, made to expire on a given date, and be associated with a given node type, so that clients are distinct from servers, and management servers from data nodes. (Every NDB TLS certificate can be used for MGM client connections.) Private keys are created in place, so that copying of files containing private keys is minimized. Both private keys and certificates are labeled as either active or pending; ndb_sign_keys also provides help with rotating keys to allow for pending keys to replace active keys before the active keys expire.

  Testing of node TLS connections can be done from the system shell using ndb_mgm client with --test-tls, or within the ndb_mgm client using the TLS INFO command. You can obtain information about certificates used by cluster nodes by checking the ndbinfo certificates table.

  To enforce a requirement for TLS, set the client option ndb-mgm-tls=strict in my.cnf on each cluster host, then set RequireTls=true in the [mgm default] section of the cluster config.ini file, and set RequireTls=true in the [ndbd default] section of the configuration file as well. Then perform a rolling restart of the cluster, restarting the management server with --reload --config-file.

  Use of TLS connections is also supported in NDB Cluster API applications in NDB 8.3 and later. For information about MGM API support, see TLS Functions. The NDB API Ndb_cluster_connection class adds configure_tls() get_tls_certificate_path() methods for setting up TLS connections by clients.

  For more information, see Section 25.6.15, “TLS Link Encryption for NDB Cluster”, as well as Section 25.5.28, “ndb_sign_keys — Create, Sign, and Manage TLS Keys and Certificates for NDB Cluster”.

• Binary log injector memory allocation.  In previous versions of NDB Cluster, when the NDB binary log injector was engaged in handling schema changes and tracking the state of the binary log, the choice of arena for allocation of memory for these purposes was forced by changing thread local pointers, thus attempting to try and catch all allocations performed during epoch processing. At the end of the epoch, those pointers were reset, arena memory was released, and the arena structures discarded; this released the memory, but also required setting it up again for the next epoch. The thread local pointer changes also introduced the risk of memory being allocated wrongly when activating functionality in different subsystems. MySQL NDB Cluster 8.3 makes the following improvements to this functionality:

  • Changes to thread local pointers are removed, and replaced by explicit arguments to provide the arena used for allocation during the epoch.

  • Re-use of the arena for next epoch, thus avoiding the need to set it up repeatedly.

  These changes are internal only but should provide a noticeable improvememnt by saving on memory release and re-allocation over successive epochs.

• NDB API primary key updates.  Previously, when using any other mechanism than NdbRecord in an attempt to update a primary key value, the NDB API returned error 4202 Set value on tuple key attribute is not allowed, even setting a value identical to the existing one. In NDB 8.1 and later, checking when performing updates by other means is handed off to the data nodes, as it was already when using NdbRecord to perform the update.

  This means that you can now perform primary key updates using NdbOperation::setValue(), NdbInterpretedCode::write_attr(), and other methods of NdbOperation and NdbInterpretedCode which set column values (including the NdbOperation methods incValue(), subValue(), the NdbInterpretedCode methods add_val(), sub_val(), and so on). This also applies to the NdbOperation interface's OperationOptions::OO_SETVALUE extension.

• Improved warnings.  Made the following improvements in warning output:

  • The maximum time allowed without any progress is now also printed in addition to local checkpoint (LCP) elapsed time.

  • When an LCP reaches WAIT_END_LCP state, table IDs and fragment IDs are undefined and so no longer relevant; for this reason, we no longer attempt to print them at that point.

  • Removed duplicated information printed when the maximum limit was reached (the same information was shown as both warning and crash information).

  In addition, we no longer print the message Validating excluded objects to the SQL node's error log every ndb_metadata_check_interval (default 60) seconds when log_error_verbosity is greater than or equal to 3 (INFO level), due ot the fact that such messages tended to flood the error log, making it difficult to examine, and using excess disk space, while not providing any additional benefit to the user.

• Pushdown joins between queries featuring very large and possibly overlapping IN() and NOT IN() lists are now handled in a correct and safe manner.

• ndbcluster plugin log messages now use SYSTEM as the log level and NDB as the subsystem for logging. This means that informational messages from the ndbcluster plugin are always printed; their verbosity can be controlled by using --ndb_extra_logging.