Added several new columns to the transporters table and counters for the counters table of the ndbinfo information database. The information provided may help in troublehsooting of transport overloads and problems with send buffer memory allocation. For more information, see the descriptions of these tables. (Bug #15935206)

To provide information which can help in assessing the current state of arbitration in a MySQL Cluster as well as in diagnosing and correcting arbitration problems, 3 new tables—membership, arbitrator_validity_detail, and arbitrator_validity_summary—have been added to the ndbinfo information database. (Bug #13336549)

When an NDB table grew to contain approximately one million rows or more per partition, it became possible to insert rows having duplicate primary or unique keys into it. In addition, primary key lookups began to fail, even when matching rows could be found in the table by other means.

This issue was introduced in MySQL Cluster NDB 7.0.36, MySQL Cluster NDB 7.1.26, and MySQL Cluster NDB 7.2.9. Signs that you may have been affected include the following:

This issue does not affect simple scans, so you can see all rows in a given table using SELECT * FROM table and similar queries that do not depend on a primary or unique key.

Upgrading to or downgrading from an affected release can be troublesome if there are rows with duplicate primary or unique keys in the table; such rows should be merged, but the best means of doing so is application dependent.

In addition, since the key operations themselves are faulty, a merge can be difficult to achieve without taking the MySQL Cluster offline, and it may be necessary to dump, purge, process, and reload the data. Depending on the circumstances, you may want or need to process the dump with an external application, or merely to reload the dump while ignoring duplicates if the result is acceptable.

Another possibility is to copy the data into another table without the original table' unique key constraints or primary key (recall that CREATE TABLE t2 SELECT * FROM t1 does not by default copy t1's primary or unique key definitions to t2). Following this, you can remove the duplicates from the copy, then add back the unique constraints and primary key definitions. Once the copy is in the desired state, you can either drop the original table and rename the copy, or make a new dump (which can be loaded later) from the copy. (Bug #16023068, Bug #67928)

The management client command ALL REPORT BackupStatus failed with an error when used with data nodes having multiple LQH worker threads (ndbmtd data nodes). The issue did not effect the node_id REPORT BackupStatus form of this command. (Bug #15908907)

The multithreaded job scheduler could be suspended prematurely when there were insufficient free job buffers to allow the threads to continue. The general rule in the job thread is that any queued messages should be sent before the thread is allowed to suspend itself, which guarantees that no other threads or API clients are kept waiting for operations which have already completed. However, the number of messages in the queue was specified incorrectly, leading to increased latency in delivering signals, sluggish response, or otherwise suboptimal performance. (Bug #15908684)

The setting for the DefaultOperationRedoProblemAction API node configuration parameter was ignored, and the default value used instead. (Bug #15855588)

Node failure during the dropping of a table could lead to the node hanging when attempting to restart. (Bug #14787522)

The recently added LCP fragment scan watchdog occasionally reported problems with LCP fragment scans having very high table id, fragment id, and row count values.

This was due to the watchdog not accounting for the time spent draining the backup buffer used to buffer rows before writing to the fragment checkpoint file.

Now, in the final stage of an LCP fragment scan, the watchdog switches from monitoring rows scanned to monitoring the buffer size in bytes. The buffer size should decrease as data is written to the file, after which the file should be promptly closed. (Bug #14680057)

During an online upgrade, certain SQL statements could cause the server to hang, resulting in the error Got error 4012 'Request ndbd time-out, maybe due to high load or communication problems' from NDBCLUSTER. (Bug #14702377)

Job buffers act as the internal queues for work requests (signals) between block threads in ndbmtd and could be exhausted if too many signals are sent to a block thread.

Performing pushed joins in the DBSPJ kernel block can execute multiple branches of the query tree in parallel, which means that the number of signals being sent can increase as more branches are executed. If DBSPJ execution cannot be completed before the job buffers are filled, the data node can fail.

This problem could be identified by multiple instances of the message sleeploop 10!! in the cluster out log, possibly followed by job buffer full. If the job buffers overflowed more gradually, there could also be failures due to error 1205 (Lock wait timeout exceeded), shutdowns initiated by the watchdog timer, or other timeout related errors. These were due to the slowdown caused by the 'sleeploop'.

Normally up to a 1:4 fanout ratio between consumed and produced signals is permitted. However, since there can be a potentially unlimited number of rows returned from the scan (and multiple scans of this type executing in parallel), any ratio greater 1:1 in such cases makes it possible to overflow the job buffers.

The fix for this issue defers any lookup child which otherwise would have been executed in parallel with another is deferred, to resume when its parallel child completes one of its own requests. This restricts the fanout ratio for bushy scan-lookup joins to 1:1. (Bug #14709490)

References: See also Bug #14648712.

Under certain rare circumstances, MySQL Cluster data nodes could crash in conjunction with a configuration change on the data nodes from a single-threaded to a multi-threaded transaction coordinator (using the ThreadConfig configuration parameter for ndbmtd). The problem occurred when a mysqld that had been started prior to the change was shut down following the rolling restart of the data nodes required to effect the configuration change. (Bug #14609774)

Added 3 new columns to the transporters table in the ndbinfo database. The remote_address, bytes_sent, and bytes_received columns help to provide an overview of data transfer across the transporter links in a MySQL Cluster. This information can be useful in verifying system balance, partitioning, and front-end server load balancing; it may also be of help when diagnosing network problems arising from link saturation, hardware faults, or other causes. (Bug #14685458)

Data node logs now provide tracking information about arbitrations, including which nodes have assumed the arbitrator role and at what times. (Bug #11761263, Bug #53736)

A slow filesystem during local checkpointing could exert undue pressure on DBDIH kernel block file page buffers, which in turn could lead to a data node crash when these were exhausted. This fix limits the number of table definition updates that DBDIH can issue concurrently. (Bug #14828998)

The management server process, when started with --config-cache=FALSE, could sometimes hang during shutdown. (Bug #14730537)

The output from ndb_config --configinfo now contains the same information as that from ndb_config --configinfo --xml, including explicit indicators for parameters that do not require restarting a data node with --initial to take effect. In addition, ndb_config indicated incorrectly that the LogLevelCheckpoint data node configuration parameter requires an initial node restart to take effect, when in fact it does not; this error was also present in the MySQL Cluster documentation, where it has also been corrected. (Bug #14671934)

Receiver threads could wait unnecessarily to process incomplete signals, greatly reducing performance of ndbmtd. (Bug #14525521)

Concurrent ALTER TABLE with other DML statements on the same NDB table returned Got error -1 'Unknown error code' from NDBCLUSTER. (Bug #14578595)

On platforms where epoll was not available, setting multiple receiver threads with the ThreadConfig parameter caused ndbmtd to fail. (Bug #14524939)

CPU consumption peaked several seconds after the forced termination an NDB client application due to the fact that the DBTC kernel block waited for any open transactions owned by the disconnected API client to be terminated in a busy loop, and did not break between checks for the correct state. (Bug #14550056)

Setting BackupMaxWriteSize to a very large value as compared with DiskCheckpointSpeed caused excessive writes to disk and CPU usage. (Bug #14472648)

Added the --connect-retries and --connect-delay startup options for ndbd and ndbmtd. --connect-retries (default 12) controls how many times the data node tries to connect to a management server before giving up; setting it to -1 means that the data node never stops trying to make contact. --connect-delay sets the number of seconds to wait between retries; the default is 5. (Bug #14329309, Bug #66550)

It was possible in some cases for two transactions to try to drop tables at the same time. If the master node failed while one of these operations was still pending, this could lead either to additional node failures (and cluster shutdown) or to new dictionary operations being blocked. This issue is addressed by ensuring that the master will reject requests to start or stop a transaction while there are outstanding dictionary takeover requests. In addition, table-drop operations now correctly signal when complete, as the DBDICT kernel block could not confirm node takeovers while such operations were still marked as pending completion. (Bug #14190114)

Following a failed ALTER TABLE ... REORGANIZE PARTITION statement, a subsequent execution of this statement after adding new data nodes caused a failure in the DBDIH kernel block which led to an unplanned shutdown of the cluster. (Bug #14220269)

The DBSPJ kernel block had no information about which tables or indexes actually existed, or which had been modified or dropped, since execution of a given query began. Thus, DBSPJ might submit dictionary requests for nonexistent tables or versions of tables, which could cause a crash in the DBDIH kernel block.

This fix introduces a simplified dictionary into the DBSPJ kernel block such that DBSPJ can now check reliably for the existence of a particular table or version of a table on which it is about to request an operation. (Bug #14103195)

When using ndbmtd and performing joins, data nodes could fail where ndbmtd processes were configured to use a large number of local query handler threads (as set by the ThreadConfig configuration parameter), the tables accessed by the join had a large number of partitions, or both. (Bug #13799800, Bug #14143553)

When reloading the redo log during a node or system restart, and with NoOfFragmentLogFiles greater than or equal to 42, it was possible for metadata to be read for the wrong file (or files). Thus, the node or nodes involved could try to reload the wrong set of data. (Bug #14389746)

If the Transaction Coordinator aborted a transaction in the “prepared” state, this could cause a resource leak. (Bug #14208924)

When attempting to connect using a socket with a timeout, it was possible (if the timeout was exceeded) for the socket not to be set back to blocking. (Bug #14107173)

In some circumstances, transactions could be lost during an online upgrade. (Bug #13834481)

When an NDB table was created during a data node restart, the operation was rolled back in the NDB engine, but not on the SQL node where it was executed. This was due to the table .FRM files not being cleaned up following the operation that was rolled back by NDB. Now in such cases these files are removed. (Bug #13824846)

Attempting to add both a column and an index on that column in the same online ALTER TABLE statement caused mysqld to fail. Although this issue affected only the mysqld shipped with MySQL Cluster, the table named in the ALTER TABLE could use any storage engine for which online operations are supported. (Bug #12755722)

Cluster API: When an NDB API application called NdbScanOperation::nextResult() again after the previous call had returned end-of-file (return code 1), a transaction object was leaked. Now when this happens, NDB returns error code 4210 (Ndb sent more info than length specified); previouslyu in such cases, -1 was returned. In addition, the extra transaction object associated with the scan is freed, by returning it to the transaction coordinator's idle list. (Bug #11748194)

DUMP 2303 in the ndb_mgm client now includes the status of the single fragment scan record reserved for a local checkpoint. (Bug #13986128)

A shortage of scan fragment records in DBTC resulted in a leak of concurrent scan table records and key operation records. (Bug #13966723)

Important Change: The ALTER ONLINE TABLE ... REORGANIZE PARTITION statement can be used to create new table partitions after new empty nodes have been added to a MySQL Cluster. Usually, the number of partitions to create is determined automatically, such that, if no new partitions are required, then none are created. This behavior can be overridden by creating the original table using the MAX_ROWS option, which indicates that extra partitions should be created to store a large number of rows. However, in this case ALTER ONLINE TABLE ... REORGANIZE PARTITION simply uses the MAX_ROWS value specified in the original CREATE TABLE statement to determine the number of partitions required; since this value remains constant, so does the number of partitions, and so no new ones are created. This means that the table is not rebalanced, and the new data nodes remain empty.

To solve this problem, support is added for ALTER ONLINE TABLE ... MAX_ROWS=newvalue, where newvalue is greater than the value used with MAX_ROWS in the original CREATE TABLE statement. This larger MAX_ROWS value implies that more partitions are required; these are allocated on the new data nodes, which restores the balanced distribution of the table data.

For more information, see ALTER TABLE Syntax, and Adding MySQL Cluster Data Nodes Online. (Bug #13714648)

In some cases, restarting data nodes spent a very long time in Start Phase 101, when API nodes must connect to the starting node (using NdbEventOperation), when the API nodes trying to connect failed in a live-lock scenario. This connection process uses a handshake during which a small number of messages are exchanged, with a timeout used to detect failures during the handshake.

Prior to this fix, this timeout was set such that, if one API node encountered the timeout, all other nodes connecting would do the same. The fix also decreases this timeout. This issue (and the effects of the fix) are most likely to be observed on relatively large configurations having 10 or more data nodes and 200 or more API nodes. (Bug #13825163)

ALTER ONLINE TABLE failed when a DEFAULT option was used. (Bug #13830980)

When the --skip-config-cache and --initial options were used together, ndb_mgmd failed to start. (Bug #13857301)

ndbmtd failed to restart when the size of a table definition exceeded 32K.

(The size of a table definition is dependent upon a number of factors, but in general the 32K limit is encountered when a table has 250 to 300 columns.) (Bug #13824773)

An initial start using ndbmtd could sometimes hang. This was due to a state which occurred when several threads tried to flush a socket buffer to a remote node. In such cases, to minimize flushing of socket buffers, only one thread actually performs the send, on behalf of all threads. However, it was possible in certain cases for there to be data in the socket buffer waiting to be sent with no thread ever being chosen to perform the send. (Bug #13809781)

When trying to use ndb_size.pl --hostname=host:port to connect to a MySQL server running on a nonstandard port, the port argument was ignored. (Bug #13364905, Bug #62635)

Important Change: A number of changes have been made in the configuration of transporter send buffers.

(Bug #13633845, Bug #11760629, Bug #53053)

Setting insert_id had no effect on the auto-increment counter for NDB tables. (Bug #13731134)

A very large value for BackupWriteSize, as compared to BackupMaxWriteSize, BackupDataBufferSize, or BackupLogBufferSize, could cause a local checkpoint or backup to hang. (Bug #13613344)

Queries using LIKE ... ESCAPE on NDB tables failed when pushed down to the data nodes. Such queries are no longer pushed down, regardless of the value of engine_condition_pushdown. (Bug #13604447, Bug #61064)

Several instances in the NDB code affecting the operation of multi-threaded data nodes, where SendBufferMemory was associated with a specific thread for an unnecessarily long time, have been identified and fixed, by minimizing the time that any of these buffers can be held exclusively by a given thread (send buffer memory being critical to operation of the entire node). (Bug #13618181)

To avoid TCP transporter overload, an overload flag is kept in the NDB kernel for each data node; this flag is used to abort key requests if needed, yielding error 1218 Send Buffers overloaded in NDB kernel in such cases. Scans can also put significant pressure on transporters, especially where scans with a high degree of parallelism are executed in a configuration with relatively small send buffers. However, in these cases, overload flags were not checked, which could lead to node failures due to send buffer exhaustion. Now, overload flags are checked by scans, and in cases where returning sufficient rows to match the batch size (--ndb-batch-size server option) would cause an overload, the number of rows is limited to what can be accommodated by the send buffer.

See also Configuring MySQL Cluster Send Buffer Parameters. (Bug #13602508)

A data node crashed when more than 16G fixed-size memory was allocated by DBTUP to one fragment (because the DBACC kernel block was not prepared to accept values greater than 32 bits from it, leading to an overflow). Now in such cases, the data node returns Error 889 Table fragment fixed data reference has reached maximum possible value.... When this happens, you can work around the problem by increasing the number of partitions used by the table (such as by using the MAXROWS option with CREATE TABLE). (Bug #13637411)

References: See also Bug #11747870, Bug #34348.

A SELECT from an NDB table using LIKE with a multi-byte column (such as utf8) did not return the correct result when engine_condition_pushdown was enabled. (Bug #13579318, Bug #64039)

References: See also Bug #13608135.

A node failure and recovery while performing a scan on more than 32 partitions led to additional node failures during node takeover. (Bug #13528976)

The --skip-config-cache option now causes ndb_mgmd to skip checking for the configuration directory, and thus to skip creating it in the event that it does not exist. (Bug #13428853)

Accessing a table having a BLOB column but no primary key following a restart of the SQL node failed with Error 1 (Unknown error code). (Bug #13563280)

At the beginning of a local checkpoint, each data node marks its local tables with a “to be checkpointed” flag. A failure of the master node during this process could cause either the LCP to hang, or one or more data nodes to be forcibly shut down. (Bug #13436481)

A node failure while a ANALYZE TABLE statement was executing resulted in a hung connection (and the user was not informed of any error that would cause this to happen). (Bug #13416603)

References: See also Bug #13407848.

Added the MinFreePct data node configuration parameter, which specifies a percentage of data node resources to hold in reserve for restarts. The resources monitored are DataMemory, IndexMemory, and any per-table MAX_ROWS settings (see CREATE TABLE Syntax). The default value of MinFreePct is 5, which means that 5% from each these resources is now set aside for restarts. (Bug #13436216)

The BatchSize and BatchByteSize configuration parameters, used to control the maximum sizes of result batches, are defined as integers. However, the values used to store these were incorrectly interpreted as numbers of bytes in the NDB kernel. This caused the DBLQH kernel block to fail to detect when the specified BatchByteSize was consumed.

In addition, the DBSPJ kernel block could miscalculate statistics for adaptive parallelism. (Bug #13355055)

Because the log event buffer used internally by data nodes was circular, periodic events such as statistics events caused it to be overwritten too quickly. Now the buffer is partitioned by log event category, and its default size has been increased from 4K to 8K. (Bug #13394771)

Previously, forcing simultaneously the shutdown of multiple data nodes using SHUTDOWN -F in the ndb_mgm management client could cause the entire cluster to fail. Now in such cases, any such nodes are forced to abort immediately. (Bug #12928429)

A SubscriberNodeIdUndefined error was previously unhandled, resulting in a data node crash, but is now handled by NDB Error 1429, Subscriber node undefined in SubStartReq. (Bug #12598496)

SELECT statements using LIKE CONCAT(...) OR LIKE CONCAT(...) in the WHERE clause returned incorrect results when run against NDB tables. (Bug #11765142, Bug #58073)

Introduced the CrashOnCorruptedTuple data node configuration parameter. When enabled, this parameter causes data nodes to handle corrupted tuples in a fail-fast manner—in other words, whenever the data node detects a corrupted tuple, it forcibly shuts down if CrashOnCorruptedTuple is enabled. For backward compatibility, this parameter is disabled by default. (Bug #12598636)

Added the ThreadConfig data node configuration parameter to enable control of multiple threads and CPUs when using ndbmtd, by assigning threads of one or more specified types to execute on one or more CPUs. This can provide more precise and flexible control over multiple threads than can be obtained using the LockExecuteThreadToCPU parameter. (Bug #11795581)

Added the ndbinfo_select_all utility.

When a failure of multiple data nodes during a local checkpoint (LCP) that took a long time to complete included the node designated as master, any new data nodes attempting to start before all ongoing LCPs were completed later crashed. This was due to the fact that node takeover by the new master cannot be completed until there are no pending local checkpoints. Long-running LCPs such as those which triggered this issue can occur when fragment sizes are sufficiently large (see MySQL Cluster Nodes, Node Groups, Replicas, and Partitions, for more information). Now in such cases, data nodes (other than the new master) are kept from restarting until the takeover is complete. (Bug #13323589)

When deleting from multiple tables using a unique key in the WHERE condition, the wrong rows were deleted. In addition, UPDATE triggers failed when rows were changed by deleting from or updating multiple tables. (Bug #12718336, Bug #61705, Bug #12728221)

Shutting down a mysqld while under load caused the spurious error messages Opening ndb_binlog_index: killed and Unable to lock table ndb_binlog_index to be written in the cluster log. (Bug #11930428)

Cluster API: When more than 32KB of data must be sent in a single signal using the NDB API, the data is split across 2 or more signals each of which is smaller than 32kB, and these are then reassembled back into the original, full-length signal by the receiver. Such fragmented signals are used for some scan requests, as well as for SPJ QueryOperation requests. However, extra (spurious) signals could sometimes be sent when using fragmented signals, causing errors on the receiver; these implementation artifacts have now been eliminated. (Bug #13087016)

It is now possible to filter the output from ndb_config so that it displays only system, data node, or connection parameters and values, using one of the options --system, --nodes, or --connections, respectively. In addition, it is now possible to specify from which data node the configuration data is obtained, using the --config_from_node option that is added in this release.

For more information, see ndb_config — Extract MySQL Cluster Configuration Information. (Bug #11766870)

Incompatible Change; Cluster API: Restarting a machine hosting data nodes, SQL nodes, or both, caused such nodes when restarting to time out while trying to obtain node IDs.

As part of the fix for this issue, the behavior and default values for the NDB API Ndb_cluster_connection::connect() method have been improved. Due to these changes, the version number for the included NDB client library (libndbclient.so) has been increased from 4.0.0 to 5.0.0. For NDB API applications, this means that as part of any upgrade, you must do both of the following:

Also in connection with this issue, the default value for each of the two mysqld options --ndb-wait-connected and --ndb-wait-setup has been increased to 30 seconds (from 0 and 15, respectively). In addition, a hard-coded 30-second delay was removed, so that the value of --ndb-wait-connected is now handled correctly in all cases. (Bug #12543299)

When replicating DML statements with IGNORE between clusters, the number of operations that failed due to nonexistent keys was expected to be no greater than the number of defined operations of any single type. Because the slave SQL thread defines operations of multiple types in batches together, code which relied on this assumption could cause mysqld to fail. (Bug #12859831)

The maximum effective value for the OverloadLimit configuration parameter was limited by the value of SendBufferMemory. Now the value set for OverloadLimit is used correctly, up to this parameter's stated maximum (4G). (Bug #12712109)

AUTO_INCREMENT values were not set correctly for INSERT IGNORE statements affecting NDB tables. This could lead such statements to fail with Got error 4350 'Transaction already aborted' from NDBCLUSTER when inserting multiple rows containing duplicate values. (Bug #11755237, Bug #46985)

When failure handling of an API node takes longer than 300 seconds, extra debug information is included in the resulting output. In cases where the API node's node ID was greater than 48, these extra debug messages could lead to a crash, and confuing output otherwise. This was due to an attempt to provide information specific to data nodes for API nodes as well. (Bug #62208)

In rare cases, a series of node restarts and crashes during restarts could lead to errors while reading the redo log. (Bug #62206)

Setting IndexMemory or sometimes DataMemory to 2 GB or higher could lead to data node failures under some conditions. (Bug #12873640)

Added the MaxDMLOperationsPerTransaction data node configuration parameter, which can be used to limit the number of DML operations used by a transaction; if the transaction requires more than this many DML operations, the transaction is aborted. (Bug #12589613)

Restarting a mysqld during a rolling upgrade with data nodes running a mix of old and new versions of the MySQL Cluster software caused the mysqld to run in read-only mode. (Bug #12651364, Bug #61498)

When global checkpoint indexes were written with no intervening end-of-file or megabyte border markers, this could sometimes lead to a situation in which the end of the redo log was mistakenly regarded as being between these GCIs, so that if the restart of a data node took place before the start of the next redo log was overwritten, the node encountered an Error while reading the REDO log. (Bug #12653993, Bug #61500)

References: See also Bug #56961.

Under certain rare circumstances, a data node process could fail with Signal 11 during a restart. This was due to uninitialized variables in the QMGR kernel block. (Bug #12586190)

Error reporting has been improved for cases in which API nodes are unable to connect due to apparent unavailability of node IDs. (Bug #12598398)

Error messages for Failed to convert connection transporter registration problems were inspecific. (Bug #12589691)

Multiple management servers were unable to detect one another until all nodes had fully started. As part of the fix for this issue, two new status values RESUME and CONNECTED can be reported for management nodes in the output of the ndb_mgm client SHOW command (see Commands in the MySQL Cluster Management Client). Two corresponding status values NDB_MGM_NODE_STATUS_RESUME and NDB_MGM_NODE_STATUS_CONNECTED are also added to the list of possible values for an ndb_mgm_node_status data structure in the MGM API. (Bug #12352191, Bug #48301)

Handling of the MaxNoOfTables and MaxNoOfAttributes configuration parameters was not consistent in all parts of the NDB kernel, and were only strictly enforced by the DBDICT and SUMA kernel blocks. This could lead to problems when tables could be created but not replicated. Now these parameters are treated by SUMA and DBDICT as suggested maximums rather than hard limits, as they are elsewhere in the NDB kernel. (Bug #61684)

It was not possible to shut down a management node while one or more data nodes were stopped (for whatever reason). This issue was a regression introduced in MySQL Cluster NDB 7.0.24 and MySQL Cluster NDB 7.1.13. (Bug #61607)

References: See also Bug #61147.

Cluster API: Within a transaction, after creating, executing, and closing a scan, calling NdbTransaction::refresh() after creating and executing but not closing a second scan caused the application to crash. (Bug #12646659)

Cluster API: Applications that included the header file ndb_logevent.h could not be built using the Microsoft Visual Studio C compiler or the Oracle (Sun) Studio C compiler due to empty struct definitions. (Bug #12678971)

The internal Ndb_getinaddr() function has been rewritten to use getaddrinfo() instead of my_gethostbyname_r() (which is removed in a later version of the MySQL Server). (Bug #12542120)

mysql_upgrade failed when performing an online upgrade from MySQL Cluster NDB 7.1.8 or an earlier release to MySQL Cluster NDB 7.1.9 or later in which the SQL nodes were upgraded before the data nodes. This issue could occur during any online upgrade or downgrade where one or more ndbinfo tables had more, fewer, or differing columns between the two versions, and when the data nodes were not upgraded before the SQL nodes.

For more information, see Upgrade and downgrade compatibility: MySQL Cluster NDB 7.x. (Bug #11885602, Bug #12581895, Bug #12581954)

Two unused test files in storage/ndb/test/sql contained incorrect versions of the GNU Lesser General Public License. The files and the directory containing them have been removed. (Bug #11810156)

References: See also Bug #11810224.

Error 1302 gave the wrong error message (Out of backup record). This has been corrected to A backup is already running. (Bug #11793592)

Renaming a table having BLOB or TEXT columns (or both) to another database caused the SQL node to crash, and the table to become inaccessible afterwards. (Bug #60484)

In ndbmtd, a node connection event is detected by a CMVMI thread which sends a CONNECT_REP signal to the QMGR kernel block. In a few isolated circumstances, a signal might be transferred to QMGR directly by the NDB transporter before the CONNECT_REP signal actually arrived. This resulted in reports in the error log with status Temporary error, restart node, and the message Internal program error. (Bug #61025)

Under heavy loads with many concurrent inserts, temporary failures in transactions could occur (and were misreported as being due to NDB Error 899 Rowid already allocated). As part of the fix for this issue, NDB Error 899 has been reclassified as an internal error, rather than as a temporary transaction error. (Bug #56051, Bug #11763354)

When using two management servers, issuing in an ndb_mgm client connected to one management server a STOP command for stopping the other management server caused Error 2002 (Stop failed ... Send to process or receive failed.: Permanent error: Application error), even though the STOP command actually succeeded, and the second ndb_mgmd was shut down. (Bug #61147)

Disk Data: Accounting for MaxNoOfOpenFiles was incorrect with regard to data files in MySQL Cluster Disk Data tablespaces. This could lead to a crash when MaxNoOfOpenFiles was exceeded. (Bug #12581213)

It is now possible to add data nodes online to a running MySQL Cluster without performing a rolling restart of the cluster or starting data node processes with the --nowait-nodes option. This can be done by setting Nodegroup = 65536 in the config.ini file for any data nodes that should be started at a later time, when first starting the cluster. (It was possible to set NodeGroup to this value previously, but the management server failed to start.)

As part of this fix, a new data node configuration parameter StartNoNodeGroupTimeout has been added. When the management server sees that there are data nodes with no node group (that is, nodes for which Nodegroup = 65536), it waits StartNoNodeGroupTimeout milliseconds before treating these nodes as though they were listed with the --nowait-nodes option, and proceeds to start.

For more information, see Adding MySQL Cluster Data Nodes Online. (Bug #11766167, Bug #59213)

A config_generation column has been added to the nodes table of the ndbinfo database. By checking this column, it is now possible to determine which version or versions of the MySQL Cluster configuration file are in effect on the data nodes. This information can be especially useful when performing a rolling restart of the cluster to update its configuration.

Cluster API: A unique index operation is executed in two steps: a lookup on an index table, and an operation on the base table. When the operation on the base table failed, while being executed in a batch with other operations that succeeded, this could lead to a hanging execute, eventually timing out with Error 4012 (Request ndbd time-out, maybe due to high load or communication problems). (Bug #12315582)

A memory leak in LGMAN, that leaked 8 bytes of log buffer memory per 32k written, was introduced in MySQL Cluster NDB 7.0.9, effecting all MySQL Cluster NDB 7.1 releases as well as MySQL Cluster NDB 7.0.9 and later MySQL Cluster NDB 7.0 releases. (For example, when 128MB log buffer memory was used, it was exhausted after writing 512GB to the undo log.) This led to a GCP stop and data node failure. (Bug #60946)

References: This bug was introduced by Bug #47966.

When using ndbmtd, a MySQL Cluster configured with 32 data nodes failed to start correctly. (Bug #60943)

When performing a TUP scan with locks in parallel, and with a highly concurrent load of inserts and deletions, the scan could sometimes fail to notice that a record had moved while waiting to acquire a lock on it, and so read the wrong record. During node recovery, this could lead to a crash of a node that was copying data to the node being started, and a possible forced shutdown of the cluster.

Cluster API: Performing interpreted operations using a unique index did not work correctly, because the interpret bit was kept when sending the lookup to the index table.

Improved scaling of ordered index scans performance by removing a hard-coded limit (MAX_PARALLEL_INDEX_SCANS_PER_FRAG) and making the number of TUP or TUX scans per fragment configurable by adding the MaxParallelScansPerFragment data node configuration parameter. (Bug #11769048)

Important Change: Formerly, the --ndb-cluster-connection-pool server option set a status variable as well as a system variable. The status variable has been removed as redundant. (Bug #60119)

A scan with a pushed condition (filter) using the CommittedRead lock mode could hang for a short interval when it was aborted when just as it had decided to send a batch. (Bug #11932525)

When aborting a multi-read range scan exactly as it was changing ranges in the local query handler, LQH could fail to detect it, leaving the scan hanging. (Bug #11929643)

Schema distribution did not take place for tables converted from another storage engine to NDB using ALTER TABLE; this meant that such tables were not always visible to all SQL nodes attached to the cluster. (Bug #11894966)

A GCI value inserted by ndb_restore --restore_epoch into the ndb_apply_status table was actually 1 less than the correct value. (Bug #11885852)

Disk Data: Limits imposed by the size of SharedGlobalMemory were not always enforced consistently with regard to Disk Data undo buffers and log files. This could sometimes cause a CREATE LOGFILE GROUP or ALTER LOGFILE GROUP statement to fail for no apparent reason, or cause the log file group specified by InitialLogFileGroup not to be created when starting the cluster. (Bug #57317)

Disk Data: The INFORMATION_SCHEMA.TABLES table now provides disk usage as well as memory usage information for Disk Data tables. Also, INFORMATION_SCHEMA.PARTITIONS, formerly did not show any statistics for NDB tables. Now the TABLE_ROWS, AVG_ROW_LENGTH, DATA_LENGTH, MAX_DATA_LENGTH, and DATA_FREE columns contain correct information for the table's partitions.

A new --rewrite-database option is added for ndb_restore, which makes it possible to restore to a database having a different name from that of the database in the backup.

For more information, see ndb_restore — Restore a MySQL Cluster Backup. (Bug #54327)

Made it possible to enable multi-threaded building of ordered indexes during initial restarts, using the new TwoPassInitialNodeRestartCopy data node configuration parameter.

Added the --lossy-conversions option for ndb_restore, which makes it possible to enable attribute demotion when restoring a MySQL Cluster from an NDB native backup.

For additional information about type conversions currently supported by MySQL Cluster for attribute promotion and demotion, see Replication of Columns Having Different Data Types.

The NDB kernel now implements a number of statistical counters relating to actions performed by or affecting Ndb objects, such as starting, closing, or aborting transactions; primary key and unique key operations; table, range, and pruned scans; blocked threads waiting for various operations to complete; and data and events sent and received by NDBCLUSTER. These NDB API counters are incremented inside the NDB kernel whenever NDB API calls are made or data is sent to or received by the data nodes. mysqld exposes these counters as system status variables; their values can be read in the output of SHOW STATUS, or by querying the SESSION_STATUS or GLOBAL_STATUS table in the INFORMATION_SCHEMA database. By comparing the values of these status variables prior to and following the execution of SQL statements that act on NDB tables, you can observe the corresponding actions taken on the NDB API level, which can be beneficial for monitoring and performance tuning of MySQL Cluster.

For more information, see NDB API Statistics Counters and Variables, as well as MySQL Cluster Status Variables.

When a CREATE TABLE statement failed due to NDB error 1224 (Too many fragments), it was not possible to create the table afterward unless either it had no ordered indexes, or a DROP TABLE statement was issued first, even if the subsequent CREATE TABLE was valid and should otherwise have succeeded. (Bug #59756)

References: See also Bug #59751.

When a query used multiple references to or instances of the same physical tables, NDB failed to recognize these multiple instances as different tables; in such a case, NDB could incorrectly use condition pushdown on a condition referring to these other instances to be pushed to the data nodes, even though the condition should have been rejected as unpushable, leading to invalid results. (Bug #58791)

Successive queries on the counters table from the same SQL node returned unchanging results. To fix this issue, and to prevent similar issues from occurring in the future, ndbinfo tables are now excluded from the query cache. (Bug #59831)

This issue affects all previous MySQL Cluster NDB 7.1 releases. (Bug #60045)

When attempting to create a table on a MySQL Cluster with many standby data nodes (setting Nodegroup=65536 in config.ini for the nodes that should wait, starting the nodes that should start immediately with the --nowait-nodes option, and using the CREATE TABLE statement's MAX_ROWS option), mysqld miscalculated the number of fragments to use. This caused the CREATE TABLE to fail.

(Bug #59751)

The logic used in determining whether to collapse a range to a simple equality was faulty. In certain cases, this could cause NDB to treat a range as if it were a primary key lookup when determining the query plan to be used. Although this did not affect the actual result returned by the query, it could in such cases result in inefficient execution of queries due to the use of an inappropriate query plan. (Bug #59517)

NDB sometimes treated a simple (not unique) ordered index as unique. (Bug #59519)

ndb_restore --rebuild-indexes caused multi-threaded index building to occur on the master node only. (Bug #59920)

Cluster API: When an NDBAPI client application was waiting for more scan results after calling NdbScanOperation::nextResult(), the calling thread sometimes woke up even if no new batches for any fragment had arrived, which was unnecessary, and which could have a negative impact on the application's performance. (Bug #52298)

Cluster API: When calling NdbEventOperation::execute() during a node restart, it was possible to get a spurious error 711 (System busy with node restart, schema operations not allowed when a node is starting). (Bug #59723)

Important Change: The following changes have been made with regard to the TimeBetweenEpochsTimeout data node configuration parameter:

For more information, see Disk Data and GCP Stop errors. (Bug #58383)

Added the --skip-broken-objects option for ndb_restore. This option causes ndb_restore to ignore tables corrupted due to missing blob parts tables, and to continue reading from the backup file and restoring the remaining tables. (Bug #54613)

References: See also Bug #51652.

Made it possible to exercise more direct control over handling of timeouts occurring when trying to flush redo logs to disk using two new data node configuration parameters RedoOverCommitCounter and RedoOverCommitLimit, as well as the new API node configuration parameter DefaultOperationRedoProblemAction, all added in this release. Now, when such timeouts occur more than a specified number of times for the flushing of a given redo log, any transactions that were to be written are instead aborted, and the operations contained in those transactions can be either re-tried or themselves aborted.

For more information, see Redo log over-commit handling.

Cluster API: It is now possible to stop or restart a node even while other nodes are starting, using the MGM API ndb_mgm_stop4() or ndb_mgm_restart4() function, respectively, with the force parameter set to 1. (Bug #58451)

References: See also Bug #58319.

Cluster API: In some circumstances, very large BLOB read and write operations in MySQL Cluster applications can cause excessive resource usage and even exhaustion of memory. To fix this issue and to provide increased stability when performing such operations, it is now possible to set limits on the volume of BLOB data to be read or written within a given transaction in such a way that when these limits are exceeded, the current transaction implicitly executes any accumulated operations. This avoids an excessive buildup of pending data which can result in resource exhaustion in the NDB kernel. The limits on the amount of data to be read and on the amount of data to be written before this execution takes place can be configured separately. (In other words, it is now possible in MySQL Cluster to specify read batching and write batching that is specific to BLOB data.) These limits can be configured either on the NDB API level, or in the MySQL Server.

On the NDB API level, four new methods are added to the NdbTransaction object. getMaxPendingBlobReadBytes() and setMaxPendingBlobReadBytes() can be used to get and to set, respectively, the maximum amount of BLOB data to be read that accumulates before this implicit execution is triggered. getMaxPendingBlobWriteBytes() and setMaxPendingBlobWriteBytes() can be used to get and to set, respectively, the maximum volume of BLOB data to be written that accumulates before implicit execution occurs.

For the MySQL server, two new options are added. The --ndb-blob-read-batch-bytes option sets a limit on the amount of pending BLOB data to be read before triggering implicit execution, and the --ndb-blob-write-batch-bytes option controls the amount of pending BLOB data to be written. These limits can also be set using the mysqld configuration file, or read and set within the mysql client and other MySQL client applications using the corresponding server system variables. (Bug #59113)

In some circumstances, an SQL trigger on an NDB table could read stale data. (Bug #58538)

MySQL Cluster failed to compile correctly on FreeBSD 8.1 due to misplaced #include statements. (Bug #58034)

Trying to drop an index while it was being used to perform scan updates caused data nodes to crash. (Bug #58277, Bug #57057)

A query using BETWEEN as part of a pushed-down WHERE condition could cause mysqld to hang or crash. (Bug #57735)

Queries using column IS [NOT] NULL on a table with a unique index created with USING HASH on column always returned an empty result. (Bug #57032)

Some queries of the form SELECT ... WHERE column IN (subquery) against an NDB table could cause mysqld to hang in an endless loop. (Bug #58163)

A query having multiple predicates joined by OR in the WHERE clause and which used the sort_union access method (as shown using EXPLAIN) could return duplicate rows. (Bug #58280)

With engine_condition_pushdown enabled, a query using LIKE on an ENUM column of an NDB table failed to return any results. This issue is resolved by disabling engine_condition_pushdown when performing such queries. (Bug #53360)

A row insert or update followed by a delete operation on the same row within the same transaction could in some cases lead to a buffer overflow. (Bug #59242)

References: See also Bug #56524. This bug was introduced by Bug #35208.

Data nodes no longer allocated all memory prior to being ready to exchange heartbeat and other messages with management nodes, as in NDB 6.3 and earlier versions of MySQL Cluster. This caused problems when data nodes configured with large amounts of memory failed to show as connected or showed as being in the wrong start phase in the ndb_mgm client even after making their initial connections to and fetching their configuration data from the management server. With this fix, data nodes now allocate all memory as they did in earlier MySQL Cluster versions. (Bug #57568)

When a slash character (/) was used as part of the name of an index on an NDB table, attempting to execute a TRUNCATE TABLE statement on the table failed with the error Index not found, and the table was rendered unusable. (Bug #38914)

Data nodes configured with very large amounts (multiple gigabytes) of DiskPageBufferMemory failed during startup with NDB error 2334 (Job buffer congestion). (Bug #58945)

References: See also Bug #47984.

On Windows platforms, issuing a SHUTDOWN command in the ndb_mgm client caused management processes that had been started with the --nodaemon option to exit abnormally. (Bug #59437)

The functions strncasecmp and strcasecmp were declared in ndb_global.h but never defined or used. The declarations have been removed. (Bug #58204)

When handling failures of multiple data nodes, an error in the construction of internal signals could cause the cluster's remaining nodes to crash. This issue was most likely to affect clusters with large numbers of data nodes. (Bug #58240)

The number of rows affected by a statement that used a WHERE clause having an IN condition with a value list containing a great many elements, and that deleted or updated enough rows such that NDB processed them in batches, was not computed or reported correctly. (Bug #58040)

When executing a full table scan caused by a WHERE condition using unique_key IS NULL in combination with a join, NDB failed to close the scan. (Bug #58750)

References: See also Bug #57481.

The FAIL_REP signal, used inside the NDB kernel to declare that a node has failed, now includes the node ID of the node that detected the failure. This information can be useful in debugging. (Bug #58904)

In some circumstances, it was possible for mysqld to begin a new multi-range read scan without having closed a previous one. This could lead to exhaustion of all scan operation objects, transaction objects, or lock objects (or some combination of these) in NDB, causing queries to fail with such errors as Lock wait timeout exceeded or Connect failure - out of connection objects. (Bug #57481)

References: See also Bug #58750.

During a node takeover, it was possible in some circumstances for one of the remaining nodes to send an extra transaction confirmation (LQH_TRANSCONF) signal to the DBTC kernel block, conceivably leading to a crash of the data node trying to take over as the new transaction coordinator. (Bug #58453)

Two related problems could occur with read-committed scans made in parallel with transactions combining multiple (concurrent) operations:

(Bug #59496)

Partitioning; Disk Data: When using multi-threaded data nodes, an NDB table created with a very large value for the MAX_ROWS option could—if this table was dropped and a new table with fewer partitions, but having the same table ID, was created—cause ndbmtd to crash when performing a system restart. This was because the server attempted to examine each partition whether or not it actually existed.

This issue is the same as that reported in Bug #45154, except that the current issue is specific to ndbmtd instead of ndbd. (Bug #58638)

Disk Data: Performing what should have been an online drop of a multi-column index was actually performed offline. (Bug #55618)

Disk Data: In certain cases, a race condition could occur when DROP LOGFILE GROUP removed the logfile group while a read or write of one of the effected files was in progress, which in turn could lead to a crash of the data node. (Bug #59502)

Disk Data: A race condition could sometimes be created when DROP TABLESPACE was run concurrently with a local checkpoint; this could in turn lead to a crash of the data node. (Bug #59501)

Disk Data: When at least one data node was not running, queries against the INFORMATION_SCHEMA.FILES table took an excessive length of time to complete because the MySQL server waited for responses from any stopped nodes to time out. Now, in such cases, MySQL does not attempt to contact nodes which are not known to be running. (Bug #54199)

Cluster API: It was not possible to obtain the status of nodes accurately after an attempt to stop a data node using ndb_mgm_stop() failed without returning an error. (Bug #58319)

Cluster API: Attempting to read the same value (using getValue()) more than 9000 times within the same transaction caused the transaction to hang when executed. Now when more reads are performed in this way than can be accommodated in a single transaction, the call to execute() fails with a suitable error. (Bug #58110)

Important Note: Issuing an ALL DUMP command during a rolling upgrade to MySQL Cluster NDB 7.1.9 caused the cluster to crash. (Bug #58256)

InnoDB; Packaging: The InnoDB plugin was not included in MySQL Cluster RPM packages. (Bug #58283)

References: See also Bug #54912.

Important Change; InnoDB: Building the MySQL Server with the InnoDB plugin is now supported when building MySQL Cluster. For more information, see MySQL Cluster Installation and Upgrades. (Bug #54912)

References: See also Bug #58283.

Important Change: ndbd now bypasses use of Non-Uniform Memory Access support on Linux hosts by default. If your system supports NUMA, you can enable it and override ndbd use of interleaving by setting the Numa data node configuration parameter which is added in this release. See Defining Data Nodes: Realtime Performance Parameters, for more information. (Bug #57807)

Important Change: The Id configuration parameter used with MySQL Cluster management, data, and API nodes (including SQL nodes) is now deprecated, and the NodeId parameter (long available as a synonym for Id when configuring these types of nodes) should be used instead. Id continues to be supported for reasons of backward compatibility, but now generates a warning when used with these types of nodes, and is subject to removal in a future release of MySQL Cluster.

This change affects the name of the configuration parameter only, establishing a clear preference for NodeId over Id in the [mgmd], [ndbd], [mysql], and [api] sections of the MySQL Cluster global configuration (config.ini) file. The behavior of unique identifiers for management, data, and SQL and API nodes in MySQL Cluster has not otherwise been altered.

The Id parameter as used in the [computer] section of the MySQL Cluster global configuration file is not affected by this change.

A new diskpagebuffer table, providing statistics on disk page buffer usage by Disk Data tables, is added to the ndbinfo information database. These statistics can be used to monitor performance of reads and writes on Disk Data tables, and to assist in the tuning of related parameters such as DiskPageBufferMemory.

Packaging: MySQL Cluster RPM distributions did not include a shared-compat RPM for the MySQL Server, which meant that MySQL applications depending on libmysqlclient.so.15 (MySQL 5.0 and earlier) no longer worked. (Bug #38596)

The method for calculating table schema versions used by schema transactions did not follow the established rules for recording schemas used in the P0.SchemaLog file. (Bug #57897)

References: See also Bug #57896.

Specifying a WHERE clause of the form range1 OR range2 when selecting from an NDB table having a primary key on multiple columns could result in Error 4259 Invalid set of range scan bounds if range2 started exactly where range1 ended and the primary key definition declared the columns in a different order relative to the order in the table's column list. (Such a query should simply return all rows in the table, since any expression value < constant OR value >= constant is always true.)

Example.  Suppose t is an NDB table defined by the following CREATE TABLE statement:

CREATE TABLE t (a, b, PRIMARY KEY (b, a)) ENGINE NDB;

This issue could then be triggered by a query such as this one:

SELECT * FROM t WHERE b < 8 OR b >= 8;

In addition, the order of the ranges in the WHERE clause was significant; the issue was not triggered, for example, by the query SELECT * FROM t WHERE b <= 8 OR b > 8. (Bug #57396)

ndb_restore now retries failed transactions when replaying log entries, just as it does when restoring data. (Bug #57618)

When a data node angel process failed to fork off a new worker process (to replace one that had failed), the failure was not handled. This meant that the angel process either transformed itself into a worker process, or itself failed. In the first case, the data node continued to run, but there was no longer any angel to restart it in the event of failure, even with StopOnError set to 0. (Bug #53456)

Transient errors during a local checkpoint were not retried, leading to a crash of the data node. Now when such errors occur, they are retried up to 10 times if necessary. (Bug #57650)

Queries using WHERE varchar_pk_column LIKE 'pattern%' or WHERE varchar_pk_column LIKE 'pattern_' against an NDB table having a VARCHAR column as its primary key failed to return all matching rows. (Bug #56853)

Successive CREATE NODEGROUP and DROP NODEGROUP commands could cause mysqld processes to crash. (Bug #57164)

Data nodes compiled with gcc 4.5 or higher crashed during startup. (Bug #57761)

The LQHKEYREQ request message used by the local query handler when checking the major schema version of a table, being only 16 bits wide, could cause this check to fail with an Invalid schema version error (NDB error code 1227). This issue occurred after creating and dropping (and re-creating) the same table 65537 times, then trying to insert rows into the table. (Bug #57896)

References: See also Bug #57897.

The disconnection of an API or management node due to missed heartbeats led to a race condition which could cause data nodes to crash. (Bug #57946)

During a GCP takeover, it was possible for one of the data nodes not to receive a SUB_GCP_COMPLETE_REP signal, with the result that it would report itself as GCP_COMMITTING while the other data nodes reported GCP_PREPARING. (Bug #57522)

The MAX_ROWS option for CREATE TABLE was ignored, which meant that it was not possible to enable multi-threaded building of indexes. (Bug #57360)

On Windows, the angel process which monitors and (when necessary) restarts the data node process failed to spawn a new worker in some circumstances where the arguments vector contained extra items placed at its beginning. This could occur when the path to ndbd.exe or ndbmtd.exe contained one or more spaces. (Bug #57949)

A number of cluster log warning messages relating to deprecated configuration parameters contained spelling, formatting, and other errors. (Bug #57381)

A GCP stop is detected using 2 parameters which determine the maximum time that a global checkpoint or epoch can go unchanged; one of these controls this timeout for GCPs and one controls the timeout for epochs. Suppose the cluster is configured such that TimeBetweenEpochsTimeout is 100 ms but HeartbeatIntervalDbDb is 1500 ms. A node failure can be signalled after 4 missed heartbeats—in this case, 6000 ms. However, this would exceed TimeBetweenEpochsTimeout, causing false detection of a GCP. To prevent this from happening, the configured value for TimeBetweenEpochsTimeout is automatically adjusted, based on the values of HeartbeatIntervalDbDb and ArbitrationTimeout.

The current issue arose when the automatic adjustment routine did not correctly take into consideration the fact that, during cascading node-failures, several intervals of length 4 * (HeartbeatIntervalDBDB + ArbitrationTimeout) may elapse before all node failures have internally been resolved. This could cause false GCP detection in the event of a cascading node failure. (Bug #57322)

The SUMA kernel block has a 10-element ring buffer for storing out-of-order SUB_GCP_COMPLETE_REP signals received from the local query handlers when global checkpoints are completed. In some cases, exceeding the ring buffer capacity on all nodes of a node group at the same time caused the node group to fail with an assertion. (Bug #57563)

Aborting a native NDB backup in the ndb_mgm client using the ABORT BACKUP command did not work correctly when using ndbmtd, in some cases leading to a crash of the cluster. (Bug #56285)

Disk Data: When performing online DDL on Disk Data tables, scans and moving of the relevant tuples were done in more or less random order. This fix causes these scans to be done in the order of the tuples, which should improve performance of such operations due to the more sequential ordering of the scans. (Bug #57848)

References: See also Bug #57827.

Disk Data: Adding unique indexes to NDB Disk Data tables could take an extremely long time. This was particularly noticeable when using ndb_restore --rebuild-indexes. (Bug #57827)

Cluster API: An application dropping a table at the same time that another application tried to set up a replication event on the same table could lead to a crash of the data node. The same issue could sometimes cause NdbEventOperation::execute() to hang. (Bug #57886)

Cluster API: An NDB API client program under load could abort with an assertion error in TransporterFacade::remove_from_cond_wait_queue. (Bug #51775)

References: See also Bug #32708.

It is now possible using the ndb_mgm management client or the MGM API to force a data node shutdown or restart even if this would force the shutdown or restart of the entire cluster.

In the management client, this is implemented through the addition of the -f (force) option to the STOP and RESTART commands. For more information, see Commands in the MySQL Cluster Management Client.

The MGM API also adds two new methods for forcing such a node shutdown or restart; see ndb_mgm_stop4(), and ndb_mgm_restart4(), for more information about these methods. (Bug #54226)

mysqldump as supplied with MySQL Cluster now has an --add-drop-trigger option which adds a DROP TRIGGER IF EXISTS statement before each dumped trigger definition. (Bug #55691)

References: See also Bug #34325, Bug #11747863.

Cluster API: The MGM API function ndb_mgm_get_version(), which was previously internal, has now been moved to the public API. This function can be used to get NDB storage engine and other version information from the management server. (Bug #51310)

References: See also Bug #51273.

The failure of a data node during some scans could cause other data nodes to fail. (Bug #54945)

The text file cluster_change_hist.txt containing old MySQL Cluster changelog information was no longer being maintained, and so has been removed from the tree. (Bug #56116)

MySQL Cluster stores, for each row in each NDB table, a Global Checkpoint Index (GCI) which identifies the last committed transaction that modified the row. As such, a GCI can be thought of as a coarse-grained row version.

Due to changes in the format used by NDB to store local checkpoints (LCPs) in MySQL Cluster NDB 6.3.11, it could happen that, following cluster shutdown and subsequent recovery, the GCI values for some rows could be changed unnecessarily; this could possibly, over the course of many node or system restarts (or both), lead to an inconsistent database. (Bug #56770)

Under certain rare conditions, attempting to start more than one ndb_mgmd process simultaneously using the --reload option caused a race condition such that none of the ndb_mgmd processes could start. (Bug #56844)

At startup, an ndbd or ndbmtd process creates directories for its file system without checking to see whether they already exist. Portability code added in MySQL Cluster NDB 7.0.18 and MySQL Cluster NDB 7.1.7 did not account for this fact, printing a spurious error message when a directory to be created already existed. This unneeded printout has been removed. (Bug #57087)

A data node can be shut down having completed and synchronized a given GCI x, while having written a great many log records belonging to the next GCI x + 1, as part of normal operations. However, when starting, completing, and synchronizing GCI x + 1, then the log records from original start must not be read. To make sure that this does not happen, the REDO log reader finds the last GCI to restore, scans forward from that point, and erases any log records that were not (and should never be) used.

The current issue occurred because this scan stopped immediately as soon as it encountered an empty page. This was problematic because the REDO log is divided into several files; thus, it could be that there were log records in the beginning of the next file, even if the end of the previous file was empty. These log records were never invalidated; following a start or restart, they could be reused, leading to a corrupt REDO log. (Bug #56961)

When multiple SQL nodes were connected to the cluster and one of them stopped in the middle of a DDL operation, the mysqld process issuing the DDL timed out with the error distributing tbl_name timed out. Ignoring. (Bug #56763)

Exhausting the number of available commit-ack markers (controlled by the MaxNoOfConcurrentTransactions parameter) led to a data node crash. (Bug #54944)

An online ALTER TABLE ... ADD COLUMN operation that changed the table schema such that the number of 32-bit words used for the bitmask allocated to each DML operation increased during a transaction in DML which was performed prior to DDL which was followed by either another DML operation or—if using replication—a commit, led to data node failure.

This was because the data node did not take into account that the bitmask for the before-image was smaller than the current bitmask, which caused the node to crash. (Bug #56524)

References: This bug is a regression of Bug #35208.

An error in program flow in ndbd.cpp could result in data node shutdown routines being called multiple times. (Bug #56890)

On Windows, a data node refused to start in some cases unless the ndbd.exe executable was invoked using an absolute rather than a relative path. (Bug #56257)

Memory pages used for DataMemory, once assigned to ordered indexes, were not ever freed, even after any rows that belonged to the corresponding indexes had been deleted. (Bug #56829)

When distributing CREATE TABLE and DROP TABLE operations among several SQL nodes attached to a MySQL Cluster. the LOCK_OPEN lock normally protecting mysqld's internal table list is released so that other queries or DML statements are not blocked. However, to make sure that other DDL is not executed simultaneously, a global schema lock (implemented as a row-level lock by NDB) is used, such that all operations that can modify the state of the mysqld internal table list also need to acquire this global schema lock. The SHOW TABLE STATUS statement did not acquire this lock. (Bug #56841)

When running a SELECT on an NDB table with BLOB or TEXT columns, memory was allocated for the columns but was not freed until the end of the SELECT. This could cause problems with excessive memory usage when dumping (using for example mysqldump) tables with such columns and having many rows, large column values, or both. (Bug #52313)

References: See also Bug #56488, Bug #50310.

In certain cases, DROP DATABASE could sometimes leave behind a cached table object, which caused problems with subsequent DDL operations. (Bug #56840)

Cluster API: The MGM API function ndb_mgm_get_version() did not set the error message before returning with an error. With this fix, it is now possible to call ndb_mgm_get_latest_error() after a failed call to this function such that ndb_mgm_get_latest_error() returns an error number and error message, as expected of MGM API calls. (Bug #57088)

Cluster API: The MGM API functions ndb_mgm_stop() and ndb_mgm_restart() set the error code and message without first checking whether the management server handle was NULL, which could lead to fatal errors in MGM API applications that depended on these functions. (Bug #57089)

Important Change: More finely grained control over restart-on-failure behavior is provided with two new data node configuration parameters MaxStartFailRetries and StartFailRetryDelay. MaxStartFailRetries limits the total number of retries made before giving up on starting the data node; StartFailRetryDelay sets the number of seconds between retry attempts.

These parameters are used only if StopOnError is set to 0.

For more information, see Defining MySQL Cluster Data Nodes. (Bug #54341)

Important Change: It is no longer possible to make a dump of the ndbinfo database using mysqldump. (Bug #54316)

Following a failure of the master data node, the new master sometimes experienced a race condition which caused the node to terminate with a GcpStop error. (Bug #56044)

Startup messages previously written by ndb_mgmd to stdout are now written to the cluster log instead when LogDestination is set. (Bug #47595)

Trying to create a table having a BLOB or TEXT column with DEFAULT '' failed with the error Illegal null attribute. (An empty default is permitted and ignored by MyISAM; NDB should do the same.) (Bug #55121)

The LockExecuteThreadToCPU configuration parameter was not handled correctly for CPU ID values greater than 255. (Bug #56185)

ndb_restore always reported 0 for the GCPStop (end point of the backup). Now it provides useful binary log position and epoch information. (Bug #56298)

The warning MaxNoOfExecutionThreads (#) > LockExecuteThreadToCPU count (#), this could cause contention could be logged when running ndbd, even though the condition described can occur only when using ndbmtd. (Bug #54342)

ndb_mgmd --nodaemon logged to the console in addition to the configured log destination. (Bug #54779)

The graceful shutdown of a data node could sometimes cause transactions to be aborted unnecessarily. (Bug #18538)

References: See also Bug #55641.

Added the --server-id-bits option for mysqld and mysqlbinlog.

For mysqld, the --server-id-bits option indicates the number of least significant bits within the 32-bit server ID which actually identify the server. Indicating that the server ID uses less than 32 bits permits the remaining bits to be used for other purposes by NDB API applications using the Event API and OperationOptions::AnyValue.

For mysqlbinlog, the --server-id-bits option tells mysqlbinlog how to interpret the server IDs in the binary log when the binary log was written by a mysqld having its server_id_bits set to less than the maximum (32). (Bug #52305)

Added the DictTrace data node configuration parameter, for use in debugging NDB code. For more information, see Defining MySQL Cluster Data Nodes. (Bug #55963)

Important Change; Cluster API: The poll and select calls made by the MGM API were not interrupt-safe; that is, a signal caught by the process while waiting for an event on one or more sockets returned error -1 with errno set to EINTR. This caused problems with MGM API functions such as ndb_logevent_get_next() and ndb_mgm_get_status2().

To fix this problem, the internal ndb_socket_poller::poll() function has been made EINTR-safe.

The old version of this function has been retained as poll_unsafe(), for use by those parts of NDB that do not need the EINTR-safe version of the function. (Bug #55906)

When another data node failed, a given data node DBTC kernel block could time out while waiting for DBDIH to signal commits of pending transactions, leading to a crash. Now in such cases the timeout generates a prinout, and the data node continues to operate. (Bug #55715)

Starting ndb_mgmd with --config-cache=0 caused it to leak memory. (Bug #55205)

An excessive number of timeout warnings (normally used only for debugging) were written to the data node logs. (Bug #53987)

The TCP configuration parameters HostName1 and HostName2 were not displayed in the output of ndb_config --configinfo. (Bug #55839)

The configure.js option WITHOUT_DYNAMIC_PLUGINS=TRUE was ignored when building MySQL Cluster for Windows using CMake. Among the effects of this issue was that CMake attempted to build the InnoDB storage engine as a plugin (.DLL file) even though the InnoDB Plugin is not currently supported by MySQL Cluster. (Bug #54913)

It was possible for a DROP DATABASE statement to remove NDB hidden blob tables without removing the parent tables, with the result that the tables, although hidden to MySQL clients, were still visible in the output of ndb_show_tables but could not be dropped using ndb_drop_table. (Bug #54788)

Disk Data: As an optimization when inserting a row to an empty page, the page is not read, but rather simply initialized. However, this optimzation was performed in all cases when an empty row was inserted, even though it should have been done only if it was the first time that the page had been used by a table or fragment. This is because, if the page had been in use, and then all records had been released from it, the page still needed to be read to learn its log sequence number (LSN).

This caused problems only if the page had been flushed using an incorrect LSN and the data node failed before any local checkpoint was completed—which would remove any need to apply the undo log, hence the incorrect LSN was ignored.

The user-visible result of the incorrect LSN was that it caused the data node to fail during a restart. It was perhaps also possible (although not conclusively proven) that this issue could lead to incorrect data. (Bug #54986)

Cluster API: Calling NdbTransaction::refresh() did not update the timer for TransactionInactiveTimeout. (Bug #54724)

Restrictions on some types of mismatches in column definitions when restoring data using ndb_restore have been relaxed. These include the following types of mismatches:

Now, when one of these types of mismatches in column definitions is encountered, ndb_restore no longer stops with an error; instead, it accepts the data and inserts it into the target table, while issuing a warning to the user.

For more information, see ndb_restore — Restore a MySQL Cluster Backup. (Bug #54423)

References: See also Bug #53810, Bug #54178, Bug #54242, Bug #54279.

It is now possible to install management node and data node processes as Windows services. (See Installing MySQL Cluster Processes as Windows Services, for more information.) In addition, data node processes on Windows are now maintained by angel processes, just as they are on other platforms supported by MySQL Cluster.

The disconnection of all API nodes (including SQL nodes) during an ALTER TABLE caused a memory leak. (Bug #54685)

The presence of duplicate [tcp] sections in the config.ini file caused the management server to crash. Now in such cases, ndb_mgmd fails gracefully with an appropriate error message. (Bug #49400)

A table having the maximum number of attributes permitted could not be backed up using the ndb_mgm client.

(Bug #54155)

When performing an online alter table where 2 or more SQL nodes connected to the cluster were generating binary logs, an incorrect message could be sent from the data nodes, causing mysqld processes to crash. This problem was often difficult to detect, because restarting SQL node or data node processes could clear the error, and because the crash in mysqld did not occur until several minutes after the erroneous message was sent and received. (Bug #54168)

The two MySQL Server options, --ndb-wait-connected and --ndb-wait-setup, did not set the corresponding system variables. (Bug #48402)

The setting for BuildIndexThreads was ignored by ndbmtd, which made it impossible to use more than 4 cores for rebuilding indexes. (Bug #54521)

If a node shutdown (either in isolation or as part of a system shutdown) occurred directly following a local checkpoint, it was possible that this local checkpoint would not be used when restoring the cluster. (Bug #54611)

When adding multiple new node groups to a MySQL Cluster, it was necessary for each new node group to add only the nodes to be assigned to the new node group, create that node group using CREATE NODEGROUP, then repeat this process for each new node group to be added to the cluster. The fix for this issue makes it possible to add all of the new nodes at one time, and then issue several CREATE NODEGROUP commands in succession. (Bug #54497)

Cluster API: When using the NDB API, it was possible to rename a table with the same name as that of an existing table.

(Bug #54651)

Cluster API: An excessive number of client connections, such that more than 1024 file descriptors, sockets, or both were open, caused NDB API applications to crash. (Bug #34303)

Important Change: Commercial binary releases of MySQL Cluster NDB 7.1 now include support for the InnoDB storage engine. (Bug #52945)

References: The patch for the following bug was reverted: Bug #31989.

Cluster API: The value of an internal constant used in the implementation of the NdbOperation and NdbScanOperation classes caused MySQL Cluster NDB 7.0 NDB API applications compiled against MySQL Cluster NDB 7.0.14 or earlier to fail when run with MySQL Cluster 7.0.15, and MySQL Cluster NDB 7.1 NDB API applications compiled against MySQL Cluster NDB 7.1.3 or earlier to break when used with MySQL Cluster 7.1.4. (Bug #54516)

When using mysqldump to back up and restore schema information while using ndb_restore for restoring only the data, restoring to MySQL Cluster NDB 7.1.4 from an older version failed on tables having columns with default values. This was because versions of MySQL Cluster prior to MySQL Cluster NDB 7.1.4 did not have native support for default values.

In addition, the MySQL Server supports TIMESTAMP columns having dynamic default values, such as DEFAULT CURRENT_TIMESTAMP; however, the current implementation of NDB-native default values permits only a constant default value.

To fix this issue, the manner in which NDB treats TIMESTAMP columns is reverted to its pre-NDB-7.1.4 behavior (obtaining the default value from mysqld rather than NDBCLUSTER) except where a TIMESTAMP column uses a constant default, as in the case of a column declared as TIMESTAMP DEFAULT 0 or TIMESTAMP DEFAULT 20100607174832. (Bug #54242)

Important Change: The maximum number of attributes (columns plus indexes) per table has increased to 512.

A --wait-nodes option has been added for ndb_waiter. When this option is used, the program waits only for the nodes having the listed IDs to reach the desired state. For more information, see ndb_waiter — Wait for MySQL Cluster to Reach a Given Status. (Bug #52323)

Added the --skip-unknown-objects option for ndb_restore. This option causes ndb_restore to ignore any schema objects which it does not recognize. Currently, this is useful chiefly for restoring native backups made from a cluster running MySQL Cluster NDB 7.0 to a cluster running MySQL Cluster NDB 6.3.

As part of this change, new methods relating to default values have been added to the Column and Table classes in the NDB API. For more information, see Column::getDefaultValue(), Column::setDefaultValue(), and Table::hasDefaultValues(). (Bug #30529)

Added the MySQL Cluster management server option --config-cache, which makes it possible to enable and disable configuration caching. This option is turned on by default; to disable configuration caching, start ndb_mgmd with --config-cache=0, or with --skip-config-cache. See ndb_mgmd — The MySQL Cluster Management Server Daemon, for more information.

Incompatible Change; Cluster API: The default behavior of the NDB API Event API has changed as follows:

Previously, when creating an Event, DDL operations (alter and drop operations on tables) were automatically reported on any event operation that used this event, but as a result of this change, this is no longer the case. Instead, you must now invoke the event's setReport() method, with the new EventReport value ER_DDL, to get this behavior.

For existing NDB API applications where you wish to retain the old behavior, you must update the code as indicated previously, then recompile, following an upgrade. Otherwise, DDL operations are no longer reported after upgrading libndbnclient.

For more information, see The Event::EventReport Type, and Event::setReport(). (Bug #53308)

An internal buffer allocator used by NDB has the form alloc(wanted, minimum) and attempts to allocate wanted pages, but is permitted to allocate a smaller number of pages, between wanted and minimum. However, this allocator could sometimes allocate fewer than minimum pages, causing problems with multi-threaded building of ordered indexes. (Bug #53580)

Setting DataMemory higher than 4G on 32-bit platforms caused ndbd to crash, instead of failing gracefully with an error. (Bug #52536, Bug #50928)

Specifying the node ID as part of the --ndb-connectstring option to mysqld was not handled correctly.

The fix for this issue includes the following changes:

(Bug #44299)

When an NDB log handler failed, the memory allocated to it was freed twice. (Bug #53200)

NDB did not distinguish correctly between table names differing only by lettercase when lower_case_table_names was set to 0. (Bug #33158)

After creating NDB tables until creation of a table failed due to NDB error 905 Out of attribute records (increase MaxNoOfAttributes), then increasing MaxNoOfAttributes and restarting all management node and data node processes, attempting to drop and re-create one of the tables failed with the error Out of table records..., even when sufficient table records were available. (Bug #53944)

References: See also Bug #52055. This bug is a regression of Bug #44294.

When compiled with support for epoll but this functionality is not available at runtime, MySQL Cluster tries to fall back to use the select() function in its place. However, an extra ndbout_c() call in the transporter registry code caused ndbd to fail instead. (Bug #53482)

Creating a Disk Data table, dropping it, then creating an in-memory table and performing a restart, could cause data node processes to fail with errors in the DBTUP kernel block if the new table's internal ID was the same as that of the old Disk Data table. This could occur because undo log handling during the restart did not check that the table having this ID was now in-memory only. (Bug #53935)

A table created while ndb_table_no_logging was enabled was not always stored to disk, which could lead to a data node crash with Error opening DIH schema files for table. (Bug #53934)

When the LogDestination parameter was set using with a relative path, the management server failed to store its value unless started with --initial or --reload. (Bug #52268)

When creating an index, NDB failed to check whether the internal ID allocated to the index was within the permissible range, leading to an assertion. This issue could manifest itself as a data node failure with NDB error 707 (No more table metadata records (increase MaxNoOfTables)), when creating tables in rapid succession (for example, by a script, or when importing from mysqldump), even with a relatively high value for MaxNoOfTables and a relatively low number of tables. (Bug #52055)

ndb_restore did not raise any errors if hashmap creation failed during execution. (Bug #51434)

The value set for the ndb_mgmd option --ndb-nodeid was not verified prior to use as being within the permitted range (1 to 255, inclusive), leading to a crash of the management server. (Bug #53412)

NDB truncated a column declared as DECIMAL(65,0) to a length of 64. Now such a column is accepted and handled correctly. In cases where the maximum length (65) is exceeded, NDB now raises an error instead of truncating. (Bug #53352)

ndb_mgm -e "ALL STATUS" erroneously reported that data nodes remained in start phase 0 until they had actually started.

Incompatible Change: The schema for the memoryusage table of the ndbinfo information database has changed, as detailed in the following list:

For more information, aee The ndbinfo memoryusage Table.

Important Change: The experimental pools table has been removed from the ndbinfo database. Information useful to MySQL Cluster administration that was contained in this table should be available from other ndbinfo tables.

Important Note: MySQL Cluster 7.1 is now supported for production use on Windows platforms.

As with MySQL Cluster on other supported platforms, you cannot build MySQL Cluster for Windows from the MySQL Server 5.1 sources; you must use the source code from the MySQL Cluster NDB 7.1 tree.

A mysqld, when attempting to access the ndbinfo database, crashed if could not contact the management server. (Bug #51067)

The mysql client system command did not work properly. This issue was only known to affect the version of the mysql client that was included with MySQL Cluster NDB 7.0 and MySQL Cluster NDB 7.1 releases. (Bug #48574)

ha_ndbcluster.cc was not compiled with the same SAFEMALLOC and SAFE_MUTEX flags as the MySQL Server. (Bug #51857)

The internal variable ndb_new_handler, which is no longer used, has been removed. (Bug #51858)

Restoring a MySQL Cluster backup between platforms having different endianness failed when also restoring metadata and the backup contained a hashmap not already present in the database being restored to. This issue was discovered when trying to restore a backup made on Solaris/SPARC to a MySQL Cluster running on Solaris/x86, but could conceivably occur in other cases where the endianness of the platform on which the backup was taken differed from that of the platform being restored to. (Bug #51432)

When performing a complex mix of node restarts and system restarts, the node that was elected as master sometimes required optimized node recovery due to missing REDO information. When this happened, the node crashed with Failure to recreate object ... during restart, error 721 (because the DBDICT restart code was run twice). Now when this occurs, node takeover is executed immediately, rather than being made to wait until the remaining data nodes have started. (Bug #52135)

References: See also Bug #48436.

Some values shown in the memoryusage table did not match corresponding values shown by the ndb_mgm client ALL REPORT MEMORYUSAGE command. (Bug #51735)

When debug compiling MySQL Cluster on Windows, the mysys library was not compiled with -DSAFEMALLOC and -DSAFE_MUTEX, due to the fact that my_socket.c was misnamed as my_socket.cc. (Bug #51856)

If a node or cluster failure occurred while mysqld was scanning the ndb.ndb_schema table (which it does when attempting to connect to the cluster), insufficient error handling could lead to a crash by mysqld in certain cases. This could happen in a MySQL Cluster with a great many tables, when trying to restart data nodes while one or more mysqld processes were restarting. (Bug #52325)

After running a mixed series of node and system restarts, a system restart could hang or fail altogether. This was caused by setting the value of the newest completed global checkpoint too low for a data node performing a node restart, which led to the node reporting incorrect GCI intervals for its first local checkpoint. (Bug #52217)

In MySQL Cluster NDB 7.0 and later, DDL operations are performed within schema transactions; the NDB kernel code for starting a schema transaction checks that all data nodes are at the same version before permitting a schema transaction to start. However, when a version mismatch was detected, the client was not actually informed of this problem, which caused the client to hang. (Bug #52228)

The redo log protects itself from being filled up by periodically checking how much space remains free. If insufficient redo log space is available, it sets the state TAIL_PROBLEM which results in transactions being aborted with error code 410 (out of redo log). However, this state was not set following a node restart, which meant that if a data node had insufficient redo log space following a node restart, it could crash a short time later with Fatal error due to end of REDO log. Now, this space is checked during node restarts. (Bug #51723)

Packaging; Cluster API: The file META-INF/services/org.apache.openjpa.lib.conf.ProductDerivation was missing from the clusterjpa JAR file. This could cause setting openjpa.BrokerFactory to “ndb” to be rejected. (Bug #52106)

References: See also Bug #14192154.

Disk Data: Inserts of blob column values into a MySQL Cluster Disk Data table that exhausted the tablespace resulted in misleading no such tuple error messages rather than the expected error tablespace full.

This issue appeared similar to Bug #48113, but had a different underlying cause. (Bug #52201)

Disk Data: DDL operations on Disk Data tables having a relatively small UNDO_BUFFER_SIZE could fail unexpectedly.

Cluster API: A number of issues were corrected in the NDB API coding examples found in the storage/ndb/ndbapi-examples directory in the MySQL Cluster source tree. These included possible endless recursion in ndbapi_scan.cpp as well as problems running some of the examples on systems using Windows or Mac OS X due to the lettercase used for some table names. (Bug #30552, Bug #30737)

Cluster API: It is now possible to determine, using the ndb_desc utility or the NDB API, which data nodes contain replicas of which partitions. For ndb_desc, a new --extra-node-info option is added to cause this information to be included in its output. A new method Table::getFragmentNodes() is added to the NDB API for obtaining this information programmatically. (Bug #51184)

A new configuration parameter HeartbeatThreadPriority makes it possible to select between a first-in, first-out or round-round scheduling policy for management node and API node heartbeat threads, as well as to set the priority of these threads. See Defining a MySQL Cluster Management Server, or Defining SQL and Other API Nodes in a MySQL Cluster, for more information. (Bug #49617)

Start phases are now written to the data node logs. (Bug #49158)

Numeric codes used in management server status update messages in the cluster logs have been replaced with text descriptions. (Bug #49627)

References: See also Bug #44248.

Formerly, the REPORT and DUMP commands returned output to all ndb_mgm clients connected to the same MySQL Cluster. Now, these commands return their output only to the ndb_mgm client that actually issued the command. (Bug #40865)

Disk Data: The ndb_desc utility can now show the extent space and free extent space for subordinate BLOB and TEXT columns (stored in hidden BLOB tables by NDB). A --blob-info option has been added for this program that causes ndb_desc to generate a report for each subordinate BLOB table. For more information, see ndb_desc — Describe NDB Tables. (Bug #50599)

Important Change: The DATA_MEMORY column of the memoryusage table was renamed to memory_type. (Bug #50926)

The output of the ndb_mgm client REPORT BACKUPSTATUS command could sometimes contain errors due to uninitialized data. (Bug #51316)

An initial restart of a data node configured with a large amount of memory could fail with a Pointer too large error. (Bug #51027)

References: This bug was introduced by Bug #47818.

ndbmtd started on a single-core machine could sometimes fail with a Job Buffer Full error when MaxNoOfExecutionThreads was set greater than LockExecuteThreadToCPU. Now a warning is logged when this occurs. (Bug #50582)

When the MemReportFrequency configuration parameter was set in config.ini, the ndb_mgm client REPORT MEMORYUSAGE command printed its output multiple times. (Bug #37632)

Setting IndexMemory greater than 2GB could cause data nodes to crash while starting. (Bug #51256)

When deciding how to divide the REDO log, the DBDIH kernel block saved more than was needed to restore the previous local checkpoint, which could cause REDO log space to be exhausted prematurely (NDB error 410). (Bug #51547)

A side effect of the ndb_restore --disable-indexes and --rebuild-indexes options is to change the schema versions of indexes. When a mysqld later tried to drop a table that had been restored from backup using one or both of these options, the server failed to detect these changed indexes. This caused the table to be dropped, but the indexes to be left behind, leading to problems with subsequent backup and restore operations. (Bug #51374)

The following issues were fixed in the ndb_mgm client REPORT MEMORYUSAGE command:

(Bug #50196)

DML operations can fail with NDB error 1220 (REDO log files overloaded...) if the opening and closing of REDO log files takes too much time. If this occurred as a GCI marker was being written in the REDO log while REDO log file 0 was being opened or closed, the error could persist until a GCP stop was encountered. This issue could be triggered when there was insufficient REDO log space (for example, with configuration parameter settings NoOfFragmentLogFiles = 6 and FragmentLogFileSize = 6M) with a load including a very high number of updates. (Bug #51512)

References: See also Bug #20904.

When using NoOfReplicas equal to 1 or 2, if data nodes from one node group were restarted 256 times and applications were running traffic such that it would encounter NDB error 1204 (Temporary failure, distribution changed), the live node in the node group would crash, causing the cluster to crash as well. The crash occurred only when the error was encountered on the 256th restart; having the error on any previous or subsequent restart did not cause any problems. (Bug #50930)

Information about several management client commands was missing from (that is, truncated in) the output of the HELP command. (Bug #46114)

An attempted online upgrade from a MySQL Cluster NDB 6.3 or 7.0 release to a MySQL Cluster NDB 7.1 release failed, as the first upgraded data node rejected the remaining data nodes as using incompatible versions. (Bug #51429)

Replication of a MySQL Cluster using multi-threaded data nodes could fail with forced shutdown of some data nodes due to the fact that ndbmtd exhausted LongMessageBuffer much more quickly than ndbd. After this fix, passing of replication data between the DBTUP and SUMA NDB kernel blocks is done using DataMemory rather than LongMessageBuffer.

Until you can upgrade, you may be able to work around this issue by increasing the LongMessageBuffer setting; doubling the default should be sufficient in most cases. (Bug #46914)

Issuing a command in the ndb_mgm client after it had lost its connection to the management server could cause the client to crash. (Bug #49219)

The ndb_restore message Successfully created index `PRIMARY`... was directed to stderr instead of stdout. (Bug #51037)

A GROUP BY query against NDB tables sometimes did not use any indexes unless the query included a FORCE INDEX option. With this fix, indexes are used by such queries (where otherwise possible) even when FORCE INDEX is not specified. (Bug #50736)

A SELECT requiring a sort could fail with the error Can't find record in 'table' when run concurrently with a DELETE from the same table. (Bug #45687)

ndb_restore crashed while trying to restore a corrupted backup, due to missing error handling. (Bug #51223)

ndb_mgm -e "... REPORT ..." did not write any output to stdout.

The fix for this issue also prevents the cluster log from being flooded with INFO messages when DataMemory usage reaches 100%, and insures that when the usage is decreased, an appropriate message is written to the cluster log. (Bug #31542, Bug #44183, Bug #49782)

Disk Data: The error message returned after atttempting to execute ALTER LOGFILE GROUP on an nonexistent logfile group did not indicate the reason for the failure. (Bug #51111)

Cluster API: An issue internal to ndb_mgm could cause problems when trying to start a large number of data nodes at the same time. (Bug #51273)

References: See also Bug #51310.

Cluster API: When reading blob data with lock mode LM_SimpleRead, the lock was not upgraded as expected. (Bug #51034)

The ndbinfo database is added to provide MySQL Cluster metadata in real time. The tables making up this database contain information about memory, buffer, and other resource usage, as well as configuration parameters and settings, event counts, and other useful data. Access to ndbinfo is done by executing standard SQL queries on its tables using the mysql command-line client or other MySQL client application. No special setup procedures are required; ndbinfo is created automatically and visible in the output of SHOW DATABASES when the MySQL Server is connected to a MySQL Cluster.

For more information, see The ndbinfo MySQL Cluster Information Database.

Cluster API: ClusterJ 1.0 and ClusterJPA 1.0 are now available for programming Java applications with MySQL Cluster. ClusterJ is a Java connector providing an object-relational API for performing high-speed operations such as primary key lookups on a MySQL Cluster database, but does not require the use of the MySQL Server or JDBC (Connector/J). ClusterJ uses a new library NdbJTie which enables direct access from Java to the NDB API and thus to the NDBCLUSTER storage engine. ClusterJPA is a new implementation of OpenJPA, and can use either a JDBC connection to a MySQL Cluster SQL node (MySQL Server) or a direct connection to MySQL Cluster using NdbJTie, depending on availability and operational performance.

ClusterJ, ClusterJPA, and NdbJTie require Java 1.5 or 1.6, and MySQL Cluster NDB 7.0 or later.

All necessary libraries and other files for ClusterJ, ClusterJPA, and NdbJTie can be found in the MySQL Cluster NDB 7.1.1 or later distribution.

When a primary key lookup on an NDB table containing one or more BLOB columns was executed in a transaction, a shared lock on any blob tables used by the NDB table was held for the duration of the transaction. (This did not occur for indexed or non-indexed WHERE conditions.)

Now in such cases, the lock is released after all BLOB data has been read. (Bug #49190)

Important Change: The default value of the DiskIOThreadPool data node configuration parameter has changed from 8 to 2.

Incompatible Change; Cluster API: Several NDB API methods were declared as const, but did not return an lvalue, which caused compiler warnings when using gcc 4.3 or newer to perform the build. The methods affected are NdbEventOperation::tableNameChanged(), NdbEventOperation::tableFrmChanged(), NdbEventOperation::tableFragmentationChanged(), NdbEventOperation::tableRangeListChanged(), and NdbOperation::getType(). (Bug #44840)

Important Change: The --with-ndb-port-base option for configure has been removed. It is now handled as an unknown and invalid option if you attempt to use it when configuring a build of MySQL Cluster. (Bug #47941)

References: See also Bug #38502.

The value set by the --ndb-cluster-connection-pool option was not shown in the output of SHOW STATUS LIKE 'NDB%'. (Bug #44118)

mysqld could sometimes crash during a commit while trying to handle NDB Error 4028 Node failure caused abort of transaction. (Bug #38577)