Changes in the MySQL Cluster NDB 7.0 Series

This section contains unified change history highlights for all MySQL Cluster releases based on version 7.0 of the NDB storage engine through MySQL Cluster NDB 7.0.42. Included are all changelog entries in the categories MySQL Cluster, Disk Data, and Cluster API.

Early MySQL Cluster NDB 7.0 releases tagged NDB 6.4.x are also included in this listing.

For an overview of features that were added in MySQL Cluster NDB 7.0, see What is New in MySQL Cluster NDB 7.0.

Changes in MySQL Cluster NDB 7.0.42 (5.1.73-ndb-7.0.42)

Bugs Fixed

  • Data nodes running ndbmtd could stall while performing an online upgrade of a MySQL Cluster containing a great many tables from a version prior to NDB 7.0.31 to version 7.1.31 or later. (Bug #16693068)

  • Cluster API: Refactoring that was performed in MySQL Cluster NDB 7.0.41 inadvertently introduced a dependency in Ndb.hpp on a file that is not included in the distribution, which caused NDB API applications to fail to compile. The dependency has been removed. (Bug #18293112, Bug #71803)

    References: This issue is a regression of: Bug #17647637.

Changes in MySQL Cluster NDB 7.0.41 (5.1.73-ndb-7.0.41)

Bugs Fixed

  • Packaging: Compilation of ndbmtd failed on Solaris 10 and 11 for 32-bit x86, and the binary was not included in the binary distributions for these platforms. (Bug #16620938)

  • Disk Data: When using Disk Data tables and ndbmtd data nodes, it was possible for the undo buffer to become overloaded, leading to a crash of the data nodes. This issue was more likely to be encountered when using Disk Data columns whose size was approximately 8K or larger. (Bug #16766493)

  • Cluster API: UINT_MAX64 was treated as a signed value by Visual Studio 2010. To prevent this from happening, the value is now explicitly defined as unsigned. (Bug #17947674)

    References: See also: Bug #17647637.

  • Poor support or lack of support on some platforms for monotonic timers caused issues with delayed signal handling by the job scheduler for the multithreaded data node. Variances (timer leaps) on such platforms are now handled in the same way the multithreaded data node process that they are by the singlethreaded version. (Bug #17857442)

    References: See also: Bug #17475425, Bug #17647637.

  • When using single-threaded (ndbd) data nodes with RealTimeScheduler enabled, the CPU did not, as intended, temporarily lower its scheduling priority to normal every 10 milliseconds to give other, non-realtime threads a chance to run. (Bug #17739131)

  • Timers used in timing scheduler events in the NDB kernel have been refactored, in part to insure that they are monotonic on all platforms. In particular, on Windows, event intervals were previously calculated using values obtained from GetSystemTimeAsFileTime(), which reads directly from the system time (wall clock), and which may arbitrarily be reset backward or forward, leading to false watchdog or heartbeat alarms, or even node shutdown. Lack of timer monotonicity could also cause slow disk writes during backups and global checkpoints. To fix this issue, the Windows implementation now uses QueryPerformanceCounters() instead of GetSystemTimeAsFileTime(). In the event that a monotonic timer is not found on startup of the data nodes, a warning is logged.

    In addition, on all platforms, a check is now performed at compile time for available system monotonic timers, and the build fails if one cannot be found; note that CLOCK_HIGHRES is now supported as an alternative for CLOCK_MONOTONIC if the latter is not available. (Bug #17647637)

  • The global checkpoint lag watchdog tracking the number of times a check for GCP lag was performed using the system scheduler and used this count to check for a timeout condition, but this caused a number of issues. To overcome these limitations, the GCP watchdog has been refactored to keep track of its own start times, and to calculate elapsed time by reading the (real) clock every time it is called.

    In addition, any backticks (rare in any case) are now handled by taking the backward time as the new current time and calculating the elapsed time for this round as 0. Finally, any ill effects of a forward leap, which possibly could expire the watchdog timer immediately, are reduced by never calculating an elapsed time longer than the requested delay time for the watchdog timer. (Bug #17647469)

    References: See also: Bug #17842035.

  • In certain rare cases on commit of a transaction, an Ndb object was released before the transaction coordinator (DBTC kernel block) sent the expected COMMIT_CONF signal; NDB failed to send a COMMIT_ACK signal in response, which caused a memory leak in the NDB kernel could later lead to node failure.

    Now an Ndb object is not released until the COMMIT_CONF signal has actually been received. (Bug #16944817)

  • After restoring the database metadata (but not any data) by running ndb_restore --restore_meta (or -m), SQL nodes would hang while trying to SELECT from a table in the database to which the metadata was restored. In such cases the attempt to query the table now fails as expected, since the table does not actually exist until ndb_restore is executed with --restore_data (-r). (Bug #16890703)

    References: See also: Bug #21184102.

  • The ndbd_redo_log_reader utility now supports a --help option. Using this options causes the program to print basic usage information, and then to exit. (Bug #11749591, Bug #36805)

  • Cluster API: It was possible for an Ndb object to receive signals for handling before it was initialized, leading to thread interleaving and possible data node failure when executing a call to Ndb::init(). To guard against this happening, a check is now made when it is starting to receive signals that the Ndb object is properly initialized before any signals are actually handled. (Bug #17719439)

Changes in MySQL Cluster NDB 7.0.40 (5.1.72-ndb-7.0.40)

Functionality Added or Changed

  • The length of time a management node waits for a heartbeat message from another management node is now configurable using the HeartbeatIntervalMgmdMgmd management node configuration parameter added in this release. The connection is considered dead after 3 missed heartbeats. The default value is 1500 milliseconds, or a timeout of approximately 6000 ms. (Bug #17807768, Bug #16426805)

  • BLOB and TEXT columns are now reorganized by the ALTER ONLINE TABLE ... REORGANIZE PARTITION statement. (Bug #13714148)

Bugs Fixed

  • Monotonic timers on several platforms can experience issues which might result in the monotonic clock doing small jumps back in time. This is due to imperfect synchronization of clocks between multiple CPU cores and does not normally have an adverse effect on the scheduler and watchdog mechanisms; so we handle some of these cases by making backtick protection less strict, although we continue to ensure that the backtick is less than 10 milliseconds. This fix also removes several checks for backticks which are thereby made redundant. (Bug #17973819)

  • Trying to restore to a table having a BLOB column in a different position from that of the original one caused ndb_restore --restore_data to fail. (Bug #17395298)

  • ndb_restore could abort during the last stages of a restore using attribute promotion or demotion into an existing table. This could happen if a converted attribute was nullable and the backup had been run on active database. (Bug #17275798)

  • The DBUTIL data node block is now less strict about the order in which it receives certain messages from other nodes. (Bug #17052422)

  • The Windows error ERROR_FILE_EXISTS was not recognized by NDB, which treated it as an unknown error. (Bug #16970960)

  • RealTimeScheduler did not work correctly with data nodes running ndbmtd. (Bug #16961971)

  • File system errors occurring during a local checkpoint could sometimes cause an LCP to hang with no obvious cause when they were not handled correctly. Now in such cases, such errors always cause the node to fail. Note that the LQH block always shuts down the node when a local checkpoint fails; the change here is to make likely node failure occur more quickly and to make the original file system error more visible. (Bug #16961443)

  • Maintenance and checking of parent batch completion in the SPJ block of the NDB kernel was reimplemented. Among other improvements, the completion state of all ancestor nodes in the tree are now preserved. (Bug #16925513)

  • Added the ndb_error_reporter options --connection-timeout, which makes it possible to set a timeout for connecting to nodes, --dry-scp, which disables scp connections to remote hosts, and --skip-nodegroup, which skips all nodes in a given node group. (Bug #16602002)

    References: See also: Bug #11752792, Bug #44082.

  • ndb_mgm treated backup IDs provided to ABORT BACKUP commands as signed values, so that backup IDs greater than 231 wrapped around to negative values. This issue also affected out-of-range backup IDs, which wrapped around to negative values instead of causing errors as expected in such cases. The backup ID is now treated as an unsigned value, and ndb_mgm now performs proper range checking for backup ID values greater than MAX_BACKUPS (232). (Bug #16585497, Bug #68798)

  • The NDB receive thread waited unnecessarily for additional job buffers to become available when receiving data. This caused the receive mutex to be held during this wait, which could result in a busy wait when the receive thread was running with real-time priority.

    This fix also handles the case where a negative return value from the initial check of the job buffer by the receive thread prevented further execution of data reception, which could possibly lead to communication blockage or configured ReceiveBufferMemory underutilization. (Bug #15907515)

  • When the available job buffers for a given thread fell below the critical threshold, the internal multi-threading job scheduler waited for job buffers for incoming rather than outgoing signals to become available, which meant that the scheduler waited the maximum timeout (1 millisecond) before resuming execution. (Bug #15907122)

  • Under some circumstances, a race occurred where the wrong watchdog state could be reported. A new state name Packing Send Buffers is added for watchdog state number 11, previously reported as Unknown place. As part of this fix, the state numbers for states without names are always now reported in such cases. (Bug #14824490)

  • When a node fails, the Distribution Handler (DBDIH kernel block) takes steps together with the Transaction Coordinator (DBTC) to make sure that all ongoing transactions involving the failed node are taken over by a surviving node and either committed or aborted. Transactions taken over which are then committed belong in the epoch that is current at the time the node failure occurs, so the surviving nodes must keep this epoch available until the transaction takeover is complete. This is needed to maintain ordering between epochs.

    A problem was encountered in the mechanism intended to keep the current epoch open which led to a race condition between this mechanism and that normally used to declare the end of an epoch. This could cause the current epoch to be closed prematurely, leading to failure of one or more surviving data nodes. (Bug #14623333, Bug #16990394)

  • When using dynamic listening ports for accepting connections from API nodes, the port numbers were reported to the management server serially. This required a round trip for each API node, causing the time required for data nodes to connect to the management server to grow linearly with the number of API nodes. To correct this problem, each data node now reports all dynamic ports at once. (Bug #12593774)

  • ndb_error-reporter did not support the --help option. (Bug #11756666, Bug #48606)

    References: See also: Bug #11752792, Bug #44082.

  • Program execution failed to break out of a loop after meeting a desired condition in a number of internal methods, performing unneeded work in all cases where this occurred. (Bug #69610, Bug #69611, Bug #69736, Bug #17030606, Bug #17030614, Bug #17160263)

  • ABORT BACKUP in the ndb_mgm client (see Commands in the MySQL Cluster Management Client) took an excessive amount of time to return (approximately as long as the backup would have required to complete, had it not been aborted), and failed to remove the files that had been generated by the aborted backup. (Bug #68853, Bug #17719439)

  • Attribute promotion and demotion when restoring data to NDB tables using ndb_restore --restore_data with the --promote-attributes and --lossy-conversions options has been improved as follows:

    • Columns of types CHAR, and VARCHAR can now be promoted to BINARY and VARBINARY, and columns of the latter two types can be demoted to one of the first two.

      Note that converted character data is not checked to conform to any character set.

    • Any of the types CHAR, VARCHAR, BINARY, and VARBINARY can now be promoted to TEXT or BLOB.

      When performing such promotions, the only other sort of type conversion that can be performed at the same time is between character types and binary types.

  • Cluster API: The Event::setTable() method now supports a pointer or a reference to table as its required argument. If a null table pointer is used, the method now returns -1 to make it clear that this is what has occurred. (Bug #16329082)

Changes in MySQL Cluster NDB 7.0.39 (5.1.69-ndb-7.0.39)

Functionality Added or Changed

Bugs Fixed

  • Performance: In a number of cases found in various locations in the MySQL Cluster codebase, unnecessary iterations were performed; this was caused by failing to break out of a repeating control structure after a test condition had been met. This community-contributed fix removes the unneeded repetitions by supplying the missing breaks. (Bug #16904243, Bug #69392, Bug #16904338, Bug #69394, Bug #16778417, Bug #69171, Bug #16778494, Bug #69172, Bug #16798410, Bug #69207, Bug #16801489, Bug #69215, Bug #16904266, Bug #69393)

  • The planned or unplanned shutdown of one or more data nodes while reading table data from the ndbinfo database caused a memory leak. (Bug #16932989)

  • Executing DROP TABLE while DBDIH was updating table checkpoint information subsequent to a node failure could lead to a data node failure. (Bug #16904469)

  • In certain cases, when starting a new SQL node, mysqld failed with Error 1427 Api node died, when SUB_START_REQ reached node. (Bug #16840741)

  • Failure to use container classes specific NDB during node failure handling could cause leakage of commit-ack markers, which could later lead to resource shortages or additional node crashes. (Bug #16834416)

  • Use of an uninitialized variable employed in connection with error handling in the DBLQH kernel block could sometimes lead to a data node crash or other stability issues for no apparent reason. (Bug #16834333)

  • A race condition in the time between the reception of a execNODE_FAILREP signal by the QMGR kernel block and its reception by the DBLQH and DBTC kernel blocks could lead to data node crashes during shutdown. (Bug #16834242)

  • The CLUSTERLOG command (see Commands in the MySQL Cluster Management Client) caused ndb_mgm to crash on Solaris SPARC systems. (Bug #16834030)

  • The LCP fragment scan watchdog periodically checks for lack of progress in a fragment scan performed as part of a local checkpoint, and shuts down the node if there is no progress after a given amount of time has elapsed. This interval, formerly hard-coded as 60 seconds, can now be configured using the LcpScanProgressTimeout data node configuration parameter added in this release.

    This configuration parameter sets the maximum time the local checkpoint can be stalled before the LCP fragment scan watchdog shuts down the node. The default is 60 seconds, which provides backward compatibility with previous releases.

    You can disable the LCP fragment scan watchdog by setting this parameter to 0. (Bug #16630410)

  • After issuing START BACKUP id WAIT STARTED, if id had already been used for a backup ID, an error caused by the duplicate ID occurred as expected, but following this, the START BACKUP command never completed. (Bug #16593604, Bug #68854)

  • When trying to specify a backup ID greater than the maximum allowed, the value was silently truncated. (Bug #16585455, Bug #68796)

  • The unexpected shutdown of another data node as a starting data node received its node ID caused the latter to hang in Start Phase 1. (Bug #16007980)

    References: See also: Bug #18993037.

  • SELECT ... WHERE ... LIKE from an NDB table could return incorrect results when using engine_condition_pushdown=ON. (Bug #15923467, Bug #67724)

  • Creating more than 32 hash maps caused data nodes to fail. Usually new hashmaps are created only when performing reorganzation after data nodes have been added or when explicit partitioning is used, such as when creating a table with the MAX_ROWS option, or using PARTITION BY KEY() PARTITIONS n. (Bug #14710311)

  • When performing an INSERT ... ON DUPLICATE KEY UPDATE on an NDB table where the row to be inserted already existed and was locked by another transaction, the error message returned from the INSERT following the timeout was Transaction already aborted instead of the expected Lock wait timeout exceeded. (Bug #14065831, Bug #65130)

  • When START BACKUP WAIT STARTED was run from the command line using ndb_mgm --execute (-e), the client did not exit until the backup completed. (Bug #11752837, Bug #44146)

  • Formerly, the node used as the coordinator or leader for distributed decision making between nodes (also known as the DICT manager—see The DBDICT Block) was indicated in the output of the ndb_mgm client SHOW command as the master node, although this node has no relationship to a master server in MySQL Replication. (It should also be noted that it is not necessary to know which node is the leader except when debugging NDBCLUSTER source code.) To avoid possible confusion, this label has been removed, and the leader node is now indicated in SHOW command output using an asterisk (*) character. (Bug #11746263, Bug #24880)

  • Cluster API: For each log event retrieved using the MGM API, the log event category (ndb_mgm_event_category) was simply cast to an enum type, which resulted in invalid category values. Now an offset is added to the category following the cast to ensure that the value does not fall out of the allowed range.


    This change was reverted by the fix for Bug #18354165. See the MySQL Cluster API Developer documentation for ndb_logevent_get_next(), for more information.

    (Bug #16723708)

    References: See also: Bug #18354165.

Changes in MySQL Cluster NDB 7.0.38 (5.1.69-ndb-7.0.38)

Functionality Added or Changed

  • Cluster API: Added DUMP code 2514, which provides information about counts of transaction objects per API node. For more information, see DUMP 2514. See also Commands in the MySQL Cluster Management Client. (Bug #15878085)

  • When ndb_restore fails to find a table, it now includes in the error output an NDB API error code giving the reason for the failure. (Bug #16329067)

  • Following an upgrade to MySQL Cluster NDB 7.2.7 or later, it was not possible to downgrade online again to any previous version, due to a change in that version in the default size (number of LDM threads used) for NDB table hash maps. The fix for this issue makes the size configurable, with the addition of the DefaultHashMapSize configuration parameter.

    To retain compatibility with an older release that does not support large hash maps, you can set this parameter in the cluster' config.ini file to the value used in older releases (240) before performing an upgrade, so that the data nodes continue to use smaller hash maps that are compatible with the older release. You can also now employ this parameter in MySQL Cluster NDB 7.0 and MySQL Cluster NDB 7.1 to enable larger hash maps prior to upgrading to MySQL Cluster NDB 7.2. For more information, see the description of the DefaultHashMapSize parameter. (Bug #14800539)

    References: See also: Bug #14645319.

Bugs Fixed

  • Important Change; Cluster API: When checking—as part of evaluating an if predicate—which error codes should be propagated to the application, any error code less than 6000 caused the current row to be skipped, even those codes that should have caused the query to be aborted. In addition, a scan that aborted due to an error from DBTUP when no rows had been sent to the API caused DBLQH to send a SCAN_FRAGCONF signal rather than a SCAN_FRAGREF signal to DBTC. This caused DBTC to time out waiting for a SCAN_FRAGREF signal that was never sent, and the scan was never closed.

    As part of this fix, the default ErrorCode value used by NdbInterpretedCode::interpret_exit_nok() has been changed from 899 (Rowid already allocated) to 626 (Tuple did not exist). The old value continues to be supported for backward compatibility. User-defined values in the range 6000-6999 (inclusive) are also now supported. You should also keep in mind that the result of using any other ErrorCode value not mentioned here is not defined or guaranteed.

    See also The NDB Communication Protocol, and NDB Kernel Blocks, for more information. (Bug #16176006)

  • The NDB Error-Reporting Utility (ndb_error_reporter) failed to include the cluster nodes' log files in the archive it produced when the FILE option was set for the parameter LogDestination. (Bug #16765651)

    References: See also: Bug #11752792, Bug #44082.

  • A WHERE condition that contained a boolean test of the result of an IN subselect was not evaluated correctly. (Bug #16678033)

  • In some cases a data node could stop with an exit code but no error message other than (null) was logged. (This could occur when using ndbd or ndbmtd for the data node process.) Now in such cases the appropriate error message is used instead (see ndbd Error Messages). (Bug #16614114)

  • When using tables having more than 64 fragments in a MySQL Cluster where multiple TC threads were configured (on data nodes running ndbmtd, using ThreadConfig), AttrInfo and KeyInfo memory could be freed prematurely, before scans relying on these objects could be completed, leading to a crash of the data node. (Bug #16402744)

    References: See also: Bug #13799800. This issue is a regression of: Bug #14143553.

  • When started with --initial and an invalid --config-file (-f) option, ndb_mgmd removed the old configuration cache before verifying the configuration file. Now in such cases, ndb_mgmd first checks for the file, and continues with removing the configuration cache only if the configuration file is found and is valid. (Bug #16299289)

  • Executing a DUMP 2304 command during a data node restart could cause the data node to crash with a Pointer too large error. (Bug #16284258)

  • Improved handling of lagging row change event subscribers by setting size of the GCP pool to the value of MaxBufferedEpochs. This fix also introduces a new MaxBufferedEpochBytes data node configuration parameter, which makes it possible to set a total number of bytes per node to be reserved for buffering epochs. In addition, a new DUMP code (8013) has been added which causes a list a lagging subscribers for each node to be printed to the cluster log (see DUMP 8013). (Bug #16203623)

  • Data nodes could fail during a system restart when the host ran short of memory, due to signals of the wrong types (ROUTE_ORD and TRANSID_AI_R) being sent to the DBSPJ kernel block. (Bug #16187976)

  • Attempting to perform additional operations such as ADD COLUMN as part of an ALTER [ONLINE | OFFLINE] TABLE ... RENAME ... statement is not supported, and now fails with an ER_NOT_SUPPORTED_YET error. (Bug #16021021)

  • Purging the binary logs could sometimes cause mysqld to crash. (Bug #15854719)

  • Due to a known issue in the MySQL Server, it is possible to drop the PERFORMANCE_SCHEMA database. (Bug #15831748) In addition, when executed on a MySQL Server acting as a MySQL Cluster SQL node, DROP DATABASE caused this database to be dropped on all SQL nodes in the cluster. Now, when executing a distributed drop of a database, NDB does not delete tables that are local only. This prevents MySQL system databases from being dropped in such cases. (Bug #14798043)

    References: See also: Bug #15831748.

  • Executing OPTIMIZE TABLE on an NDB table containing TEXT or BLOB columns could sometimes cause mysqld to fail. (Bug #14725833)

  • An error message in src/mgmsrv/MgmtSrvr.cpp was corrected. (Bug #14548052, Bug #66518)

  • Executing a DUMP 1000 command (see DUMP 1000) that contained extra or malformed arguments could lead to data node failures. (Bug #14537622)

  • Exhaustion of LongMessageBuffer memory under heavy load could cause data nodes running ndbmtd to fail. (Bug #14488185)

  • The help text for ndb_select_count did not include any information about using table names. (Bug #11755737, Bug #47551)

  • The ndb_mgm client HELP command did not show the complete syntax for the REPORT command.

  • Cluster API: The Ndb::computeHash() API method performs a malloc() if no buffer is provided for it to use. However, it was assumed that the memory thus returned would always be suitably aligned, which is not always the case. Now when malloc() provides a buffer to this method, the buffer is aligned after it is allocated, and before it is used. (Bug #16484617)

Changes in MySQL Cluster NDB 7.0.37 (5.1.67-ndb-7.0.37)

Functionality Added or Changed

  • Added several new columns to the transporters table and counters for the counters table of the ndbinfo information database. The information provided may help in troublehsooting of transport overloads and problems with send buffer memory allocation. For more information, see the descriptions of these tables. (Bug #15935206)

  • To provide information which can help in assessing the current state of arbitration in a MySQL Cluster as well as in diagnosing and correcting arbitration problems, 3 new tables—membership, arbitrator_validity_detail, and arbitrator_validity_summary—have been added to the ndbinfo information database. (Bug #13336549)

Bugs Fixed

  • When an NDB table grew to contain approximately one million rows or more per partition, it became possible to insert rows having duplicate primary or unique keys into it. In addition, primary key lookups began to fail, even when matching rows could be found in the table by other means.

    This issue was introduced in MySQL Cluster NDB 7.0.36, MySQL Cluster NDB 7.1.26, and MySQL Cluster NDB 7.2.9. Signs that you may have been affected include the following:

    • Rows left over that should have been deleted

    • Rows unchanged that should have been updated

    • Rows with duplicate unique keys due to inserts or updates (which should have been rejected) that failed to find an existing row and thus (wrongly) inserted a new one

    This issue does not affect simple scans, so you can see all rows in a given table using SELECT * FROM table and similar queries that do not depend on a primary or unique key.

    Upgrading to or downgrading from an affected release can be troublesome if there are rows with duplicate primary or unique keys in the table; such rows should be merged, but the best means of doing so is application dependent.

    In addition, since the key operations themselves are faulty, a merge can be difficult to achieve without taking the MySQL Cluster offline, and it may be necessary to dump, purge, process, and reload the data. Depending on the circumstances, you may want or need to process the dump with an external application, or merely to reload the dump while ignoring duplicates if the result is acceptable.

    Another possibility is to copy the data into another table without the original table' unique key constraints or primary key (recall that CREATE TABLE t2 SELECT * FROM t1 does not by default copy t1's primary or unique key definitions to t2). Following this, you can remove the duplicates from the copy, then add back the unique constraints and primary key definitions. Once the copy is in the desired state, you can either drop the original table and rename the copy, or make a new dump (which can be loaded later) from the copy. (Bug #16023068, Bug #67928)

  • The management client command ALL REPORT BackupStatus failed with an error when used with data nodes having multiple LQH worker threads (ndbmtd data nodes). The issue did not effect the node_id REPORT BackupStatus form of this command. (Bug #15908907)

  • The multi-threaded job scheduler could be suspended prematurely when there were insufficient free job buffers to allow the threads to continue. The general rule in the job thread is that any queued messages should be sent before the thread is allowed to suspend itself, which guarantees that no other threads or API clients are kept waiting for operations which have already completed. However, the number of messages in the queue was specified incorrectly, leading to increased latency in delivering signals, sluggish response, or otherwise suboptimal performance. (Bug #15908684)

  • Node failure during the dropping of a table could lead to the node hanging when attempting to restart.

    When this happened, the NDB internal dictionary (DBDICT) lock taken by the drop table operation was held indefinitely, and the logical global schema lock taken by the SQL the drop table operation from which the drop operation originated was held until the NDB internal operation timed out. To aid in debugging such occurrences, a new dump code, DUMP 1228 (or DUMP DictDumpLockQueue), which dumps the contents of the DICT lock queue, has been added in the ndb_mgm client. (Bug #14787522)

  • Job buffers act as the internal queues for work requests (signals) between block threads in ndbmtd and could be exhausted if too many signals are sent to a block thread.

    Performing pushed joins in the DBSPJ kernel block can execute multiple branches of the query tree in parallel, which means that the number of signals being sent can increase as more branches are executed. If DBSPJ execution cannot be completed before the job buffers are filled, the data node can fail.

    This problem could be identified by multiple instances of the message sleeploop 10!! in the cluster out log, possibly followed by job buffer full. If the job buffers overflowed more gradually, there could also be failures due to error 1205 (Lock wait timeout exceeded), shutdowns initiated by the watchdog timer, or other timeout related errors. These were due to the slowdown caused by the 'sleeploop'.

    Normally up to a 1:4 fanout ratio between consumed and produced signals is permitted. However, since there can be a potentially unlimited number of rows returned from the scan (and multiple scans of this type executing in parallel), any ratio greater 1:1 in such cases makes it possible to overflow the job buffers.

    The fix for this issue defers any lookup child which otherwise would have been executed in parallel with another is deferred, to resume when its parallel child completes one of its own requests. This restricts the fanout ratio for bushy scan-lookup joins to 1:1. (Bug #14709490)

    References: See also: Bug #14648712.

  • During an online upgrade, certain SQL statements could cause the server to hang, resulting in the error Got error 4012 'Request ndbd time-out, maybe due to high load or communication problems' from NDBCLUSTER. (Bug #14702377)

  • The recently added LCP fragment scan watchdog occasionally reported problems with LCP fragment scans having very high table id, fragment id, and row count values.

    This was due to the watchdog not accounting for the time spent draining the backup buffer used to buffer rows before writing to the fragment checkpoint file.

    Now, in the final stage of an LCP fragment scan, the watchdog switches from monitoring rows scanned to monitoring the buffer size in bytes. The buffer size should decrease as data is written to the file, after which the file should be promptly closed. (Bug #14680057)

  • Under certain rare circumstances, MySQL Cluster data nodes could crash in conjunction with a configuration change on the data nodes from a single-threaded to a multi-threaded transaction coordinator (using the ThreadConfig configuration parameter for ndbmtd). The problem occurred when a mysqld that had been started prior to the change was shut down following the rolling restart of the data nodes required to effect the configuration change. (Bug #14609774)

Changes in MySQL Cluster NDB 7.0.36 (5.1.66-ndb-7.0.36)

Functionality Added or Changed

  • Added 3 new columns to the transporters table in the ndbinfo database. The remote_address, bytes_sent, and bytes_received columns help to provide an overview of data transfer across the transporter links in a MySQL Cluster. This information can be useful in verifying system balance, partitioning, and front-end server load balancing; it may also be of help when diagnosing network problems arising from link saturation, hardware faults, or other causes. (Bug #14685458)

  • Data node logs now provide tracking information about arbitrations, including which nodes have assumed the arbitrator role and at what times. (Bug #11761263, Bug #53736)

Bugs Fixed

  • A slow filesystem during local checkpointing could exert undue pressure on DBDIH kernel block file page buffers, which in turn could lead to a data node crash when these were exhausted. This fix limits the number of table definition updates that DBDIH can issue concurrently. (Bug #14828998)

  • The management server process, when started with --config-cache=FALSE, could sometimes hang during shutdown. (Bug #14730537)

  • The output from ndb_config --configinfo now contains the same information as that from ndb_config --configinfo --xml, including explicit indicators for parameters that do not require restarting a data node with --initial to take effect. In addition, ndb_config indicated incorrectly that the LogLevelCheckpoint data node configuration parameter requires an initial node restart to take effect, when in fact it does not; this error was also present in the MySQL Cluster documentation, where it has also been corrected. (Bug #14671934)

  • Concurrent ALTER TABLE with other DML statements on the same NDB table returned Got error -1 'Unknown error code' from NDBCLUSTER. (Bug #14578595)

  • CPU consumption peaked several seconds after the forced termination an NDB client application due to the fact that the DBTC kernel block waited for any open transactions owned by the disconnected API client to be terminated in a busy loop, and did not break between checks for the correct state. (Bug #14550056)

  • Receiver threads could wait unnecessarily to process incomplete signals, greatly reducing performance of ndbmtd. (Bug #14525521)

  • On platforms where epoll was not available, setting multiple receiver threads with the ThreadConfig parameter caused ndbmtd to fail. (Bug #14524939)

  • Setting BackupMaxWriteSize to a very large value as compared with DiskCheckpointSpeed caused excessive writes to disk and CPU usage. (Bug #14472648)

  • Added the --connect-retries and --connect-delay startup options for ndbd and ndbmtd. --connect-retries (default 12) controls how many times the data node tries to connect to a management server before giving up; setting it to -1 means that the data node never stops trying to make contact. --connect-delay sets the number of seconds to wait between retries; the default is 5. (Bug #14329309, Bug #66550)

  • Following a failed ALTER TABLE ... REORGANIZE PARTITION statement, a subsequent execution of this statement after adding new data nodes caused a failure in the DBDIH kernel block which led to an unplanned shutdown of the cluster.

    DUMP code 7019 was added as part of this fix. It can be used to obtain diagnostic information relating to a failed data node. See DUMP 7019, for more information. (Bug #14220269)

    References: See also: Bug #18550318.

  • It was possible in some cases for two transactions to try to drop tables at the same time. If the master node failed while one of these operations was still pending, this could lead either to additional node failures (and cluster shutdown) or to new dictionary operations being blocked. This issue is addressed by ensuring that the master will reject requests to start or stop a transaction while there are outstanding dictionary takeover requests. In addition, table-drop operations now correctly signal when complete, as the DBDICT kernel block could not confirm node takeovers while such operations were still marked as pending completion. (Bug #14190114)

  • The DBSPJ kernel block had no information about which tables or indexes actually existed, or which had been modified or dropped, since execution of a given query began. Thus, DBSPJ might submit dictionary requests for nonexistent tables or versions of tables, which could cause a crash in the DBDIH kernel block.

    This fix introduces a simplified dictionary into the DBSPJ kernel block such that DBSPJ can now check reliably for the existence of a particular table or version of a table on which it is about to request an operation. (Bug #14103195)

  • Previously, it was possible to store a maximum of 46137488 rows in a single MySQL Cluster partition. This limitation has now been removed. (Bug #13844405, Bug #14000373)

    References: See also: Bug #13436216.

  • When using ndbmtd and performing joins, data nodes could fail where ndbmtd processes were configured to use a large number of local query handler threads (as set by the ThreadConfig configuration parameter), the tables accessed by the join had a large number of partitions, or both. (Bug #13799800, Bug #14143553)

Changes in MySQL Cluster NDB 7.0.35 (5.1.63-ndb-7.0.35)

Bugs Fixed

  • When reloading the redo log during a node or system restart, and with NoOfFragmentLogFiles greater than or equal to 42, it was possible for metadata to be read for the wrong file (or files). Thus, the node or nodes involved could try to reload the wrong set of data. (Bug #14389746)

Changes in MySQL Cluster NDB 7.0.34 (5.1.63-ndb-7.0.34)

Bugs Fixed

  • Important Change: When FILE was used for the value of the LogDestination parameter without also specifying the filename, the log file name defaulted to logger.log. Now in such cases, the name defaults to ndb_nodeid_cluster.log. (Bug #11764570, Bug #57417)

  • If the Transaction Coordinator aborted a transaction in the prepared state, this could cause a resource leak. (Bug #14208924)

  • When attempting to connect using a socket with a timeout, it was possible (if the timeout was exceeded) for the socket not to be set back to blocking. (Bug #14107173)

  • An error handling routine in the local query handler (DBLQH) used the wrong code path, which could corrupt the transaction ID hash, causing the data node process to fail. This could in some cases possibly lead to failures of other data nodes in the same node group when the failed node attempted to restart. (Bug #14083116)

  • When a fragment scan occurring as part of a local checkpoint (LCP) stopped progressing, this kept the entire LCP from completing, which could result it redo log exhaustion, write service outage, inability to recover nodes, and longer system recovery times. To help keep this from occurring, MySQL Cluster now implements an LCP watchdog mechanism, which monitors the fragment scans making up the LCP and takes action if the LCP is observed to be delinquent.

    This is intended to guard against any scan related system-level I/O errors or other issues causing problems with LCP and thus having a negative impact on write service and recovery times. Each node independently monitors the progress of local fragment scans occurring as part of an LCP. If no progress is made for 20 seconds, warning logs are generated every 10 seconds thereafter for up to 1 minute. At this point, if no progress has been made, the fragment scan is considered to have hung, and the node is restarted to enable the LCP to continue.

    In addition, a new ndbd exit code NDBD_EXIT_LCP_SCAN_WATCHDOG_FAIL is added to identify when this occurs. See LQH Errors, for more information. (Bug #14075825)

  • In some circumstances, transactions could be lost during an online upgrade. (Bug #13834481)

  • Attempting to add both a column and an index on that column in the same online ALTER TABLE statement caused mysqld to fail. Although this issue affected only the mysqld shipped with MySQL Cluster, the table named in the ALTER TABLE could use any storage engine for which online operations are supported. (Bug #12755722)

  • Cluster API: When an NDB API application called NdbScanOperation::nextResult() again after the previous call had returned end-of-file (return code 1), a transaction object was leaked. Now when this happens, NDB returns error code 4210 (Ndb sent more info than length specified); previouslyu in such cases, -1 was returned. In addition, the extra transaction object associated with the scan is freed, by returning it to the transaction coordinator's idle list. (Bug #11748194)

Changes in MySQL Cluster NDB 7.0.33 (5.1.61-ndb-7.0.33)

Bugs Fixed

  • DUMP 2303 in the ndb_mgm client now includes the status of the single fragment scan record reserved for a local checkpoint. (Bug #13986128)

  • A shortage of scan fragment records in DBTC resulted in a leak of concurrent scan table records and key operation records. (Bug #13966723)

Changes in MySQL Cluster NDB 7.0.32 (5.1.61-ndb-7.0.32)

Bugs Fixed

  • Important Change: The ALTER ONLINE TABLE ... REORGANIZE PARTITION statement can be used to create new table partitions after new empty nodes have been added to a MySQL Cluster. Usually, the number of partitions to create is determined automatically, such that, if no new partitions are required, then none are created. This behavior can be overridden by creating the original table using the MAX_ROWS option, which indicates that extra partitions should be created to store a large number of rows. However, in this case ALTER ONLINE TABLE ... REORGANIZE PARTITION simply uses the MAX_ROWS value specified in the original CREATE TABLE statement to determine the number of partitions required; since this value remains constant, so does the number of partitions, and so no new ones are created. This means that the table is not rebalanced, and the new data nodes remain empty.

    To solve this problem, support is added for ALTER ONLINE TABLE ... MAX_ROWS=newvalue, where newvalue is greater than the value used with MAX_ROWS in the original CREATE TABLE statement. This larger MAX_ROWS value implies that more partitions are required; these are allocated on the new data nodes, which restores the balanced distribution of the table data.

    For more information, see ALTER TABLE Syntax, and Adding MySQL Cluster Data Nodes Online. (Bug #13714648)

  • When the --skip-config-cache and --initial options were used together, ndb_mgmd failed to start. (Bug #13857301)

  • ALTER ONLINE TABLE failed when a DEFAULT option was used. (Bug #13830980)

  • In some cases, restarting data nodes spent a very long time in Start Phase 101, when API nodes must connect to the starting node (using NdbEventOperation), when the API nodes trying to connect failed in a live-lock scenario. This connection process uses a handshake during which a small number of messages are exchanged, with a timeout used to detect failures during the handshake.

    Prior to this fix, this timeout was set such that, if one API node encountered the timeout, all other nodes connecting would do the same. The fix also decreases this timeout. This issue (and the effects of the fix) are most likely to be observed on relatively large configurations having 10 or more data nodes and 200 or more API nodes. (Bug #13825163)

  • ndbmtd failed to restart when the size of a table definition exceeded 32K.

    (The size of a table definition is dependent upon a number of factors, but in general the 32K limit is encountered when a table has 250 to 300 columns.) (Bug #13824773)

  • An initial start using ndbmtd could sometimes hang. This was due to a state which occurred when several threads tried to flush a socket buffer to a remote node. In such cases, to minimize flushing of socket buffers, only one thread actually performs the send, on behalf of all threads. However, it was possible in certain cases for there to be data in the socket buffer waiting to be sent with no thread ever being chosen to perform the send. (Bug #13809781)

  • When trying to use --hostname=host:port to connect to a MySQL server running on a nonstandard port, the port argument was ignored. (Bug #13364905, Bug #62635)

Changes in MySQL Cluster NDB 7.0.31 (5.1.61-ndb-7.0.31)

Bugs Fixed

  • Important Change: A number of changes have been made in the configuration of transporter send buffers.

    1. The data node configuration parameter ReservedSendBufferMemory is now deprecated, and thus subject to removal in a future MySQL Cluster release. ReservedSendBufferMemory has been non-functional since it was introduced and remains so.

    2. TotalSendBufferMemory now works correctly with data nodes using ndbmtd.

    3. SendBufferMemory can now over-allocate into SharedGlobalMemory for ndbmtd data nodes (only).

    4. A new data node configuration parameter ExtraSendBufferMemory is introduced. Its purpose is to control how much additional memory can be allocated to the send buffer over and above that specified by TotalSendBufferMemory or SendBufferMemory. The default setting (0) allows up to 16MB to be allocated automatically.

    (Bug #13633845, Bug #11760629, Bug #53053)

  • Setting insert_id had no effect on the auto-increment counter for NDB tables. (Bug #13731134)

  • A data node crashed when more than 16G fixed-size memory was allocated by DBTUP to one fragment (because the DBACC kernel block was not prepared to accept values greater than 32 bits from it, leading to an overflow). Now in such cases, the data node returns Error 889 Table fragment fixed data reference has reached maximum possible value.... When this happens, you can work around the problem by increasing the number of partitions used by the table (such as by using the MAXROWS option with CREATE TABLE). (Bug #13637411)

    References: See also: Bug #11747870, Bug #34348.

  • Several instances in the NDB code affecting the operation of multi-threaded data nodes, where SendBufferMemory was associated with a specific thread for an unnecessarily long time, have been identified and fixed, by minimizing the time that any of these buffers can be held exclusively by a given thread (send buffer memory being critical to operation of the entire node). (Bug #13618181)

  • A very large value for BackupWriteSize, as compared to BackupMaxWriteSize, BackupDataBufferSize, or BackupLogBufferSize, could cause a local checkpoint or backup to hang. (Bug #13613344)

  • Queries using LIKE ... ESCAPE on NDB tables failed when pushed down to the data nodes. Such queries are no longer pushed down, regardless of the value of engine_condition_pushdown. (Bug #13604447, Bug #61064)

  • To avoid TCP transporter overload, an overload flag is kept in the NDB kernel for each data node; this flag is used to abort key requests if needed, yielding error 1218 Send Buffers overloaded in NDB kernel in such cases. Scans can also put significant pressure on transporters, especially where scans with a high degree of parallelism are executed in a configuration with relatively small send buffers. However, in these cases, overload flags were not checked, which could lead to node failures due to send buffer exhaustion. Now, overload flags are checked by scans, and in cases where returning sufficient rows to match the batch size (--ndb-batch-size server option) would cause an overload, the number of rows is limited to what can be accommodated by the send buffer.

    See also Configuring MySQL Cluster Send Buffer Parameters. (Bug #13602508)

  • A node failure and recovery while performing a scan on more than 32 partitions led to additional node failures during node takeover. (Bug #13528976)

  • The --skip-config-cache option now causes ndb_mgmd to skip checking for the configuration directory, and thus to skip creating it in the event that it does not exist. (Bug #13428853)

Changes in MySQL Cluster NDB 7.0.30 (5.1.56-ndb-7.0.30)

Bugs Fixed

  • At the beginning of a local checkpoint, each data node marks its local tables with a to be checkpointed flag. A failure of the master node during this process could cause either the LCP to hang, or one or more data nodes to be forcibly shut down. (Bug #13436481)

  • A node failure while a ANALYZE TABLE statement was executing resulted in a hung connection (and the user was not informed of any error that would cause this to happen). (Bug #13416603)

    References: See also: Bug #13407848.

Changes in MySQL Cluster NDB 7.0.29 (5.1.56-ndb-7.0.29)

Bugs Fixed

  • Added the MinFreePct data node configuration parameter, which specifies a percentage of data node resources to hold in reserve for restarts. The resources monitored are DataMemory, IndexMemory, and any per-table MAX_ROWS settings (see CREATE TABLE Syntax). The default value of MinFreePct is 5, which means that 5% from each these resources is now set aside for restarts. (Bug #13436216)

  • Because the log event buffer used internally by data nodes was circular, periodic events such as statistics events caused it to be overwritten too quickly. Now the buffer is partitioned by log event category, and its default size has been increased from 4K to 8K. (Bug #13394771)

  • The BatchSize and BatchByteSize configuration parameters, used to control the maximum sizes of result batches, are defined as integers. However, the values used to store these were incorrectly interpreted as numbers of bytes in the NDB kernel. This caused the DBLQH kernel block to fail to detect when the specified BatchByteSize was consumed. (Bug #13355055)

  • Previously, forcing simultaneously the shutdown of multiple data nodes using SHUTDOWN -F in the ndb_mgm management client could cause the entire cluster to fail. Now in such cases, any such nodes are forced to abort immediately. (Bug #12928429)

  • SELECT statements using LIKE CONCAT(...) OR LIKE CONCAT(...) in the WHERE clause returned incorrect results when run against NDB tables. (Bug #11765142, Bug #58073)

Changes in MySQL Cluster NDB 7.0.28 (5.1.56-ndb-7.0.28)

Functionality Added or Changed

  • Introduced the CrashOnCorruptedTuple data node configuration parameter. When enabled, this parameter causes data nodes to handle corrupted tuples in a fail-fast manner—in other words, whenever the data node detects a corrupted tuple, it forcibly shuts down if CrashOnCorruptedTuple is enabled. For backward compatibility, this parameter is disabled by default. (Bug #12598636)

Bugs Fixed

  • When adding data nodes online, if the SQL nodes were not restarted before starting the new data nodes, the next query to be executed crashed the SQL node on which it was run. (Bug #13715216, Bug #62847)

    References: This issue is a regression of: Bug #13117187.

  • When a failure of multiple data nodes during a local checkpoint (LCP) that took a long time to complete included the node designated as master, any new data nodes attempting to start before all ongoing LCPs were completed later crashed. This was due to the fact that node takeover by the new master cannot be completed until there are no pending local checkpoints. Long-running LCPs such as those which triggered this issue can occur when fragment sizes are sufficiently large (see MySQL Cluster Nodes, Node Groups, Replicas, and Partitions, for more information). Now in such cases, data nodes (other than the new master) are kept from restarting until the takeover is complete. (Bug #13323589)

  • When deleting from multiple tables using a unique key in the WHERE condition, the wrong rows were deleted. In addition, UPDATE triggers failed when rows were changed by deleting from or updating multiple tables. (Bug #12718336, Bug #61705, Bug #12728221)

  • A SubscriberNodeIdUndefined error was previously unhandled, resulting in a data node crash, but is now handled by NDB Error 1429, Subscriber node undefined in SubStartReq. (Bug #12598496)

  • Shutting down a mysqld while under load caused the spurious error messages Opening ndb_binlog_index: killed and Unable to lock table ndb_binlog_index to be written in the cluster log. (Bug #11930428)

Changes in MySQL Cluster NDB 7.0.27 (5.1.56-ndb-7.0.27)

Functionality Added or Changed

Bugs Fixed

  • Incompatible Change; Cluster API: Restarting a machine hosting data nodes, SQL nodes, or both, caused such nodes when restarting to time out while trying to obtain node IDs.

    As part of the fix for this issue, the behavior and default values for the NDB API Ndb_cluster_connection::connect() method have been improved. Due to these changes, the version number for the included NDB client library ( has been increased from 4.0.0 to 5.0.0. For NDB API applications, this means that as part of any upgrade, you must do both of the following:

    • Review and possibly modify any NDB API code that uses the connect() method, in order to take into account its changed default retry handling.

    • Recompile any NDB API applications using the new version of the client library.

    Also in connection with this issue, the default value for each of the two mysqld options --ndb-wait-connected and --ndb-wait-setup has been increased to 30 seconds (from 0 and 15, respectively). In addition, a hard-coded 30-second delay was removed, so that the value of --ndb-wait-connected is now handled correctly in all cases. (Bug #12543299)

  • Setting IndexMemory or sometimes DataMemory to 2 GB or higher could lead to data node failures under some conditions. (Bug #12873640)

  • When replicating DML statements with IGNORE between clusters, the number of operations that failed due to nonexistent keys was expected to be no greater than the number of defined operations of any single type. Because the slave SQL thread defines operations of multiple types in batches together, code which relied on this assumption could cause mysqld to fail. (Bug #12859831)

  • The maximum effective value for the OverloadLimit configuration parameter was limited by the value of SendBufferMemory. Now the value set for OverloadLimit is used correctly, up to this parameter's stated maximum (4G). (Bug #12712109)

  • AUTO_INCREMENT values were not set correctly for INSERT IGNORE statements affecting NDB tables. This could lead such statements to fail with Got error 4350 'Transaction already aborted' from NDBCLUSTER when inserting multiple rows containing duplicate values. (Bug #11755237, Bug #46985)

  • When failure handling of an API node takes longer than 300 seconds, extra debug information is included in the resulting output. In cases where the API node's node ID was greater than 48, these extra debug messages could lead to a crash, and confuing output otherwise. This was due to an attempt to provide information specific to data nodes for API nodes as well. (Bug #62208)

  • In rare cases, a series of node restarts and crashes during restarts could lead to errors while reading the redo log. (Bug #62206)

Changes in MySQL Cluster NDB 7.0.26 (5.1.56-ndb-7.0.26)

Functionality Added or Changed

  • Added the MaxDMLOperationsPerTransaction data node configuration parameter, which can be used to limit the number of DML operations used by a transaction; if the transaction requires more than this many DML operations, the transaction is aborted. (Bug #12589613)

Bugs Fixed

  • When global checkpoint indexes were written with no intervening end-of-file or megabyte border markers, this could sometimes lead to a situation in which the end of the redo log was mistakenly regarded as being between these GCIs, so that if the restart of a data node took place before the start of the next redo log was overwritten, the node encountered an Error while reading the REDO log. (Bug #12653993, Bug #61500)

    References: See also: Bug #56961.

  • Restarting a mysqld during a rolling upgrade with data nodes running a mix of old and new versions of the MySQL Cluster software caused the mysqld to run in read-only mode. (Bug #12651364, Bug #61498)

  • Error reporting has been improved for cases in which API nodes are unable to connect due to apparent unavailability of node IDs. (Bug #12598398)

  • Error messages for Failed to convert connection transporter registration problems were inspecific. (Bug #12589691)

  • Under certain rare circumstances, a data node process could fail with Signal 11 during a restart. This was due to uninitialized variables in the QMGR kernel block. (Bug #12586190)

  • Multiple management servers were unable to detect one another until all nodes had fully started. As part of the fix for this issue, two new status values RESUME and CONNECTED can be reported for management nodes in the output of the ndb_mgm client SHOW command (see Commands in the MySQL Cluster Management Client). Two corresponding status values NDB_MGM_NODE_STATUS_RESUME and NDB_MGM_NODE_STATUS_CONNECTED are also added to the list of possible values for an ndb_mgm_node_status data structure in the MGM API. (Bug #12352191, Bug #48301)

  • Handling of the MaxNoOfTables and MaxNoOfAttributes configuration parameters was not consistent in all parts of the NDB kernel, and were only strictly enforced by the DBDICT and SUMA kernel blocks. This could lead to problems when tables could be created but not replicated. Now these parameters are treated by SUMA and DBDICT as suggested maximums rather than hard limits, as they are elsewhere in the NDB kernel. (Bug #61684)

  • It was not possible to shut down a management node while one or more data nodes were stopped (for whatever reason). This issue was a regression introduced in MySQL Cluster NDB 7.0.24 and MySQL Cluster NDB 7.1.13. (Bug #61607)

    References: See also: Bug #61147.

  • Cluster API: Applications that included the header file ndb_logevent.h could not be built using the Microsoft Visual Studio C compiler or the Oracle (Sun) Studio C compiler due to empty struct definitions. (Bug #12678971)

  • Cluster API: Within a transaction, after creating, executing, and closing a scan, calling NdbTransaction::refresh() after creating and executing but not closing a second scan caused the application to crash. (Bug #12646659)

Changes in MySQL Cluster NDB 7.0.25 (5.1.56-ndb-7.0.25)

Bugs Fixed

  • The internal Ndb_getinaddr() function has been rewritten to use getaddrinfo() instead of my_gethostbyname_r() (which is removed in a later version of the MySQL Server). (Bug #12542120)

  • Two unused test files in storage/ndb/test/sql contained incorrect versions of the GNU Lesser General Public License. The files and the directory containing them have been removed. (Bug #11810156)

    References: See also: Bug #11810224.

  • Error 1302 gave the wrong error message (Out of backup record). This has been corrected to A backup is already running. (Bug #11793592)

  • When using two management servers, issuing in an ndb_mgm client connected to one management server a STOP command for stopping the other management server caused Error 2002 (Stop failed ... Send to process or receive failed.: Permanent error: Application error), even though the STOP command actually succeeded, and the second ndb_mgmd was shut down. (Bug #61147)

  • In ndbmtd, a node connection event is detected by a CMVMI thread which sends a CONNECT_REP signal to the QMGR kernel block. In a few isolated circumstances, a signal might be transferred to QMGR directly by the NDB transporter before the CONNECT_REP signal actually arrived. This resulted in reports in the error log with status Temporary error, restart node, and the message Internal program error. (Bug #61025)

  • Renaming a table having BLOB or TEXT columns (or both) to another database caused the SQL node to crash, and the table to become inaccessible afterwards. (Bug #60484)

  • Under heavy loads with many concurrent inserts, temporary failures in transactions could occur (and were misreported as being due to NDB Error 899 Rowid already allocated). As part of the fix for this issue, NDB Error 899 has been reclassified as an internal error, rather than as a temporary transaction error. (Bug #56051, Bug #11763354)

  • Disk Data: Accounting for MaxNoOfOpenFiles was incorrect with regard to data files in MySQL Cluster Disk Data tablespaces. This could lead to a crash when MaxNoOfOpenFiles was exceeded. (Bug #12581213)

Changes in MySQL Cluster NDB 7.0.24 (5.1.56-ndb-7.0.24)

Functionality Added or Changed

  • It is now possible to add data nodes online to a running MySQL Cluster without performing a rolling restart of the cluster or starting data node processes with the --nowait-nodes option. This can be done by setting Nodegroup = 65536 in the config.ini file for any data nodes that should be started at a later time, when first starting the cluster. (It was possible to set NodeGroup to this value previously, but the management server failed to start.)

    As part of this fix, a new data node configuration parameter StartNoNodeGroupTimeout has been added. When the management server sees that there are data nodes with no node group (that is, nodes for which Nodegroup = 65536), it waits StartNoNodeGroupTimeout milliseconds before treating these nodes as though they were listed with the --nowait-nodes option, and proceeds to start.

    For more information, see Adding MySQL Cluster Data Nodes Online. (Bug #11766167, Bug #59213)

  • A config_generation column has been added to the nodes table of the ndbinfo database. By checking this column, it is now possible to determine which version or versions of the MySQL Cluster configuration file are in effect on the data nodes. This information can be especially useful when performing a rolling restart of the cluster to update its configuration.

Bugs Fixed

  • Cluster API: A unique index operation is executed in two steps: a lookup on an index table, and an operation on the base table. When the operation on the base table failed, while being executed in a batch with other operations that succeeded, this could lead to a hanging execute, eventually timing out with Error 4012 (Request ndbd time-out, maybe due to high load or communication problems). (Bug #12315582)

  • A memory leak in LGMAN, that leaked 8 bytes of log buffer memory per 32k written, was introduced in MySQL Cluster NDB 7.0.9, effecting all MySQL Cluster NDB 7.1 releases as well as MySQL Cluster NDB 7.0.9 and later MySQL Cluster NDB 7.0 releases. (For example, when 128MB log buffer memory was used, it was exhausted after writing 512GB to the undo log.) This led to a GCP stop and data node failure. (Bug #60946)

    References: This issue is a regression of: Bug #47966.

  • When using ndbmtd, a MySQL Cluster configured with 32 data nodes failed to start correctly. (Bug #60943)

  • When performing a TUP scan with locks in parallel, and with a highly concurrent load of inserts and deletions, the scan could sometimes fail to notice that a record had moved while waiting to acquire a lock on it, and so read the wrong record. During node recovery, this could lead to a crash of a node that was copying data to the node being started, and a possible forced shutdown of the cluster.

  • Cluster API: Performing interpreted operations using a unique index did not work correctly, because the interpret bit was kept when sending the lookup to the index table.

Changes in MySQL Cluster NDB 7.0.23 (5.1.51-ndb-7.0.23)

Functionality Added or Changed

  • Improved scaling of ordered index scans performance by removing a hard-coded limit (MAX_PARALLEL_INDEX_SCANS_PER_FRAG) and making the number of TUP or TUX scans per fragment configurable by adding the MaxParallelScansPerFragment data node configuration parameter. (Bug #11769048)

Bugs Fixed

  • Important Change: Formerly, the --ndb-cluster-connection-pool server option set a status variable as well as a system variable. The status variable has been removed as redundant. (Bug #60119)

  • A scan with a pushed condition (filter) using the CommittedRead lock mode could hang for a short interval when it was aborted when just as it had decided to send a batch. (Bug #11932525)

  • When aborting a multi-read range scan exactly as it was changing ranges in the local query handler, LQH could fail to detect it, leaving the scan hanging. (Bug #11929643)

  • Schema distribution did not take place for tables converted from another storage engine to NDB using ALTER TABLE; this meant that such tables were not always visible to all SQL nodes attached to the cluster. (Bug #11894966)

  • A GCI value inserted by ndb_restore --restore_epoch into the ndb_apply_status table was actually 1 less than the correct value. (Bug #11885852)

  • Disk Data: Limits imposed by the size of SharedGlobalMemory were not always enforced consistently with regard to Disk Data undo buffers and log files. This could sometimes cause a CREATE LOGFILE GROUP or ALTER LOGFILE GROUP statement to fail for no apparent reason, or cause the log file group specified by InitialLogFileGroup not to be created when starting the cluster. (Bug #57317)

Changes in MySQL Cluster NDB 7.0.22 (5.1.51-ndb-7.0.22)

Functionality Added or Changed

  • Disk Data: The INFORMATION_SCHEMA.TABLES table now provides disk usage as well as memory usage information for Disk Data tables. Also, INFORMATION_SCHEMA.PARTITIONS, formerly did not show any statistics for NDB tables. Now the TABLE_ROWS, AVG_ROW_LENGTH, DATA_LENGTH, MAX_DATA_LENGTH, and DATA_FREE columns contain correct information for the table's partitions.

  • A new --rewrite-database option is added for ndb_restore, which makes it possible to restore to a database having a different name from that of the database in the backup.

    For more information, see ndb_restore — Restore a MySQL Cluster Backup. (Bug #54327)

  • The NDB kernel now implements a number of statistical counters relating to actions performed by or affecting Ndb objects, such as starting, closing, or aborting transactions; primary key and unique key operations; table, range, and pruned scans; blocked threads waiting for various operations to complete; and data and events sent and received by NDBCLUSTER. These NDB API counters are incremented inside the NDB kernel whenever NDB API calls are made or data is sent to or received by the data nodes. mysqld exposes these counters as system status variables; their values can be read in the output of SHOW STATUS, or by querying the SESSION_STATUS or GLOBAL_STATUS table in the INFORMATION_SCHEMA database. By comparing the values of these status variables prior to and following the execution of SQL statements that act on NDB tables, you can observe the corresponding actions taken on the NDB API level, which can be beneficial for monitoring and performance tuning of MySQL Cluster.

    For more information, see NDB API Statistics Counters and Variables, as well as MySQL Cluster Status Variables.

Bugs Fixed

  • Important Note: Due to an error in merging the original fix, it did not appear MySQL Cluster NDB 7.0.21; this oversight has been corrected in the current release. (Bug #58256)

  • This issue affects all previous MySQL Cluster NDB 7.0 releases. (Bug #60045)

  • ndb_restore --rebuild-indexes caused multi-threaded index building to occur on the master node only. (Bug #59920)

  • Successive queries on the counters table from the same SQL node returned unchanging results. To fix this issue, and to prevent similar issues from occurring in the future, ndbinfo tables are now excluded from the query cache. (Bug #59831)

  • When a CREATE TABLE statement failed due to NDB error 1224 (Too many fragments), it was not possible to create the table afterward unless either it had no ordered indexes, or a DROP TABLE statement was issued first, even if the subsequent CREATE TABLE was valid and should otherwise have succeeded. (Bug #59756)

    References: See also: Bug #59751.

  • When attempting to create a table on a MySQL Cluster with many standby data nodes (setting Nodegroup=65536 in config.ini for the nodes that should wait, starting the nodes that should start immediately with the --nowait-nodes option, and using the CREATE TABLE statement's MAX_ROWS option), mysqld miscalculated the number of fragments to use. This caused the CREATE TABLE to fail.


    The CREATE TABLE failure caused by this issue in turn prevented any further attempts to create the table, even if the table structure was simplified or changed in such a way that the attempt should have succeeded. This ghosting issue is handled in Bug #59756.

    (Bug #59751)

    References: See also: Bug #59756.

  • NDB sometimes treated a simple (not unique) ordered index as unique. (Bug #59519)

  • The logic used in determining whether to collapse a range to a simple equality was faulty. In certain cases, this could cause NDB to treat a range as if it were a primary key lookup when determining the query plan to be used. Although this did not affect the actual result returned by the query, it could in such cases result in inefficient execution of queries due to the use of an inappropriate query plan. (Bug #59517)

  • When a query used multiple references to or instances of the same physical tables, NDB failed to recognize these multiple instances as different tables; in such a case, NDB could incorrectly use condition pushdown on a condition referring to these other instances to be pushed to the data nodes, even though the condition should have been rejected as unpushable, leading to invalid results. (Bug #58791)

  • Cluster API: When calling NdbEventOperation::execute() during a node restart, it was possible to get a spurious error 711 (System busy with node restart, schema operations not allowed when a node is starting). (Bug #59723)

  • Cluster API: When an NDBAPI client application was waiting for more scan results after calling NdbScanOperation::nextResult(), the calling thread sometimes woke up even if no new batches for any fragment had arrived, which was unnecessary, and which could have a negative impact on the application's performance. (Bug #52298)

Changes in MySQL Cluster NDB 7.0.21 (5.1.51-ndb-7.0.21)

Functionality Added or Changed

  • Important Change: The following changes have been made with regard to the TimeBetweenEpochsTimeout data node configuration parameter:

    • The maximum possible value for this parameter has been increased from 32000 milliseconds to 256000 milliseconds.

    • Setting this parameter to zero now has the effect of disabling GCP stops caused by save timeouts, commit timeouts, or both.

    • The current value of this parameter and a warning are written to the cluster log whenever a GCP save takes longer than 1 minute or a GCP commit takes longer than 10 seconds.

    For more information, see Disk Data and GCP Stop errors. (Bug #58383)

  • Added the --skip-broken-objects option for ndb_restore. This option causes ndb_restore to ignore tables corrupted due to missing blob parts tables, and to continue reading from the backup file and restoring the remaining tables. (Bug #54613)

    References: See also: Bug #51652.

  • Cluster API: It is now possible to stop or restart a node even while other nodes are starting, using the MGM API ndb_mgm_stop4() or ndb_mgm_restart4() function, respectively, with the force parameter set to 1. (Bug #58451)

    References: See also: Bug #58319.

Bugs Fixed

  • Cluster API: In some circumstances, very large BLOB read and write operations in MySQL Cluster applications can cause excessive resource usage and even exhaustion of memory. To fix this issue and to provide increased stability when performing such operations, it is now possible to set limits on the volume of BLOB data to be read or written within a given transaction in such a way that when these limits are exceeded, the current transaction implicitly executes any accumulated operations. This avoids an excessive buildup of pending data which can result in resource exhaustion in the NDB kernel. The limits on the amount of data to be read and on the amount of data to be written before this execution takes place can be configured separately. (In other words, it is now possible in MySQL Cluster to specify read batching and write batching that is specific to BLOB data.) These limits can be configured either on the NDB API level, or in the MySQL Server.

    On the NDB API level, four new methods are added to the NdbTransaction object. getMaxPendingBlobReadBytes() and setMaxPendingBlobReadBytes() can be used to get and to set, respectively, the maximum amount of BLOB data to be read that accumulates before this implicit execution is triggered. getMaxPendingBlobWriteBytes() and setMaxPendingBlobWriteBytes() can be used to get and to set, respectively, the maximum volume of BLOB data to be written that accumulates before implicit execution occurs.

    For the MySQL server, two new options are added. The --ndb-blob-read-batch-bytes option sets a limit on the amount of pending BLOB data to be read before triggering implicit execution, and the --ndb-blob-write-batch-bytes option controls the amount of pending BLOB data to be written. These limits can also be set using the mysqld configuration file, or read and set within the mysql client and other MySQL client applications using the corresponding server system variables. (Bug #59113)

  • Two related problems could occur with read-committed scans made in parallel with transactions combining multiple (concurrent) operations:

    1. When committing a multiple-operation transaction that contained concurrent insert and update operations on the same record, the commit arrived first for the insert and then for the update. If a read-committed scan arrived between these operations, it could thus read incorrect data; in addition, if the scan read variable-size data, it could cause the data node to fail.

    2. When rolling back a multiple-operation transaction having concurrent delete and insert operations on the same record, the abort arrived first for the delete operation, and then for the insert. If a read-committed scan arrived between the delete and the insert, it could incorrectly assume that the record should not be returned (in other words, the scan treated the insert as though it had not yet been committed).

    (Bug #59496)

  • On Windows platforms, issuing a SHUTDOWN command in the ndb_mgm client caused management processes that had been started with the --nodaemon option to exit abnormally. (Bug #59437)

  • A row insert or update followed by a delete operation on the same row within the same transaction could in some cases lead to a buffer overflow. (Bug #59242)

    References: See also: Bug #56524. This issue is a regression of: Bug #35208.

  • Data nodes configured with very large amounts (multiple gigabytes) of DiskPageBufferMemory failed during startup with NDB error 2334 (Job buffer congestion). (Bug #58945)

    References: See also: Bug #47984.

  • The FAIL_REP signal, used inside the NDB kernel to declare that a node has failed, now includes the node ID of the node that detected the failure. This information can be useful in debugging. (Bug #58904)

  • When executing a full table scan caused by a WHERE condition using unique_key IS NULL in combination with a join, NDB failed to close the scan. (Bug #58750)

    References: See also: Bug #57481.

  • In some circumstances, an SQL trigger on an NDB table could read stale data. (Bug #58538)

  • During a node takeover, it was possible in some circumstances for one of the remaining nodes to send an extra transaction confirmation (LQH_TRANSCONF) signal to the DBTC kernel block, conceivably leading to a crash of the data node trying to take over as the new transaction coordinator. (Bug #58453)

  • A query having multiple predicates joined by OR in the WHERE clause and which used the sort_union access method (as shown using EXPLAIN) could return duplicate rows. (Bug #58280)

  • Trying to drop an index while it was being used to perform scan updates caused data nodes to crash. (Bug #58277, Bug #57057)

  • When handling failures of multiple data nodes, an error in the construction of internal signals could cause the cluster's remaining nodes to crash. This issue was most likely to affect clusters with large numbers of data nodes. (Bug #58240)

  • The functions strncasecmp and strcasecmp were declared in ndb_global.h but never defined or used. The declarations have been removed. (Bug #58204)

  • Some queries of the form SELECT ... WHERE column IN (subquery) against an NDB table could cause mysqld to hang in an endless loop. (Bug #58163)

  • The number of rows affected by a statement that used a WHERE clause having an IN condition with a value list containing a great many elements, and that deleted or updated enough rows such that NDB processed them in batches, was not computed or reported correctly. (Bug #58040)

  • MySQL Cluster failed to compile correctly on FreeBSD 8.1 due to misplaced #include statements. (Bug #58034)

  • A query using BETWEEN as part of a pushed-down WHERE condition could cause mysqld to hang or crash. (Bug #57735)

  • Data nodes no longer allocated all memory prior to being ready to exchange heartbeat and other messages with management nodes, as in NDB 6.3 and earlier versions of MySQL Cluster. This caused problems when data nodes configured with large amounts of memory failed to show as connected or showed as being in the wrong start phase in the ndb_mgm client even after making their initial connections to and fetching their configuration data from the management server. With this fix, data nodes now allocate all memory as they did in earlier MySQL Cluster versions. (Bug #57568)

  • In some circumstances, it was possible for mysqld to begin a new multi-range read scan without having closed a previous one. This could lead to exhaustion of all scan operation objects, transaction objects, or lock objects (or some combination of these) in NDB, causing queries to fail with such errors as Lock wait timeout exceeded or Connect failure - out of connection objects. (Bug #57481)

    References: See also: Bug #58750.

  • Queries using column IS [NOT] NULL on a table with a unique index created with USING HASH on column always returned an empty result. (Bug #57032)

  • With engine_condition_pushdown enabled, a query using LIKE on an ENUM column of an NDB table failed to return any results. This issue is resolved by disabling engine_condition_pushdown when performing such queries. (Bug #53360)

  • When a slash character (/) was used as part of the name of an index on an NDB table, attempting to execute a TRUNCATE TABLE statement on the table failed with the error Index not found, and the table was rendered unusable. (Bug #38914)

  • Partitioning; Disk Data: When using multi-threaded data nodes, an NDB table created with a very large value for the MAX_ROWS option could—if this table was dropped and a new table with fewer partitions, but having the same table ID, was created—cause ndbmtd to crash when performing a system restart. This was because the server attempted to examine each partition whether or not it actually existed.

    This issue is the same as that reported in Bug #45154, except that the current issue is specific to ndbmtd instead of ndbd. (Bug #58638)

    References: See also: Bug #45154.

  • Disk Data: In certain cases, a race condition could occur when DROP LOGFILE GROUP removed the logfile group while a read or write of one of the effected files was in progress, which in turn could lead to a crash of the data node. (Bug #59502)

  • Disk Data: A race condition could sometimes be created when DROP TABLESPACE was run concurrently with a local checkpoint; this could in turn lead to a crash of the data node. (Bug #59501)

  • Disk Data: Performing what should have been an online drop of a multi-column index was actually performed offline. (Bug #55618)

  • Disk Data: When at least one data node was not running, queries against the INFORMATION_SCHEMA.FILES table took an excessive length of time to complete because the MySQL server waited for responses from any stopped nodes to time out. Now, in such cases, MySQL does not attempt to contact nodes which are not known to be running. (Bug #54199)

  • Cluster API: It was not possible to obtain the status of nodes accurately after an attempt to stop a data node using ndb_mgm_stop() failed without returning an error. (Bug #58319)

  • Cluster API: Attempting to read the same value (using getValue()) more than 9000 times within the same transaction caused the transaction to hang when executed. Now when more reads are performed in this way than can be accommodated in a single transaction, the call to execute() fails with a suitable error. (Bug #58110)

Changes in MySQL Cluster NDB 7.0.20a (5.1.51-ndb-7.0.20a)

Bugs Fixed

  • Important Note: Issuing an ALL DUMP command during a rolling upgrade to MySQL Cluster NDB 7.0.20 caused the cluster to crash. (Bug #58256)

Changes in MySQL Cluster NDB 7.0.20 (5.1.51-ndb-7.0.20)

Functionality Added or Changed

  • Important Change: ndbd now bypasses use of Non-Uniform Memory Access support on Linux hosts by default. If your system supports NUMA, you can enable it and override ndbd use of interleaving by setting the Numa data node configuration parameter which is added in this release. See Defining Data Nodes: Realtime Performance Parameters, for more information. (Bug #57807)

  • Important Change: The Id configuration parameter used with MySQL Cluster management, data, and API nodes (including SQL nodes) is now deprecated, and the NodeId parameter (long available as a synonym for Id when configuring these types of nodes) should be used instead. Id continues to be supported for reasons of backward compatibility, but now generates a warning when used with these types of nodes, and is subject to removal in a future release of MySQL Cluster.

    This change affects the name of the configuration parameter only, establishing a clear preference for NodeId over Id in the [mgmd], [ndbd], [mysql], and [api] sections of the MySQL Cluster global configuration (config.ini) file. The behavior of unique identifiers for management, data, and SQL and API nodes in MySQL Cluster has not otherwise been altered.

    The Id parameter as used in the [computer] section of the MySQL Cluster global configuration file is not affected by this change.

Bugs Fixed

  • Packaging: MySQL Cluster RPM distributions did not include a shared-compat RPM for the MySQL Server, which meant that MySQL applications depending on (MySQL 5.0 and earlier) no longer worked. (Bug #38596)

  • On Windows, the angel process which monitors and (when necessary) restarts the data node process failed to spawn a new worker in some circumstances where the arguments vector contained extra items placed at its beginning. This could occur when the path to ndbd.exe or ndbmtd.exe contained one or more spaces. (Bug #57949)

  • The disconnection of an API or management node due to missed heartbeats led to a race condition which could cause data nodes to crash. (Bug #57946)

  • The method for calculating table schema versions used by schema transactions did not follow the established rules for recording schemas used in the P0.SchemaLog file. (Bug #57897)

    References: See also: Bug #57896.

  • The LQHKEYREQ request message used by the local query handler when checking the major schema version of a table, being only 16 bits wide, could cause this check to fail with an Invalid schema version error (NDB error code 1227). This issue occurred after creating and dropping (and re-creating) the same table 65537 times, then trying to insert rows into the table. (Bug #57896)

    References: See also: Bug #57897.

  • Data nodes compiled with gcc 4.5 or higher crashed during startup. (Bug #57761)

  • Transient errors during a local checkpoint were not retried, leading to a crash of the data node. Now when such errors occur, they are retried up to 10 times if necessary. (Bug #57650)

  • ndb_restore now retries failed transactions when replaying log entries, just as it does when restoring data. (Bug #57618)

  • The SUMA kernel block has a 10-element ring buffer for storing out-of-order SUB_GCP_COMPLETE_REP signals received from the local query handlers when global checkpoints are completed. In some cases, exceeding the ring buffer capacity on all nodes of a node group at the same time caused the node group to fail with an assertion. (Bug #57563)

  • During a GCP takeover, it was possible for one of the data nodes not to receive a SUB_GCP_COMPLETE_REP signal, with the result that it would report itself as GCP_COMMITTING while the other data nodes reported GCP_PREPARING. (Bug #57522)

  • Specifying a WHERE clause of the form range1 OR range2 when selecting from an NDB table having a primary key on multiple columns could result in Error 4259 Invalid set of range scan bounds if range2 started exactly where range1 ended and the primary key definition declared the columns in a different order relative to the order in the table's column list. (Such a query should simply return all rows in the table, since any expression value < constant OR value >= constant is always true.)

    Example.  Suppose t is an NDB table defined by the following CREATE TABLE statement:


    This issue could then be triggered by a query such as this one:

    SELECT * FROM t WHERE b < 8 OR b >= 8;

    In addition, the order of the ranges in the WHERE clause was significant; the issue was not triggered, for example, by the query SELECT * FROM t WHERE b <= 8 OR b > 8. (Bug #57396)

  • A number of cluster log warning messages relating to deprecated configuration parameters contained spelling, formatting, and other errors. (Bug #57381)

  • The MAX_ROWS option for CREATE TABLE was ignored, which meant that it was not possible to enable multi-threaded building of indexes. (Bug #57360)

  • A GCP stop is detected using 2 parameters which determine the maximum time that a global checkpoint or epoch can go unchanged; one of these controls this timeout for GCPs and one controls the timeout for epochs. Suppose the cluster is configured such that TimeBetweenEpochsTimeout is 100 ms but HeartbeatIntervalDbDb is 1500 ms. A node failure can be signalled after 4 missed heartbeats—in this case, 6000 ms. However, this would exceed TimeBetweenEpochsTimeout, causing false detection of a GCP. To prevent this from happening, the configured value for TimeBetweenEpochsTimeout is automatically adjusted, based on the values of HeartbeatIntervalDbDb and ArbitrationTimeout.

    The current issue arose when the automatic adjustment routine did not correctly take into consideration the fact that, during cascading node-failures, several intervals of length 4 * (HeartbeatIntervalDBDB + ArbitrationTimeout) may elapse before all node failures have internally been resolved. This could cause false GCP detection in the event of a cascading node failure. (Bug #57322)

  • Successive CREATE NODEGROUP and DROP NODEGROUP commands could cause mysqld processes to crash. (Bug #57164)

  • Queries using WHERE varchar_pk_column LIKE 'pattern%' or WHERE varchar_pk_column LIKE 'pattern_' against an NDB table having a VARCHAR column as its primary key failed to return all matching rows. (Bug #56853)

  • Aborting a native NDB backup in the ndb_mgm client using the ABORT BACKUP command did not work correctly when using ndbmtd, in some cases leading to a crash of the cluster. (Bug #56285)

  • When a data node angel process failed to fork off a new worker process (to replace one that had failed), the failure was not handled. This meant that the angel process either transformed itself into a worker process, or itself failed. In the first case, the data node continued to run, but there was no longer any angel to restart it in the event of failure, even with StopOnError set to 0. (Bug #53456)

  • Disk Data: When performing online DDL on Disk Data tables, scans and moving of the relevant tuples were done in more or less random order. This fix causes these scans to be done in the order of the tuples, which should improve performance of such operations due to the more sequential ordering of the scans. (Bug #57848)

    References: See also: Bug #57827.

  • Disk Data: Adding unique indexes to NDB Disk Data tables could take an extremely long time. This was particularly noticeable when using ndb_restore --rebuild-indexes. (Bug #57827)

  • Cluster API: An application dropping a table at the same time that another application tried to set up a replication event on the same table could lead to a crash of the data node. The same issue could sometimes cause NdbEventOperation::execute() to hang. (Bug #57886)

  • Cluster API: An NDB API client program under load could abort with an assertion error in TransporterFacade::remove_from_cond_wait_queue. (Bug #51775)

    References: See also: Bug #32708.

Changes in MySQL Cluster NDB 7.0.19 (5.1.47-ndb-7.0.19)

Functionality Added or Changed

  • mysqldump as supplied with MySQL Cluster now has an --add-drop-trigger option which adds a DROP TRIGGER IF EXISTS statement before each dumped trigger definition. (Bug #55691)

    References: See also: Bug #34325, Bug #11747863.

  • It is now possible using the ndb_mgm management client or the MGM API to force a data node shutdown or restart even if this would force the shutdown or restart of the entire cluster.

    In the management client, this is implemented through the addition of the -f (force) option to the STOP and RESTART commands. For more information, see Commands in the MySQL Cluster Management Client.

    The MGM API also adds two new methods for forcing such a node shutdown or restart; see ndb_mgm_stop4(), and ndb_mgm_restart4(), for more information about these methods. (Bug #54226)

  • Cluster API: The MGM API function ndb_mgm_get_version(), which was previously internal, has now been moved to the public API. This function can be used to get NDB storage engine and other version information from the management server. (Bug #51310)

    References: See also: Bug #51273.

Bugs Fixed

  • At startup, an ndbd or ndbmtd process creates directories for its file system without checking to see whether they already exist. Portability code added in MySQL Cluster NDB 7.0.18 and MySQL Cluster NDB 7.1.7 did not account for this fact, printing a spurious error message when a directory to be created already existed. This unneeded printout has been removed. (Bug #57087)

  • A data node can be shut down having completed and synchronized a given GCI x, while having written a great many log records belonging to the next GCI x + 1, as part of normal operations. However, when starting, completing, and synchronizing GCI x + 1, then the log records from original start must not be read. To make sure that this does not happen, the REDO log reader finds the last GCI to restore, scans forward from that point, and erases any log records that were not (and should never be) used.

    The current issue occurred because this scan stopped immediately as soon as it encountered an empty page. This was problematic because the REDO log is divided into several files; thus, it could be that there were log records in the beginning of the next file, even if the end of the previous file was empty. These log records were never invalidated; following a start or restart, they could be reused, leading to a corrupt REDO log. (Bug #56961)

  • An error in program flow in ndbd.cpp could result in data node shutdown routines being called multiple times. (Bug #56890)

  • Under certain rare conditions, attempting to start more than one ndb_mgmd process simultaneously using the --reload option caused a race condition such that none of the ndb_mgmd processes could start. (Bug #56844)

  • When distributing CREATE TABLE and DROP TABLE operations among several SQL nodes attached to a MySQL Cluster. the LOCK_OPEN lock normally protecting mysqld's internal table list is released so that other queries or DML statements are not blocked. However, to make sure that other DDL is not executed simultaneously, a global schema lock (implemented as a row-level lock by NDB) is used, such that all operations that can modify the state of the mysqld internal table list also need to acquire this global schema lock. The SHOW TABLE STATUS statement did not acquire this lock. (Bug #56841)

  • In certain cases, DROP DATABASE could sometimes leave behind a cached table object, which caused problems with subsequent DDL operations. (Bug #56840)

  • Memory pages used for DataMemory, once assigned to ordered indexes, were not ever freed, even after any rows that belonged to the corresponding indexes had been deleted. (Bug #56829)

  • MySQL Cluster stores, for each row in each NDB table, a Global Checkpoint Index (GCI) which identifies the last committed transaction that modified the row. As such, a GCI can be thought of as a coarse-grained row version.

    Due to changes in the format used by NDB to store local checkpoints (LCPs) in MySQL Cluster NDB 6.3.11, it could happen that, following cluster shutdown and subsequent recovery, the GCI values for some rows could be changed unnecessarily; this could possibly, over the course of many node or system restarts (or both), lead to an inconsistent database. (Bug #56770)

  • When multiple SQL nodes were connected to the cluster and one of them stopped in the middle of a DDL operation, the mysqld process issuing the DDL timed out with the error distributing tbl_name timed out. Ignoring. (Bug #56763)

  • An online ALTER TABLE ... ADD COLUMN operation that changed the table schema such that the number of 32-bit words used for the bitmask allocated to each DML operation increased during a transaction in DML which was performed prior to DDL which was followed by either another DML operation or—if using replication—a commit, led to data node failure.

    This was because the data node did not take into account that the bitmask for the before-image was smaller than the current bitmask, which caused the node to crash. (Bug #56524)

    References: This issue is a regression of: Bug #35208.

  • On Windows, a data node refused to start in some cases unless the ndbd.exe executable was invoked using an absolute rather than a relative path. (Bug #56257)

  • The text file cluster_change_hist.txt containing old MySQL Cluster changelog information was no longer being maintained, and so has been removed from the tree. (Bug #56116)

  • The failure of a data node during some scans could cause other data nodes to fail. (Bug #54945)

  • Exhausting the number of available commit-ack markers (controlled by the MaxNoOfConcurrentTransactions parameter) led to a data node crash. (Bug #54944)

  • When running a SELECT on an NDB table with BLOB or TEXT columns, memory was allocated for the columns but was not freed until the end of the SELECT. This could cause problems with excessive memory usage when dumping (using for example mysqldump) tables with such columns and having many rows, large column values, or both. (Bug #52313)

    References: See also: Bug #56488, Bug #50310.

  • Cluster API: The MGM API functions ndb_mgm_stop() and ndb_mgm_restart() set the error code and message without first checking whether the management server handle was NULL, which could lead to fatal errors in MGM API applications that depended on these functions. (Bug #57089)

  • Cluster API: The MGM API function ndb_mgm_get_version() did not set the error message before returning with an error. With this fix, it is now possible to call ndb_mgm_get_latest_error() after a failed call to this function such that ndb_mgm_get_latest_error() returns an error number and error message, as expected of MGM API calls. (Bug #57088)

Changes in MySQL Cluster NDB 7.0.18 (5.1.47-ndb-7.0.18)

Functionality Added or Changed

Bugs Fixed

  • ndb_restore always reported 0 for the GCPStop (end point of the backup). Now it provides useful binary log position and epoch information. (Bug #56298)

  • The LockExecuteThreadToCPU configuration parameter was not handled correctly for CPU ID values greater than 255. (Bug #56185)

  • Following a failure of the master data node, the new master sometimes experienced a race condition which caused the node to terminate with a GcpStop error. (Bug #56044)

  • Trying to create a table having a BLOB or TEXT column with DEFAULT '' failed with the error Illegal null attribute. (An empty default is permitted and ignored by MyISAM; NDB should do the same.) (Bug #55121)

  • ndb_mgmd --nodaemon logged to the console in addition to the configured log destination. (Bug #54779)

  • The warning MaxNoOfExecutionThreads (#) > LockExecuteThreadToCPU count (#), this could cause contention could be logged when running ndbd, even though the condition described can occur only when using ndbmtd. (Bug #54342)

  • Startup messages previously written by ndb_mgmd to stdout are now written to the cluster log instead when LogDestination is set. (Bug #47595)

  • The graceful shutdown of a data node could sometimes cause transactions to be aborted unnecessarily. (Bug #18538)

    References: See also: Bug #55641.

Changes in MySQL Cluster NDB 7.0.17 (5.1.47-ndb-7.0.17)

Functionality Added or Changed

  • Added the DictTrace data node configuration parameter, for use in debugging NDB code. For more information, see Defining MySQL Cluster Data Nodes. (Bug #55963)

  • Added the --server-id-bits option for mysqld and mysqlbinlog.

    For mysqld, the --server-id-bits option indicates the number of least significant bits within the 32-bit server ID which actually identify the server. Indicating that the server ID uses less than 32 bits permits the remaining bits to be used for other purposes by NDB API applications using the Event API and OperationOptions::anyValue.

    For mysqlbinlog, the --server-id-bits option tells mysqlbinlog how to interpret the server IDs in the binary log when the binary log was written by a mysqld having its server_id_bits set to less than the maximum (32). (Bug #52305)

Bugs Fixed

  • Important Change; Cluster API: The poll and select calls made by the MGM API were not interrupt-safe; that is, a signal caught by the process while waiting for an event on one or more sockets returned error -1 with errno set to EINTR. This caused problems with MGM API functions such as ndb_logevent_get_next() and ndb_mgm_get_status2().

    To fix this problem, the internal ndb_socket_poller::poll() function has been made EINTR-safe.

    The old version of this function has been retained as poll_unsafe(), for use by those parts of NDB that do not need the EINTR-safe version of the function. (Bug #55906)

  • The TCP configuration parameters HostName1 and HostName2 were not displayed in the output of ndb_config --configinfo. (Bug #55839)

  • When another data node failed, a given data node DBTC kernel block could time out while waiting for DBDIH to signal commits of pending transactions, leading to a crash. Now in such cases the timeout generates a prinout, and the data node continues to operate. (Bug #55715)

  • Starting ndb_mgmd with --config-cache=0 caused it to leak memory. (Bug #55205)

  • The configure.js option WITHOUT_DYNAMIC_PLUGINS=TRUE was ignored when building MySQL Cluster for Windows using CMake. Among the effects of this issue was that CMake attempted to build the InnoDB storage engine as a plugin (.DLL file) even though the InnoDB Plugin is not currently supported by MySQL Cluster. (Bug #54913)

  • It was possible for a DROP DATABASE statement to remove NDB hidden blob tables without removing the parent tables, with the result that the tables, although hidden to MySQL clients, were still visible in the output of ndb_show_tables but could not be dropped using ndb_drop_table. (Bug #54788)

  • An excessive number of timeout warnings (normally used only for debugging) were written to the data node logs. (Bug #53987)

  • Disk Data: As an optimization when inserting a row to an empty page, the page is not read, but rather simply initialized. However, this optimzation was performed in all cases when an empty row was inserted, even though it should have been done only if it was the first time that the page had been used by a table or fragment. This is because, if the page had been in use, and then all records had been released from it, the page still needed to be read to learn its log sequence number (LSN).

    This caused problems only if the page had been flushed using an incorrect LSN and the data node failed before any local checkpoint was completed—which would remove any need to apply the undo log, hence the incorrect LSN was ignored.

    The user-visible result of the incorrect LSN was that it caused the data node to fail during a restart. It was perhaps also possible (although not conclusively proven) that this issue could lead to incorrect data. (Bug #54986)

  • Cluster API: Calling NdbTransaction::refresh() did not update the timer for TransactionInactiveTimeout. (Bug #54724)

Changes in MySQL Cluster NDB 7.0.16 (5.1.47-ndb-7.0.16)

Functionality Added or Changed

  • Restrictions on some types of mismatches in column definitions when restoring data using ndb_restore have been relaxed. These include the following types of mismatches:

    • Different COLUMN_FORMAT settings (FIXED, DYNAMIC, DEFAULT)

    • Different STORAGE settings (MEMORY, DISK)

    • Different default values

    • Different distribution key settings

    Now, when one of these types of mismatches in column definitions is encountered, ndb_restore no longer stops with an error; instead, it accepts the data and inserts it into the target table, while issuing a warning to the user.

    For more information, see ndb_restore — Restore a MySQL Cluster Backup. (Bug #54423)

    References: See also: Bug #53810, Bug #54178, Bug #54242, Bug #54279.

  • Introduced the HeartbeatOrder data node configuration parameter, which can be used to set the order in which heartbeats are transmitted between data nodes. This parameter can be useful in situations where multiple data nodes are running on the same host and a temporary disruption in connectivity between hosts would otherwise cause the loss of a node group, leading to failure of the cluster. (Bug #52182)

  • It is now possible to install management node and data node processes as Windows services. (See Installing MySQL Cluster Processes as Windows Services, for more information.) In addition, data node processes on Windows are now maintained by angel processes, just as they are on other platforms supported by MySQL Cluster.

Bugs Fixed

  • The disconnection of all API nodes (including SQL nodes) during an ALTER TABLE caused a memory leak. (Bug #54685)

  • If a node shutdown (either in isolation or as part of a system shutdown) occurred directly following a local checkpoint, it was possible that this local checkpoint would not be used when restoring the cluster. (Bug #54611)

  • The setting for BuildIndexThreads was ignored by ndbmtd, which made it impossible to use more than 4 cores for rebuilding indexes. (Bug #54521)

  • When adding multiple new node groups to a MySQL Cluster, it was necessary for each new node group to add only the nodes to be assigned to the new node group, create that node group using CREATE NODEGROUP, then repeat this process for each new node group to be added to the cluster. The fix for this issue makes it possible to add all of the new nodes at one time, and then issue several CREATE NODEGROUP commands in succession. (Bug #54497)

  • When performing an online alter table where 2 or more SQL nodes connected to the cluster were generating binary logs, an incorrect message could be sent from the data nodes, causing mysqld processes to crash. This problem was often difficult to detect, because restarting SQL node or data node processes could clear the error, and because the crash in mysqld did not occur until several minutes after the erroneous message was sent and received. (Bug #54168)

  • A table having the maximum number of attributes permitted could not be backed up using the ndb_mgm client.


    The maximum number of attributes supported per table is not the same for all MySQL Cluster releases. See Limits Associated with Database Objects in MySQL Cluster, to determine the maximum that applies in the release which you are using.

    (Bug #54155)

  • During initial node restarts, initialization of the REDO log was always performed 1 node at a time, during start phase 4. Now this is done during start phase 2, so that the initialization can be performed in parallel, thus decreasing the time required for initial restarts involving multiple nodes. (Bug #50062)

  • The presence of duplicate [tcp] sections in the config.ini file caused the management server to crash. Now in such cases, ndb_mgmd fails gracefully with an appropriate error message. (Bug #49400)

  • The two MySQL Server options, --ndb-wait-connected and --ndb-wait-setup, did not set the corresponding system variables. (Bug #48402)

  • Cluster API: When using the NDB API, it was possible to rename a table with the same name as that of an existing table.


    This issue did not affect table renames executed using SQL on MySQL servers acting as MySQL Cluster API nodes.

    (Bug #54651)

  • Cluster API: An excessive number of client connections, such that more than 1024 file descriptors, sockets, or both were open, caused NDB API applications to crash. (Bug #34303)

Changes in MySQL Cluster NDB 7.0.15b (5.1.44-ndb-7.0.15b)

Bugs Fixed

  • Cluster API: The value of an internal constant used in the implementation of the NdbOperation and NdbScanOperation classes caused MySQL Cluster NDB 7.0 NDB API applications compiled against MySQL Cluster NDB 7.0.14 or earlier to fail when run with MySQL Cluster 7.0.15, and MySQL Cluster NDB 7.1 NDB API applications compiled against MySQL Cluster NDB 7.1.3 or earlier to break when used with MySQL Cluster 7.1.4. (Bug #54516)

Changes in MySQL Cluster NDB 7.0.15a (5.1.44-ndb-7.0.15a)

Bugs Fixed

  • When using mysqldump to back up and restore schema information while using ndb_restore for restoring only the data, restoring to MySQL Cluster NDB 7.1.4 from an older version failed on tables having columns with default values. This was because versions of MySQL Cluster prior to MySQL Cluster NDB 7.1.4 did not have native support for default values.

    In addition, the MySQL Server supports TIMESTAMP columns having dynamic default values, such as DEFAULT CURRENT_TIMESTAMP; however, the current implementation of NDB-native default values permits only a constant default value.

    To fix this issue, the manner in which NDB treats TIMESTAMP columns is reverted to its pre-NDB-7.1.4 behavior (obtaining the default value from mysqld rather than NDBCLUSTER) except where a TIMESTAMP column uses a constant default, as in the case of a column declared as TIMESTAMP DEFAULT 0 or TIMESTAMP DEFAULT 20100607174832. (Bug #54242)

Changes in MySQL Cluster NDB 7.0.15 (5.1.44-ndb-7.0.15)

Functionality Added or Changed

  • Important Change: The maximum number of attributes (columns plus indexes) per table has increased to 512.

  • A --wait-nodes option has been added for ndb_waiter. When this option is used, the program waits only for the nodes having the listed IDs to reach the desired state. For more information, see ndb_waiter — Wait for MySQL Cluster to Reach a Given Status. (Bug #52323)

  • As part of this change, new methods relating to default values have been added to the Column and Table classes in the NDB API. For more information, see Column::getDefaultValue(), Column::setDefaultValue(), and Table::hasDefaultValues(). (Bug #30529)

  • Added the MySQL Cluster management server option --config-cache, which makes it possible to enable and disable configuration caching. This option is turned on by default; to disable configuration caching, start ndb_mgmd with --config-cache=0, or with --skip-config-cache. See ndb_mgmd — The MySQL Cluster Management Server Daemon, for more information.

  • Added the --skip-unknown-objects option for ndb_restore. This option causes ndb_restore to ignore any schema objects which it does not recognize. Currently, this is useful chiefly for restoring native backups made from a cluster running MySQL Cluster NDB 7.0 to a cluster running MySQL Cluster NDB 6.3.

Bugs Fixed

  • When attempting to create an NDB table on an SQL node that had not yet connected to a MySQL Cluster management server since the SQL node's last restart, the CREATE TABLE statement failed as expected, but with the unexpected Error 1495 For the partitioned engine it is necessary to define all partitions. (Bug #11747335, Bug #31853)

  • After creating NDB tables until creation of a table failed due to NDB error 905 Out of attribute records (increase MaxNoOfAttributes), then increasing MaxNoOfAttributes and restarting all management node and data node processes, attempting to drop and re-create one of the tables failed with the error Out of table records..., even when sufficient table records were available. (Bug #53944)

    References: See also: Bug #52055. This issue is a regression of: Bug #44294.

  • Creating a Disk Data table, dropping it, then creating an in-memory table and performing a restart, could cause data node processes to fail with errors in the DBTUP kernel block if the new table's internal ID was the same as that of the old Disk Data table. This could occur because undo log handling during the restart did not check that the table having this ID was now in-memory only. (Bug #53935)

  • A table created while ndb_table_no_logging was enabled was not always stored to disk, which could lead to a data node crash with Error opening DIH schema files for table. (Bug #53934)

  • An internal buffer allocator used by NDB has the form alloc(wanted, minimum) and attempts to allocate wanted pages, but is permitted to allocate a smaller number of pages, between wanted and minimum. However, this allocator could sometimes allocate fewer than minimum pages, causing problems with multi-threaded building of ordered indexes. (Bug #53580)

  • When compiled with support for epoll but this functionality is not available at runtime, MySQL Cluster tries to fall back to use the select() function in its place. However, an extra ndbout_c() call in the transporter registry code caused ndbd to fail instead. (Bug #53482)

  • The value set for the ndb_mgmd option --ndb-nodeid was not verified prior to use as being within the permitted range (1 to 255, inclusive), leading to a crash of the management server. (Bug #53412)

  • NDB truncated a column declared as DECIMAL(65,0) to a length of 64. Now such a column is accepted and handled correctly. In cases where the maximum length (65) is exceeded, NDB now raises an error instead of truncating. (Bug #53352)

  • When an NDB log handler failed, the memory allocated to it was freed twice. (Bug #53200)

  • Setting DataMemory higher than 4G on 32-bit platforms caused ndbd to crash, instead of failing gracefully with an error. (Bug #52536, Bug #50928)

  • When the LogDestination parameter was set using with a relative path, the management server failed to store its value unless started with --initial or --reload. (Bug #52268)

  • When creating an index, NDB failed to check whether the internal ID allocated to the index was within the permissible range, leading to an assertion. This issue could manifest itself as a data node failure with NDB error 707 (No more table metadata records (increase MaxNoOfTables)), when creating tables in rapid succession (for example, by a script, or when importing from mysqldump), even with a relatively high value for MaxNoOfTables and a relatively low number of tables. (Bug #52055)

  • ndb_restore did not raise any errors if hashmap creation failed during execution. (Bug #51434)

  • Specifying the node ID as part of the --ndb-connectstring option to mysqld was not handled correctly.

    The fix for this issue includes the following changes:

    • Multiple occurrences of any of the mysqld options --ndb-connectstring, --ndb-mgmd-host, and --ndb-nodeid are now handled in the same way as with other MySQL server options, in that the value set in the last occurrence of the option is the value that is used by mysqld.

      Now, if --ndb-nodeid is used, its value overrides that of any nodeid setting used in --ndb-connectstring. For example, starting mysqld with --ndb-connectstring=nodeid=1, --ndb-nodeid=3 now produces the same result as starting it with --ndb-connectstring=nodeid=3,

    • The 1024-character limit on the length of the connection string is removed, and --ndb-connectstring is now handled in this regard in the same way as other mysqld options.

    • In the NDB API, a new constructor for Ndb_cluster_connection is added which takes as its arguments a connection string and the node ID to force the API node to use.

    (Bug #44299)

  • NDB did not distinguish correctly between table names differing only by lettercase when lower_case_table_names was set to 0. (Bug #33158)

  • ndb_mgm -e "ALL STATUS" erroneously reported that data nodes remained in start phase 0 until they had actually started.

Changes in MySQL Cluster NDB 7.0.14 (5.1.44-ndb-7.0.14)

Functionality Added or Changed

  • Cluster API: It is now possible to determine, using the ndb_desc utility or the NDB API, which data nodes contain replicas of which partitions. For ndb_desc, a new --extra-node-info option is added to cause this information to be included in its output. A new method Table::getFragmentNodes() is added to the NDB API for obtaining this information programmatically. (Bug #51184)

  • Formerly, the REPORT and DUMP commands returned output to all ndb_mgm clients connected to the same MySQL Cluster. Now, these commands return their output only to the ndb_mgm client that actually issued the command. (Bug #40865)

Bugs Fixed

  • Incompatible Change; Cluster API: The default behavior of the NDB API Event API has changed as follows:

    Previously, when creating an Event, DDL operations (alter and drop operations on tables) were automatically reported on any event operation that used this event, but as a result of this change, this is no longer the case. Instead, you must now invoke the event's setReport() method, with the new EventReport value ER_DDL, to get this behavior.

    For existing NDB API applications where you wish to retain the old behavior, you must update the code as indicated previously, then recompile, following an upgrade. Otherwise, DDL operations are no longer reported after upgrading libndbnclient.

    For more information, see The Event::EventReport Type, and Event::setReport(). (Bug #53308)

  • If a node or cluster failure occurred while mysqld was scanning the ndb.ndb_schema table (which it does when attempting to connect to the cluster), insufficient error handling could lead to a crash by mysqld in certain cases. This could happen in a MySQL Cluster with a great many tables, when trying to restart data nodes while one or more mysqld processes were restarting. (Bug #52325)

  • In MySQL Cluster NDB 7.0 and later, DDL operations are performed within schema transactions; the NDB kernel code for starting a schema transaction checks that all data nodes are at the same version before permitting a schema transaction to start. However, when a version mismatch was detected, the client was not actually informed of this problem, which caused the client to hang. (Bug #52228)

  • After running a mixed series of node and system restarts, a system restart could hang or fail altogether. This was caused by setting the value of the newest completed global checkpoint too low for a data node performing a node restart, which led to the node reporting incorrect GCI intervals for its first local checkpoint. (Bug #52217)

  • When performing a complex mix of node restarts and system restarts, the node that was elected as master sometimes required optimized node recovery due to missing REDO information. When this happened, the node crashed with Failure to recreate object ... during restart, error 721 (because the DBDICT restart code was run twice). Now when this occurs, node takeover is executed immediately, rather than being made to wait until the remaining data nodes have started. (Bug #52135)

    References: See also: Bug #48436.

  • The internal variable ndb_new_handler, which is no longer used, has been removed. (Bug #51858)

  • was not compiled with the same SAFEMALLOC and SAFE_MUTEX flags as the MySQL Server. (Bug #51857)

  • When debug compiling MySQL Cluster on Windows, the mysys library was not compiled with -DSAFEMALLOC and -DSAFE_MUTEX, due to the fact that my_socket.c was misnamed as (Bug #51856)

  • The redo log protects itself from being filled up by periodically checking how much space remains free. If insufficient redo log space is available, it sets the state TAIL_PROBLEM which results in transactions being aborted with error code 410 (out of redo log). However, this state was not set following a node restart, which meant that if a data node had insufficient redo log space following a node restart, it could crash a short time later with Fatal error due to end of REDO log. Now, this space is checked during node restarts. (Bug #51723)

  • Restoring a MySQL Cluster backup between platforms having different endianness failed when also restoring metadata and the backup contained a hashmap not already present in the database being restored to. This issue was discovered when trying to restore a backup made on Solaris/SPARC to a MySQL Cluster running on Solaris/x86, but could conceivably occur in other cases where the endianness of the platform on which the backup was taken differed from that of the platform being restored to. (Bug #51432)

  • The output of the ndb_mgm client REPORT BACKUPSTATUS command could sometimes contain errors due to uninitialized data. (Bug #51316)

  • A GROUP BY query against NDB tables sometimes did not use any indexes unless the query included a FORCE INDEX option. With this fix, indexes are used by such queries (where otherwise possible) even when FORCE INDEX is not specified. (Bug #50736)

  • The following issues were fixed in the ndb_mgm client REPORT MEMORYUSAGE command:

    • The client sometimes inserted extra ndb_mgm> prompts within the output.

    • For data nodes running ndbmtd, IndexMemory was reported before DataMemory.

    • In addition, for data nodes running ndbmtd, there were multiple IndexMemory entries listed in the output.

    (Bug #50196)

  • Issuing a command in the ndb_mgm client after it had lost its connection to the management server could cause the client to crash. (Bug #49219)

  • The mysql client system command did not work properly. This issue was only known to affect the version of the mysql client that was included with MySQL Cluster NDB 7.0 and MySQL Cluster NDB 7.1 releases. (Bug #48574)

  • The internal ErrorReporter::formatMessage() method could in some cases cause a buffer overflow. (Bug #47120)

  • Information about several management client commands was missing from (that is, truncated in) the output of the HELP command. (Bug #46114)

  • The ndb_print_backup_file utility failed to function, due to a previous internal change in the NDB code. (Bug #41512, Bug #48673)

  • When the MemReportFrequency configuration parameter was set in config.ini, the ndb_mgm client REPORT MEMORYUSAGE command printed its output multiple times. (Bug #37632)

  • ndb_mgm -e "... REPORT ..." did not write any output to stdout.

    The fix for this issue also prevents the cluster log from being flooded with INFO messages when DataMemory usage reaches 100%, and insures that when the usage is decreased, an appropriate message is written to the cluster log. (Bug #31542, Bug #44183, Bug #49782)

  • Disk Data: Inserts of blob column values into a MySQL Cluster Disk Data table that exhausted the tablespace resulted in misleading no such tuple error messages rather than the expected error tablespace full.

    This issue appeared similar to Bug #48113, but had a different underlying cause. (Bug #52201)

    References: See also: Bug #48113.

  • Disk Data: The error message returned after atttempting to execute ALTER LOGFILE GROUP on an nonexistent logfile group did not indicate the reason for the failure. (Bug #51111)

  • Disk Data: DDL operations on Disk Data tables having a relatively small UNDO_BUFFER_SIZE could fail unexpectedly.

  • Cluster API: When reading blob data with lock mode LM_SimpleRead, the lock was not upgraded as expected. (Bug #51034)

  • Cluster API: A number of issues were corrected in the NDB API coding examples found in the storage/ndb/ndbapi-examples directory in the MySQL Cluster source tree. These included possible endless recursion in ndbapi_scan.cpp as well as problems running some of the examples on systems using Windows or OS X due to the lettercase used for some table names. (Bug #30552, Bug #30737)

Changes in MySQL Cluster NDB 7.0.13 (5.1.41-ndb-7.0.13)

Functionality Added or Changed

  • A new configuration parameter HeartbeatThreadPriority makes it possible to select between a first-in, first-out or round-round scheduling policy for management node and API node heartbeat threads, as well as to set the priority of these threads. See Defining a MySQL Cluster Management Server, or Defining SQL and Other API Nodes in a MySQL Cluster, for more information. (Bug #49617)

  • Start phases are now written to the data node logs. (Bug #49158)

  • Disk Data: The ndb_desc utility can now show the extent space and free extent space for subordinate BLOB and TEXT columns (stored in hidden BLOB tables by NDB). A --blob-info option has been added for this program that causes ndb_desc to generate a report for each subordinate BLOB table. For more information, see ndb_desc — Describe NDB Tables. (Bug #50599)

Bugs Fixed

  • When performing a system restart of a MySQL Cluster where multi-threaded data nodes were in use, there was a slight risk that the restart would hang due to incorrect serialization of signals passed between LQH instances and proxies; some signals were sent using a proxy, and others directly, which meant that the order in which they were sent and received could not be guaranteed. If signals arrived in the wrong order, this could cause one or more data nodes to hang. Now all signals that need to be sent and received in the same order are sent using the same path. (Bug #51645)

  • When one or more data nodes read their LCPs and applied undo logs significantly faster than others, this could lead to a race condition causing system restarts of data nodes to hang. This could most often occur when using both ndbd and ndbmtd processes for the data nodes. (Bug #51644)

  • When deciding how to divide the REDO log, the DBDIH kernel block saved more than was needed to restore the previous local checkpoint, which could cause REDO log space to be exhausted prematurely (NDB error 410). (Bug #51547)

  • DML operations can fail with NDB error 1220 (REDO log files overloaded...) if the opening and closing of REDO log files takes too much time. If this occurred as a GCI marker was being written in the REDO log while REDO log file 0 was being opened or closed, the error could persist until a GCP stop was encountered. This issue could be triggered when there was insufficient REDO log space (for example, with configuration parameter settings NoOfFragmentLogFiles = 6 and FragmentLogFileSize = 6M) with a load including a very high number of updates. (Bug #51512)

    References: See also: Bug #20904.

  • A side effect of the ndb_restore --disable-indexes and --rebuild-indexes options is to change the schema versions of indexes. When a mysqld later tried to drop a table that had been restored from backup using one or both of these options, the server failed to detect these changed indexes. This caused the table to be dropped, but the indexes to be left behind, leading to problems with subsequent backup and restore operations. (Bug #51374)

  • ndb_restore crashed while trying to restore a corrupted backup, due to missing error handling. (Bug #51223)

  • The ndb_restore message Successfully created index `PRIMARY`... was directed to stderr instead of stdout. (Bug #51037)

  • When using NoOfReplicas equal to 1 or 2, if data nodes from one node group were restarted 256 times and applications were running traffic such that it would encounter NDB error 1204 (Temporary failure, distribution changed), the live node in the node group would crash, causing the cluster to crash as well. The crash occurred only when the error was encountered on the 256th restart; having the error on any previous or subsequent restart did not cause any problems. (Bug #50930)

  • The AUTO_INCREMENT option for ALTER TABLE did not reset AUTO_INCREMENT columns of NDB tables. (Bug #50247)

  • Replication of a MySQL Cluster using multi-threaded data nodes could fail with forced shutdown of some data nodes due to the fact that ndbmtd exhausted LongMessageBuffer much more quickly than ndbd. After this fix, passing of replication data between the DBTUP and SUMA NDB kernel blocks is done using DataMemory rather than LongMessageBuffer.

    Until you can upgrade, you may be able to work around this issue by increasing the LongMessageBuffer setting; doubling the default should be sufficient in most cases. (Bug #46914)

  • A SELECT requiring a sort could fail with the error Can't find record in 'table' when run concurrently with a DELETE from the same table. (Bug #45687)

  • Disk Data: For a Disk Data tablespace whose extent size was not equal to a whole multiple of 32K, the value of the FREE_EXTENTS column in the INFORMATION_SCHEMA.FILES table was smaller than the value of TOTAL_EXTENTS.

    As part of this fix, the implicit rounding of INITIAL_SIZE, EXTENT_SIZE, and UNDO_BUFFER_SIZE performed by NDBCLUSTER (see CREATE TABLESPACE Syntax) is now done explicitly, and the rounded values are used for calculating INFORMATION_SCHEMA.FILES column values and other purposes. (Bug #49709)

    References: See also: Bug #31712.

  • Disk Data: Once all data files associated with a given tablespace had been dropped, there was no way for MySQL client applications (including the mysql client) to tell that the tablespace still existed. To remedy this problem, INFORMATION_SCHEMA.FILES now holds an additional row for each tablespace. (Previously, only the data files in each tablespace were shown.) This row shows TABLESPACE in the FILE_TYPE column, and NULL in the FILE_NAME column. (Bug #31782)

  • Disk Data: It was possible to issue a CREATE TABLESPACE or ALTER TABLESPACE statement in which INITIAL_SIZE was less than EXTENT_SIZE. (In such cases, INFORMATION_SCHEMA.FILES erroneously reported the value of the FREE_EXTENTS column as 1 and that of the TOTAL_EXTENTS column as 0.) Now when either of these statements is issued such that INITIAL_SIZE is less than EXTENT_SIZE, the statement fails with an appropriate error message. (Bug #31712)

    References: See also: Bug #49709.

  • Cluster API: An issue internal to ndb_mgm could cause problems when trying to start a large number of data nodes at the same time. (Bug #51273)

    References: See also: Bug #51310.

Changes in MySQL Cluster NDB 7.0.12b (5.1.41-ndb-7.0.12b)

Bugs Fixed

  • Setting IndexMemory greater than 2GB could cause data nodes to crash while starting. (Bug #51256)

Changes in MySQL Cluster NDB 7.0.12a (5.1.41-ndb-7.0.12a)

Bugs Fixed

  • An initial restart of a data node configured with a large amount of memory could fail with a Pointer too large error. (Bug #51027)

    References: This issue is a regression of: Bug #47818.

Changes in MySQL Cluster NDB 7.0.12 (5.1.41-ndb-7.0.12)

Functionality Added or Changed

  • Numeric codes used in management server status update messages in the cluster logs have been replaced with text descriptions. (Bug #49627)

    References: See also: Bug #44248.

Bugs Fixed

  • ndbmtd started on a single-core machine could sometimes fail with a Job Buffer Full error when MaxNoOfExecutionThreads was set greater than LockExecuteThreadToCPU. Now a warning is logged when this occurs. (Bug #50582)

  • When a primary key lookup on an NDB table containing one or more BLOB columns was executed in a transaction, a shared lock on any blob tables used by the NDB table was held for the duration of the transaction. (This did not occur for indexed or non-indexed WHERE conditions.)

    Now in such cases, the lock is released after all BLOB data has been read. (Bug #49190)

Changes in MySQL Cluster NDB 7.0.11b (5.1.41-ndb-7.0.11b)

Bugs Fixed

  • Setting IndexMemory greater than 2GB could cause data nodes to crash while starting. (Bug #51256)

Changes in MySQL Cluster NDB 7.0.11a (5.1.41-ndb-7.0.11a)

Bugs Fixed

  • An initial restart of a data node configured with a large amount of memory could fail with a Pointer too large error. (Bug #51027)

    References: This issue is a regression of: Bug #47818.

Changes in MySQL Cluster NDB 7.0.11 (5.1.41-ndb-7.0.11)

Functionality Added or Changed

  • Important Change: The maximum permitted value of the ndb_autoincrement_prefetch_sz system variable has been increased from 256 to 65536. (Bug #50621)

  • Added multi-threaded ordered index building capability during system restarts or node restarts, controlled by the BuildIndexThreads data node configuration parameter (also introduced in this release).

Bugs Fixed

  • Initial start of partitioned nodes did not work correctly. This issue was observed in MySQL Cluster NDB 7.0 only. (Bug #50661)

  • The CREATE NODEGROUP client command in ndb_mgm could sometimes cause the forced shutdown of a data node. (Bug #50594)

  • Local query handler information was not reported or written to the cluster log correctly. This issue is thought to have been introduced in MySQL Cluster NDB 7.0.10. (Bug #50467)

  • Online upgrades from MySQL Cluster NDB 7.0.9b to MySQL Cluster NDB 7.0.10 did not work correctly. Current MySQL Cluster NDB 7.0 users should upgrade directly to MySQL Cluster NDB 7.0.11 or later.

    This issue is not known to have affected MySQL Cluster NDB 6.3, and it should be possible to upgrade from MySQL Cluster NDB 6.3 to MySQL Cluster NDB 7.0.10 without problems. See Upgrade and Downgrade Compatibility: MySQL Cluster NDB 6.x, for more information. (Bug #50433)

  • Dropping unique indexes in parallel while they were in use could cause node and cluster failures. (Bug #50118)

  • When attempting to join a running cluster whose management server had been started with the --nowait-nodes option and having SQL nodes with dynamically allocated node IDs, a second management server failed with spurious INTERNAL ERROR: Found dynamic ports with value in config... error messages. (Bug #49807)

  • When setting the LockPagesInMainMemory configuration parameter failed, only the error Failed to memlock pages... was returned. Now in such cases the operating system's error code is also returned. (Bug #49724)

  • If a query on an NDB table compared a constant string value to a column, and the length of the string was greater than that of the column, condition pushdown did not work correctly. (The string was truncated to fit the column length before being pushed down.) Now in such cases, the condition is no longer pushed down. (Bug #49459)

  • ndbmtd was not built on Windows (CMake did not provide a build target for it). (Bug #49325)

  • Performing intensive inserts and deletes in parallel with a high scan load could a data node crashes due to a failure in the DBACC kernel block. This was because checking for when to perform bucket splits or merges considered the first 4 scans only. (Bug #48700)

  • During Start Phases 1 and 2, the STATUS command sometimes (falsely) returned Not Connected for data nodes running ndbmtd. (Bug #47818)

  • When performing a DELETE that included a left join from an NDB table, only the first matching row was deleted. (Bug #47054)

  • Under some circumstances, the DBTC kernel block could send an excessive number of commit and completion messages which could lead to a the job buffer filling up and node failure. This was especially likely to occur when using ndbmtd with a single data node. (Bug #45989)

  • mysqld could sometimes crash during a commit while trying to handle NDB Error 4028 Node failure caused abort of transaction. (Bug #38577)

  • When setting LockPagesInMainMemory, the stated memory was not allocated when the node was started, but rather only when the memory was used by the data node process for other reasons. (Bug #37430)

  • Trying to insert more rows than would fit into an NDB table caused data nodes to crash. Now in such situations, the insert fails gracefully with error 633 Table fragment hash index has reached maximum possible size. (Bug #34348)

Changes in MySQL Cluster NDB 7.0.10 (5.1.39-ndb-7.0.10)

Functionality Added or Changed

  • Added the ndb_mgmd --nowait-nodes option, which permits a cluster that is configured to use multiple management servers to be started using fewer than the number configured. This is most likely to be useful when a cluster is configured with two management servers and you wish to start the cluster using only one of them.

    See ndb_mgmd — The MySQL Cluster Management Server Daemon, for more information. (Bug #48669)

  • This enhanced functionality is supported for upgrades from MySQL Cluster NDB 6.3 when the NDB engine version is 6.3.29 or later. (Bug #48528, Bug #49163)

  • The output from ndb_config --configinfo --xml now indicates, for each configuration parameter, the following restart type information:

    • Whether a system restart or a node restart is required when resetting that parameter;

    • Whether cluster nodes need to be restarted using the --initial option when resetting the parameter.

    (Bug #47366)

Bugs Fixed

  • Node takeover during a system restart occurs when the REDO log for one or more data nodes is out of date, so that a node restart is invoked for that node or those nodes. If this happens while a mysqld process is attached to the cluster as an SQL node, the mysqld takes a global schema lock (a row lock), while trying to set up cluster-internal replication.

    However, this setup process could fail, causing the global schema lock to be held for an excessive length of time, which made the node restart hang as well. As a result, the mysqld failed to set up cluster-internal replication, which led to tables being read only, and caused one node to hang during the restart.


    This issue could actually occur in MySQL Cluster NDB 7.0 only, but the fix was also applied MySQL Cluster NDB 6.3, to keep the two codebases in alignment.

    (Bug #49560)

  • Sending SIGHUP to a mysqld running with the --ndbcluster and --log-bin options caused the process to crash instead of refreshing its log files. (Bug #49515)

  • If the master data node receiving a request from a newly started API or data node for a node ID died before the request has been handled, the management server waited (and kept a mutex) until all handling of this node failure was complete before responding to any other connections, instead of responding to other connections as soon as it was informed of the node failure (that is, it waited until it had received a NF_COMPLETEREP signal rather than a NODE_FAILREP signal). On visible effect of this misbehavior was that it caused management client commands such as SHOW and ALL STATUS to respond with unnecessary slowness in such circumstances. (Bug #49207)

  • Attempting to create more than 11435 tables failed with Error 306 (Out of fragment records in DIH). (Bug #49156)

  • When evaluating the options --include-databases, --include-tables, --exclude-databases, and --exclude-tables, the ndb_restore program overwrote the result of the database-level options with the result of the table-level options rather than merging these results together, sometimes leading to unexpected and unpredictable results.

    As part of the fix for this problem, the semantics of these options have been clarified; because of this, the rules governing their evaluation have changed slightly. These changes be summed up as follows:

    • All --include-* and --exclude-* options are now evaluated from right to left in the order in which they are passed to ndb_restore.

    • All --include-* and --exclude-* options are now cumulative.

    • In the event of a conflict, the first (rightmost) option takes precedence.

    For more detailed information and examples, see ndb_restore — Restore a MySQL Cluster Backup. (Bug #48907)

  • When performing tasks that generated large amounts of I/O (such as when using ndb_restore), an internal memory buffer could overflow, causing data nodes to fail with signal 6.

    Subsequent analysis showed that this buffer was not actually required, so this fix removes it. (Bug #48861)

  • Exhaustion of send buffer memory or long signal memory caused data nodes to crash. Now an appropriate error message is provided instead when this situation occurs. (Bug #48852)

  • In some situations, when it was not possible for an SQL node to start a schema transaction (necessary, for instance, as part of an online ALTER TABLE), NDBCLUSTER did not correctly indicate the error to the MySQL server, which led mysqld to crash. (Bug #48841)

  • Under certain conditions, accounting of the number of free scan records in the local query handler could be incorrect, so that during node recovery or a local checkpoint operations, the LQH could find itself lacking a scan record that is expected to find, causing the node to crash. (Bug #48697)

    References: See also: Bug #48564.

  • The creation of an ordered index on a table undergoing DDL operations could cause a data node crash under certain timing-dependent conditions. (Bug #48604)

  • During an LCP master takeover, when the newly elected master did not receive a COPY_GCI LCP protocol message but other nodes participating in the local checkpoint had received one, the new master could use an uninitialized variable, which caused it to crash. (Bug #48584)

  • When running many parallel scans, a local checkpoint (which performs a scan internally) could find itself not getting a scan record, which led to a data node crash. Now an extra scan record is reserved for this purpose, and a problem with obtaining the scan record returns an appropriate error (error code 489, Too many active scans). (Bug #48564)

  • During a node restart, logging was enabled on a per-fragment basis as the copying of each fragment was completed but local checkpoints were not enabled until all fragments were copied, making it possible to run out of redo log file space (NDB error code 410) before the restart was complete. Now logging is enabled only after all fragments has been copied, just prior to enabling local checkpoints. (Bug #48474)

  • When using very large transactions containing many inserts, ndbmtd could fail with Signal 11 without an easily detectable reason, due to an internal variable being unitialized in the event that the LongMessageBuffer was overloaded. Now, the variable is initialized in such cases, avoiding the crash, and an appropriate error message is generated. (Bug #48441)

    References: See also: Bug #46914.

  • A data node crashing while restarting, followed by a system restart could lead to incorrect handling of redo log metadata, causing the system restart to fail with Error while reading REDO log. (Bug #48436)

  • Starting a mysqld process with --ndb-nodeid (either as a command-line option or by assigning it a value in my.cnf) caused the mysqld to get only the corresponding connection from the [mysqld] section in the config.ini file having the matching ID, even when connection pooling was enabled (that is, when the mysqld process was started with --ndb-cluster-connection-pool set greater than 1). (Bug #48405)

    References: See also: Bug #27644, Bug #38590, Bug #41592.

  • The configuration check that each management server runs to verify that all connected ndb_mgmd processes have the same configuration could fail when a configuration change took place while this check was in progress. Now in such cases, the configuration check is rescheduled for a later time, after the change is complete. (Bug #48143)

  • When employing NDB native backup to back up and restore an empty NDB table that used a non-sequential AUTO_INCREMENT value, the AUTO_INCREMENT value was not restored correctly. (Bug #48005)

  • ndb_config --xml --configinfo now indicates that parameters belonging in the [SCI], [SCI DEFAULT], [SHM], and [SHM DEFAULT] sections of the config.ini file are deprecated or experimental, as appropriate. (Bug #47365)

  • NDB stores blob column data in a separate, hidden table that is not accessible from MySQL. If this table was missing for some reason (such as accidental deletion of the file corresponding to the hidden table) when making a MySQL Cluster native backup, ndb_restore crashed when attempting to restore the backup. Now in such cases, ndb_restore fails with the error message Table table_name has blob column (column_name) with missing parts table in backup instead. (Bug #47289)

  • In MySQL Cluster NDB 7.0, ndb_config and ndb_error_reporter were printing warnings about management and data nodes running on the same host to stdout instead of stderr, as was the case in earlier MySQL Cluster release series. (Bug #44689, Bug #49160)

    References: See also: Bug #25941.

  • DROP DATABASE failed when there were stale temporary NDB tables in the database. This situation could occur if mysqld crashed during execution of a DROP TABLE statement after the table definition had been removed from NDBCLUSTER but before the corresponding .ndb file had been removed from the crashed SQL node's data directory. Now, when mysqld executes DROP DATABASE, it checks for these files and removes them if there are no corresponding table definitions for them found in NDBCLUSTER. (Bug #44529)

  • Creating an NDB table with an excessive number of large BIT columns caused the cluster to fail. Now, an attempt to create such a table is rejected with error 791 (Too many total bits in bitfields). (Bug #42046)

    References: See also: Bug #42047.

  • When a long-running transaction lasting long enough to cause Error 410 (REDO log files overloaded) was later committed or rolled back, it could happen that NDBCLUSTER was not able to release the space used for the REDO log, so that the error condition persisted indefinitely.

    The most likely cause of such transactions is a bug in the application using MySQL Cluster. This fix should handle most cases where this might occur. (Bug #36500)

  • Deprecation and usage information obtained from ndb_config --configinfo regarding the PortNumber and ServerPort configuration parameters was improved. (Bug #24584)

  • Disk Data: When running a write-intensive workload with a very large disk page buffer cache, CPU usage approached 100% during a local checkpoint of a cluster containing Disk Data tables. (Bug #49532)

  • Disk Data: NDBCLUSTER failed to provide a valid error message it failed to commit schema transactions during an initial start if the cluster was configured using the InitialLogFileGroup parameter. (Bug #48517)

  • Disk Data: In certain limited cases, it was possible when the cluster contained Disk Data tables for ndbmtd to crash during a system restart. (Bug #48498)

    References: See also: Bug #47832.

  • Disk Data: Repeatedly creating and then dropping Disk Data tables could eventually lead to data node failures. (Bug #45794, Bug #48910)

  • Disk Data: When a crash occurs due to a problem in Disk Data code, the currently active page list is printed to stdout (that is, in one or more ndb_nodeid_out.log files). One of these lists could contain an endless loop; this caused a printout that was effectively never-ending. Now in such cases, a maximum of 512 entries is printed from each list. (Bug #42431)

  • Disk Data: When the FileSystemPathUndoFiles configuration parameter was set to an non-existent path, the data nodes shut down with the generic error code 2341 (Internal program error). Now in such cases, the error reported is error 2815 (File not found).

  • Cluster API: When a DML operation failed due to a uniqueness violation on an NDB table having more than one unique index, it was difficult to determine which constraint caused the failure; it was necessary to obtain an NdbError object, then decode its details property, which in could lead to memory management issues in application code.

    To help solve this problem, a new API method Ndb::getNdbErrorDetail() is added, providing a well-formatted string containing more precise information about the index that caused the unque constraint violation. The following additional changes are also made in the NDB API:

    • Use of NdbError.details is now deprecated in favor of the new method.

    • The Dictionary::listObjects() method has been modified to provide more information.

    (Bug #48851)

  • Cluster API: When using blobs, calling getBlobHandle() requires the full key to have been set using equal(), because getBlobHandle() must access the key for adding blob table operations. However, if getBlobHandle() was called without first setting all parts of the primary key, the application using it crashed. Now, an appropriate error code is returned instead. (Bug #28116, Bug #48973)

Changes in MySQL Cluster NDB 7.0.9b (5.1.39-ndb-7.0.9b)

Bugs Fixed

  • Using a large number of small fragment log files could cause NDBCLUSTER to crash while trying to read them during a restart. This issue was first observed with 1024 fragment log files of 16 MB each. (Bug #48651)

Changes in MySQL Cluster NDB 7.0.9a (5.1.39-ndb-7.0.9a)

Bugs Fixed

  • When the combined length of all names of tables using the NDB storage engine was greater than or equal to 1024 bytes, issuing the START BACKUP command in the ndb_mgm client caused the cluster to crash. (Bug #48531)

Changes in MySQL Cluster NDB 7.0.9 (5.1.39-ndb-7.0.9)

Functionality Added or Changed

  • Performance: Significant improvements in redo log handling and other file system operations can yield a considerable reduction in the time required for restarts. While actual restart times observed in a production setting will naturally vary according to database size, hardware, and other conditions, our own preliminary testing shows that these optimizations can yield startup times that are faster than those typical of previous MySQL Cluster releases by a factor of 50 or more.

Bugs Fixed

  • Important Change: The --with-ndb-port-base option for configure did not function correctly, and has been deprecated. Attempting to use this option produces the warning Ignoring deprecated option --with-ndb-port-base.

    Beginning with MySQL Cluster NDB 7.1.0, the deprecation warning itself is removed, and the --with-ndb-port-base option is simply handled as an unknown and invalid option if you try to use it. (Bug #47941)

    References: See also: Bug #38502.

  • After upgrading a MySQL Cluster containing tables having unique indexes from an NDB 6.3 release to an NDB 7.0 release, attempts to create new unique indexes failed with inconsistent trigger errors (error code 293).

    For more information (including a workaround for previous MySQL Cluster NDB 7.0 releases), see Upgrade and downgrade compatibility: MySQL Cluster NDB 7.x. (Bug #48416)

  • When a data node failed to start due to inability to recreate or drop objects during schema restoration (for example: insufficient memory was available to the data node process on account of issues not directly related to MySQL Cluster on the host machine), the reason for the failure was not provided. Now is such cases, a more informative error message is logged. (Bug #48232)

  • A table that was created following an upgrade from a MySQL Cluster NDB 6.3 release to MySQL Cluster NDB 7.0 (starting with version 6.4.0) or later was dropped by a system restart. This was due to a change in the format of NDB schema files and the fact that the upgrade of the format of existing NDB 6.3 schema files to the NDB 7.0 format failed to change the version number contained in the file; this meant that a system restart re-ran the upgrade routine, which interpreted the newly created table as an uncommitted table (which by definition ought not to be saved). Now the version number of upgraded NDB 6.3 schema files is set correctly during the upgrade process. (Bug #48227)

  • In certain cases, performing very large inserts on NDB tables when using ndbmtd caused the memory allocations for ordered or unique indexes (or both) to be exceeded. This could cause aborted transactions and possibly lead to data node failures. (Bug #48037)

    References: See also: Bug #48113.

  • For UPDATE IGNORE statements, batching of updates is now disabled. This is because such statements failed when batching of updates was employed if any updates violated a unique constraint, to the fact a unique constraint violation could not be handled without aborting the transaction. (Bug #48036)

  • Starting a data node with a very large amount of DataMemory (approximately 90G or more) could lead to crash of the node due to job buffer congestion. (Bug #47984)

  • In some cases, ndbmtd could allocate more space for the undo buffer than was actually available, leading to a failure in the LGMAN kernel block and subsequent failure of the data node. (Bug #47966)

  • When an UPDATE statement was issued against an NDB table where an index was used to identify rows but no data was actually changed, the NDB storage returned zero found rows.

    For example, consider the table created and populated using these statements:

        c1 INT NOT NULL,
        c2 INT NOT NULL,
        PRIMARY KEY(c1),
    INSERT INTO t1 VALUES(1, 1);

    The following UPDATE statements, even though they did not change any rows, each still matched a row, but this was reported incorrectly in both cases, as shown here:

    mysql> UPDATE t1 SET c2 = 1 WHERE c1 = 1;
    Query OK, 0 rows affected (0.00 sec)
    Rows matched: 0  Changed: 0  Warnings: 0
    mysql> UPDATE t1 SET c1 = 1 WHERE c2 = 1;
    Query OK, 0 rows affected (0.00 sec)
    Rows matched: 0  Changed: 0  Warnings: 0

    Now in such cases, the number of rows matched is correct. (In the case of each of the example UPDATE statements just shown, this is displayed as Rows matched: 1, as it should be.)

    This issue could affect UPDATE statements involving any indexed columns in NDB tables, regardless of the type of index (including KEY, UNIQUE KEY, and PRIMARY KEY) or the number of columns covered by the index. (Bug #47955)

  • On Solaris, shutting down a management node failed when issuing the command to do so from a client connected to a different management node. (Bug #47948)

  • After changing the value of DiskSyncSize to 4294967039 (the maximum) in the config.ini file and reloading the cluster configuration, the new value was displayed in the update information written into the cluster log as a signed number instead of unsigned. (Bug #47944)

    References: See also: Bug #47932.

  • On Solaris 10 for SPARC, ndb_mgmd failed to parse the config.ini file when the DiskSyncSize configuration parameter, whose permitted range of values is 32768 to 4294967039, was set equal to 4294967040 (which is also the value of the internal constant MAX_INT_RNIL), nor could DiskSyncSize be set successfully any higher than the minimum value. (Bug #47932)

    References: See also: Bug #47944.

  • Setting FragmentLogFileSize to a value greater than 256 MB led to errors when trying to read the redo log file. (Bug #47908)

  • SHOW CREATE TABLE did not display the AUTO_INCREMENT value for NDB tables having AUTO_INCREMENT columns. (Bug #47865)

  • An optimization in MySQL Cluster NDB 7.0 causes the DBDICT kernel block to copy several tables at a time when synchronizing the data dictionary to a newly started node; previously, this was done one table at a time. However, when NDB tables were sufficiently large and numerous, the internal buffer for storing them could fill up, causing a data node crash.

    In testing, it was found that having 100 NDB tables with 128 columns each was enough to trigger this issue. (Bug #47859)

  • Under some circumstances, when a scan encountered an error early in processing by the DBTC kernel block (see The DBTC Block), a node could crash as a result. Such errors could be caused by applications sending incorrect data, or, more rarely, by a DROP TABLE operation executed in parallel with a scan. (Bug #47831)

  • When starting a node and synchronizing tables, memory pages were allocated even for empty fragments. In certain situations, this could lead to insufficient memory. (Bug #47782)

  • During an upgrade, newer nodes (NDB kernel block DBTUP) could in some cases try to use the long signal format for communication with older nodes (DBUTIL kernel block) that did not understand the newer format, causing older data nodes to fail after restarting. (Bug #47740)

  • A very small race-condition between NODE_FAILREP and LQH_TRANSREQ signals when handling node failure could lead to operations (locks) not being taken over when they should have been, and subsequently becoming stale. This could lead to node restart failures, and applications getting into endless lock-conflicts with operations that were not released until the node was restarted. (Bug #47715)

    References: See also: Bug #41297.

  • In some cases, the MySQL Server tried to use an error status whose value had never been set. The problem in the code that caused this, in, manifested when using debug builds of mysqld in MySQL Cluster replication.

    This fix brings MySQL Cluster's error handling in in line with what is implemented in MySQL 5.4. (Bug #47548)

  • configure failed to honor the --with-zlib-dir option when trying to build MySQL Cluster from source. (Bug #47223)

  • Performing a system restart of the cluster after having performed a table reorganization which added partitions caused the cluster to become inconsistent, possibly leading to a forced shutdown, in either of the following cases:

    1. When a local checkpoint was in progress but had not yet completed, new partitions were not restored; that is, data that was supposed to be moved could be lost instead, leading to an inconsistent cluster. This was due to an issue whereby the DBDIH kernel block did not save the new table definition and instead used the old one (the version having fewer partitions).

    2. When the most recent LCP had completed, ordered indexes and unlogged tables were still not saved (since these did not participate in the LCP). In this case, the cluster crashed during a subsequent system restart, due to the inconsistency between the main table and the ordered index.

    Now, DBDIH is forced in such cases to use the version of the table definition held by the DBDICT kernel block, which was (already) correct and up to date. (Bug #46585)

  • ndbd was not built correctly when compiled using gcc 4.4.0. (The ndbd binary was built, but could not be started.) (Bug #46113)

  • ndb_mgmd failed to close client connections that had timed out. After running for some time, a race condition could develop in the management server, due to ndb_mgmd having exhausted all of its file descriptors in this fashion. (Bug #45497)

    References: See also: Bug #47712.

  • If a node failed while sending a fragmented long signal, the receiving node did not free long signal assembly resources that it had allocated for the fragments of the long signal that had already been received. (Bug #44607)

  • Numeric configuration parameters set in my.cnf were interpreted as signed rather than unsigned values. The effect of this was that values of 2G or more were truncated with the warning [MgmSrvr] Warning: option 'opt_name': signed value opt_value adjusted to 2147483647. Now such parameter values are treated as unsigned, so that this truncation does not take place.

    This issue did not effect parameters set in config.ini. (Bug #44448)

  • When starting a cluster with a great many tables, it was possible for MySQL client connections as well as the slave SQL thread to issue DML statements against MySQL Cluster tables before mysqld had finished connecting to the cluster and making all tables writeable. This resulted in Table ... is read only errors for clients and the Slave SQL thread.

    This issue is fixed by introducing the --ndb-wait-setup option for the MySQL server. This provides a configurable maximum amount of time that mysqld waits for all NDB tables to become writeable, before enabling MySQL clients or the slave SQL thread to connect. (Bug #40679)

    References: See also: Bug #46955.

  • When building MySQL Cluster, it was possible to configure the build using --with-ndb-port without supplying a port number. Now in such cases, configure fails with an error. (Bug #38502)

    References: See also: Bug #47941.

  • When the MySQL server SQL mode included STRICT_TRANS_TABLES, storage engine warnings and error codes specific to NDB were returned when errors occurred, instead of the MySQL server errors and error codes expected by some programming APIs (such as Connector/J) and applications. (Bug #35990)

  • When a copying operation exhausted the available space on a data node while copying large BLOB columns, this could lead to failure of the data node and a Table is full error on the SQL node which was executing the operation. Examples of such operations could include an ALTER TABLE that changed an INT column to a BLOB column, or a bulk insert of BLOB data that failed due to running out of space or to a duplicate key error. (Bug #34583, Bug #48040)

    References: See also: Bug #41674, Bug #45768.

  • Disk Data: A local checkpoint of an empty fragment could cause a crash during a system restart which was based on that LCP. (Bug #47832)

    References: See also: Bug #41915.

  • Disk Data: Multi-threaded data nodes could in some cases attempt to access the same memory structure in parallel, in a non-safe manner. This could result in data node failure when running ndbmtd while using Disk Data tables. (Bug #44195)

    References: See also: Bug #46507.

  • Cluster API: If an NDB API program reads the same column more than once, it is possible exceed the maximum permissible message size, in which case the operation should be aborted due to NDB error 880 Tried to read too much - too many getValue calls, however due to a change introduced in MySQL Cluster NDB 6.3.18, the check for this was not done correctly, which instead caused a data node crash. (Bug #48266)

  • Cluster API: The NDB API methods Dictionary::listEvents(), Dictionary::listIndexes(), Dictionary::listObjects(), and NdbOperation::getErrorLine() formerly had both const and non-const variants. The non-const versions of these methods have been removed. In addition, the NdbOperation::getBlobHandle() method has been re-implemented to provide consistent internal semantics. (Bug #47798)

  • Cluster API: A duplicate read of a column caused NDB API applications to crash. (Bug #45282)

  • Cluster API: The error handling shown in the example file ndbapi_scan.cpp included with the MySQL Cluster distribution was incorrect. (Bug #39573)

Changes in MySQL Cluster NDB 7.0.8a (5.1.37-ndb-7.0.8a)

Bugs Fixed

  • The disconnection of an API or SQL node having a node ID greater than 49 caused a forced shutdown of the cluster. (Bug #47844)

  • The error message text for NDB error code 410 (REDO log files overloaded...) was truncated. (Bug #23662)

Changes in MySQL Cluster NDB 7.0.8 (5.1.37-ndb-7.0.8)

Functionality Added or Changed

  • A new option --log-name is added for ndb_mgmd. This option can be used to provide a name for the current node and then to identify it in messages written to the cluster log. For more information, see ndb_mgmd — The MySQL Cluster Management Server Daemon. (Bug #47643)

  • --config-dir is now accepted by ndb_mgmd as an alias for the --configdir option. (Bug #42013)

  • Disk Data: Two new columns have been added to the output of ndb_desc to make it possible to determine how much of the disk space allocated to a given table or fragment remains free. (This information is not available from the INFORMATION_SCHEMA.FILES table, since the FILES table applies only to Disk Data files.) For more information, see ndb_desc — Describe NDB Tables. (Bug #47131)

Bugs Fixed

  • Important Change: Previously, the MySQL Cluster management node and data node programs, when run on Windows platforms, required the --nodaemon option to produce console output. Now, these programs run in the foreground when invoked from the command line on Windows, which is the same behavior that mysqld.exe displays on Windows. (Bug #45588)

  • The following issues with error logs generated by ndbmtd were addressed:

    1. The version string was sometimes truncated, or even not shown, depending on the number of threads in use (the more threads, the worse the problem). Now the version string is shown in full, as well as the file names for all tracefiles (where available).

    2. In the event of a crash, the thread number of the thread that crashed was not printed. Now this information is supplied, if available.

    (Bug #47629)

  • mysqld allocated an excessively large buffer for handling BLOB values due to overestimating their size. (For each row, enough space was allocated to accommodate every BLOB or TEXT column value in the result set.) This could adversely affect performance when using tables containing BLOB or TEXT columns; in a few extreme cases, this issue could also cause the host system to run out of memory unexpectedly. (Bug #47574)

    References: See also: Bug #47572, Bug #47573.

  • NDBCLUSTER uses a dynamically allocated buffer to store BLOB or TEXT column data that is read from rows in MySQL Cluster tables.

    When an instance of the NDBCLUSTER table handler was recycled (this can happen due to table definition cache pressure or to operations such as FLUSH TABLES or ALTER TABLE), if the last row read contained blobs of zero length, the buffer was not freed, even though the reference to it was lost. This resulted in a memory leak.

    For example, consider the table defined and populated as shown here:

    INSERT INTO t VALUES (1, REPEAT('F', 20000));
    INSERT INTO t VALUES (2, '');

    Now execute repeatedly a SELECT on this table, such that the zero-length LONGTEXT row is last, followed by a FLUSH TABLES statement (which forces the handler object to be re-used), as shown here:

    SELECT a, length(b) FROM bl ORDER BY a;

    Prior to the fix, this resulted in a memory leak proportional to the size of the stored LONGTEXT value each time these two statements were executed. (Bug #47573)

    References: See also: Bug #47572, Bug #47574.

  • Large transactions involving joins between tables containing BLOB columns used excessive memory. (Bug #47572)

    References: See also: Bug #47573, Bug #47574.

  • After an NDB table had an ALTER ONLINE TABLE operation performed on it in a MySQL Cluster running a MySQL Cluster NDB 6.3.x release, it could not be upgraded online to a MySQL Cluster NDB 7.0.x release. This issue was detected using MySQL Cluster NDB 6.3.20, but is likely to effect any MySQL Cluster NDB 6.3.x release supporting online DDL operations. (Bug #47542)

  • When using multi-threaded data nodes (ndbmtd) with NoOfReplicas set to a value greater than 2, attempting to restart any of the data nodes caused a forced shutdown of the entire cluster. (Bug #47530)

  • A variable was left uninitialized while a data node copied data from its peers as part of its startup routine; if the starting node died during this phase, this could lead a crash of the cluster when the node was later restarted. (Bug #47505)

  • Handling of LQH_TRANS_REQ signals was done incorrectly in DBLQH when the transaction coordinator failed during a LQH_TRANS_REQ session. This led to incorrect handling of multiple node failures, particularly when using ndbmtd. (Bug #47476)

  • The NDB kernel's parser (in ndb/src/common/util/Parser.cpp) did not interpret the backslash (\) character correctly. (Bug #47426)

  • During an online alter table operation, the new table definition was made available for users during the prepare-phase when it should only be exposed during and after a commit. This issue could affect NDB API applications, mysqld processes, or data node processes. (Bug #47375)

  • Aborting an online add column operation (for example, due to resource problems on a single data node, but not others) could lead to a forced node shutdown. (Bug #47364)

  • Clients attempting to connect to the cluster during shutdown could sometimes cause the management server to crash. (Bug #47325)

  • The size of the table descriptor pool used in the DBTUP kernel block was incorrect. This could lead to a data node crash when an LQH sent a CREATE_TAB_REF signal. (Bug #47215)

    References: See also: Bug #44908.

  • When a data node restarts, it first runs the redo log until reaching the latest restorable global checkpoint; after this it scans the remainder of the redo log file, searching for entries that should be invalidated so they are not used in any subsequent restarts. (It is possible, for example, if restoring GCI number 25, that there might be entries belonging to GCI 26 in the redo log.) However, under certain rare conditions, during the invalidation process, the redo log files themselves were not always closed while scanning ahead in the redo log. In rare cases, this could lead to MaxNoOfOpenFiles being exceeded, causing a the data node to crash. (Bug #47171)

  • For very large values of MaxNoOfTables + MaxNoOfAttributes, the calculation for StringMemory could overflow when creating large numbers of tables, leading to NDB error 773 (Out of string memory, please modify StringMemory config parameter), even when StringMemory was set to 100 (100 percent). (Bug #47170)

  • The default value for the StringMemory configuration parameter, unlike other MySQL Cluster configuration parameters, was not set in ndb/src/mgmsrv/ConfigInfo.cpp. (Bug #47166)

  • Signals from a failed API node could be received after an API_FAILREQ signal (see Operations and Signals) has been received from that node, which could result in invalid states for processing subsequent signals. Now, all pending signals from a failing API node are processed before any API_FAILREQ signal is received. (Bug #47039)

    References: See also: Bug #44607.

  • When reloading the management server configuration, only the last changed parameter was logged. (Bug #47036)

  • When using ndbmtd, a parallel DROP TABLE operation could cause data nodes to have different views of which tables should be included in local checkpoints; this discrepancy could lead to a node failure during an LCP. (Bug #46873)

  • Using triggers on NDB tables caused ndb_autoincrement_prefetch_sz to be treated as having the NDB kernel's internal default value (32) and the value for this variable as set on the cluster's SQL nodes to be ignored. (Bug #46712)

  • Now, when started with --initial --reload, ndb_mgmd tries to connect to and to copy the configuration of an existing ndb_mgmd process with a confirmed configuration. This works only if another management server is found, and the configuration files used by both management nodes are exactly the same.

    If no other management server is found, the local configuration file is read and used. With this change, it is now necessary when performing a rolling restart of a MySQL Cluster having multiple management nodes, to stop all ndb_mgmd processes, and when restarting them, to start the first of these with the --reload or --initial option (or both options), and then to start any remaining management nodes without using either of these two options. For more information, see Performing a Rolling Restart of a MySQL Cluster. (Bug #45495, Bug #46488, Bug #11753966, Bug #11754823)

    References: See also: Bug #42015, Bug #11751233.

  • On Windows, ndbd --initial could hang in an endless loop while attempting to remove directories. (Bug #45402)

  • For multi-threaded data nodes, insufficient fragment records were allocated in the DBDIH NDB kernel block, which could lead to error 306 when creating many tables; the number of fragment records allocated did not take into account the number of LQH instances. (Bug #44908)

  • Running an ALTER TABLE statement while an NDB backup was in progress caused mysqld to crash. (Bug #44695)

  • When performing auto-discovery of tables on individual SQL nodes, NDBCLUSTER attempted to overwrite existing MyISAM .frm files and corrupted them.

    Workaround.  In the mysql client, create a new table (t2) with same definition as the corrupted table (t1). Use your system shell or file manager to rename the old .MYD file to the new file name (for example, mv t1.MYD t2.MYD). In the mysql client, repair the new table, drop the old one, and rename the new table using the old file name (for example, RENAME TABLE t2 TO t1).

    (Bug #42614)

  • When started with the --initial and --reload options, if ndb_mgmd could not find a configuration file or connect to another management server, it appeared to hang. Now, when trying to fetch its configuration from another management node, ndb_mgmd checks and signals (Trying to get configuration from other mgmd(s)) each 30 seconds that it has not yet done so. (Bug #42015)

    References: See also: Bug #45495.

  • Running ndb_restore with the --print or --print_log option could cause it to crash. (Bug #40428, Bug #33040)

  • An insert on an NDB table was not always flushed properly before performing a scan. One way in which this issue could manifest was that LAST_INSERT_ID() sometimes failed to return correct values when using a trigger on an NDB table. (Bug #38034)

  • When a data node received a TAKE_OVERTCCONF signal from the master before that node had received a NODE_FAILREP, a race condition could in theory result. (Bug #37688)

    References: See also: Bug #25364, Bug #28717.

  • Some joins on large NDB tables having TEXT or BLOB columns could cause mysqld processes to leak memory. The joins did not need to reference the TEXT or BLOB columns directly for this issue to occur. (Bug #36701)

  • On OS X 10.5, commands entered in the management client failed and sometimes caused the client to hang, although management client commands invoked using the --execute (or -e) option from the system shell worked normally.

    For example, the following command failed with an error and hung until killed manually, as shown here:

    ndb_mgm> SHOW     
    Warning, event thread startup failed, degraded printouts as result, errno=36

    However, the same management client command, invoked from the system shell as shown here, worked correctly:

    shell> ndb_mgm -e "SHOW"

    (Bug #35751)

    References: See also: Bug #34438.

  • Disk Data: Calculation of free space for Disk Data table fragments was sometimes done incorrectly. This could lead to unnecessary allocation of new extents even when sufficient space was available in existing ones for inserted data. In some cases, this might also lead to crashes when restarting data nodes.


    This miscalculation was not reflected in the contents of the INFORMATION_SCHEMA.FILES table, as it applied to extents allocated to a fragment, and not to a file.

    (Bug #47072)

  • Cluster API: In some circumstances, if an API node encountered a data node failure between the creation of a transaction and the start of a scan using that transaction, then any subsequent calls to startTransaction() and closeTransaction() could cause the same transaction to be started and closed repeatedly. (Bug #47329)

  • Cluster API: Performing multiple operations using the same primary key within the same NdbTransaction::execute() call could lead to a data node crash.


    This fix does not make change the fact that performing multiple operations using the same primary key within the same execute() is not supported; because there is no way to determine the order of such operations, the result of such combined operations remains undefined.

    (Bug #44065)

    References: See also: Bug #44015.

Changes in MySQL Cluster NDB 7.0.7 (5.1.35-ndb-7.0.7)

Functionality Added or Changed

  • Important Change: The default value of the DiskIOThreadPool data node configuration parameter has changed from 8 to 2.

  • On Solaris platforms, the MySQL Cluster management server and NDB API applications now use CLOCK_REALTIME as the default clock. (Bug #46183)

  • Formerly, node IDs were represented in the cluster log using a complex hexadecimal/binary encoding scheme. Now, node IDs are reported in the cluster log using numbers in standard decimal notation. (Bug #44248)

  • A new option --exclude-missing-columns has been added for the ndb_restore program. In the event that any tables in the database or databases being restored to have fewer columns than the same-named tables in the backup, the extra columns in the backup's version of the tables are ignored. For more information, see ndb_restore — Restore a MySQL Cluster Backup. (Bug #43139)

  • Note

    This issue, originally resolved in MySQL 5.1.16, re-occurred due to a later (unrelated) change. The fix has been re-applied.

    (Bug #25984)

  • Previously, it was possible to disable arbitration only by setting ArbitrationRank to 0 on all management and API nodes. A new data node configuration parameter Arbitration simplifies this task; to disable arbitration, you can now use Arbitration = Disabled in the [ndbd default] section of the config.ini file.

    It is now also possible to configure arbitration in such a way that the cluster waits until the time determined by ArbitrationTimeout passes for an external manager to perform arbitration instead of handling it internally. This can be done by setting Arbitration = WaitExternal in the [ndbd default] section of the config.ini file.

    The default value for the Arbitration parameter is Default, which permits arbitration to proceed normally, as determined by the ArbitrationRank settings for the management and API nodes.

    For more information, see Defining MySQL Cluster Data Nodes.

Bugs Fixed

  • Packaging: The pkg installer for MySQL Cluster on Solaris did not perform a complete installation due to an invalid directory reference in the postinstall script. (Bug #41998)

  • The output from ndb_config --configinfo --xml contained quote characters (") within quoted XML attributes, causing it to be not well-formed. (Bug #46891)

  • When using multi-threaded data node processes (ndbmtd), it was possible for LQH threads to continue running even after all NDB tables had been dropped. This meant that dropping the last remaining NDB table during a local checkpoint could cause multi-threaded data nodes to fail. (Bug #46890)

  • During a global checkpoint, LQH threads could run unevenly, causing a circular buffer overflow by the Subscription Manager, which led to data node failure. (Bug #46782)

    References: See also: Bug #46123, Bug #46723, Bug #45612.

  • Restarting the cluster following a local checkpoint and an online ALTER TABLE on a non-empty table caused data nodes to crash. (Bug #46651)

  • A combination of index creation and drop operations (or creating and dropping tables having indexes) with node and system restarts could lead to a crash. (Bug #46552)

  • Following an upgrade from MySQL Cluster NDB 6.3.x to MySQL Cluster NDB 7.0.6, DDL and backup operations failed. (Bug #46494, Bug #46563)

  • Full table scans failed to execute when the cluster contained more than 21 table fragments.


    The number of table fragments in the cluster can be calculated as the number of data nodes, times 8 (that is, times the value of the internal constant MAX_FRAG_PER_NODE), divided by the number of replicas. Thus, when NoOfReplicas = 1 at least 3 data nodes were required to trigger this issue, and when NoOfReplicas = 2 at least 4 data nodes were required to do so.

    (Bug #46490)

  • Killing MySQL Cluster nodes immediately following a local checkpoint could lead to a crash of the cluster when later attempting to perform a system restart.

    The exact sequence of events causing this issue was as follows:

    1. Local checkpoint occurs.

    2. Immediately following the LCP, kill the master data node.

    3. Kill the remaining data nodes within a few seconds of killing the master.

    4. Attempt to restart the cluster.

    (Bug #46412)

  • Creating an index when the cluster had run out of table records could cause data nodes to crash. (Bug #46295)

  • Ending a line in the config.ini file with an extra semicolon character (;) caused reading the file to fail with a parsing error. (Bug #46242)

  • When combining an index scan and a delete with a primary key delete, the index scan and delete failed to initialize a flag properly. This could in rare circumstances cause a data node to crash. (Bug #46069)

  • OPTIMIZE TABLE on an NDB table could in some cases cause SQL and data nodes to crash. This issue was observed with both ndbd and ndbmtd. (Bug #45971)

  • The AutoReconnect configuration parameter for API nodes (including SQL nodes) has been added. This is intended to prevent API nodes from re-using allocated node IDs during cluster restarts. For more information, see Defining SQL and Other API Nodes in a MySQL Cluster.

    This fix also introduces two new methods of the NDB API Ndb_cluster_connection class: set_auto_reconnect() and get_auto_reconnect(). (Bug #45921)

  • DML statements run during an upgrade from MySQL Cluster NDB 6.3 to NDB 7.0 were not handled correctly. (Bug #45917)

  • On Windows, the internal basestring_vsprintf() function did not return a POSIX-compliant value as expected, causing the management server to crash when trying to start a MySQL Cluster with more than 4 data nodes. (Bug #45733)

  • The signals used by ndb_restore to send progress information about backups to the cluster log accessed the cluster transporter without using any locks. Because of this, it was theoretically possible that these signals could be interefered with by heartbeat signals if both were sent at the same time, causing the ndb_restore messages to be corrupted. (Bug #45646)

  • Due to changes in the way that NDBCLUSTER handles schema changes (implementation of schema transactions) in MySQL Cluster NDB 7.0, it was not possible to create MySQL Cluster tables having more than 16 indexes using a single CREATE TABLE statement.

    This issue occurs only in MySQL Cluster NDB 7.0 releases prior to 7.0.7 (including releases numbered NDB 6.4.x).

    If you are not yet able to upgrade from an earlier MySQL Cluster NDB 7.0 release, you can work around this problem by creating the table without any indexes, then adding the indexes using a separate CREATE INDEX statement for each index. (Bug #45525)

  • storage/ndb/src/common/util/CMakeLists.txt did not build the BaseString-t test program for Windows as the equivalent storage/ndb/src/common/util/ does when building MySQL Cluster on Unix platforms. (Bug #45099)

  • Problems could arise when using VARCHAR columns whose size was greater than 341 characters and which used the utf8_unicode_ci collation. In some cases, this combination of conditions could cause certain queries and OPTIMIZE TABLE statements to crash mysqld. (Bug #45053)

  • The warning message Possible bug in Dbdih::execBLOCK_COMMIT_ORD ... could sometimes appear in the cluster log. This warning is obsolete, and has been removed. (Bug #44563)

  • Debugging code causing ndbd to use file compression on NTFS file systems failed with an error. (The code was removed.) This issue affected debug builds of MySQL Cluster on Windows platforms only. (Bug #44418)

  • ALTER TABLE REORGANIZE PARTITION could fail with Error 741 (Unsupported alter table) if the appropriate hash-map was not present. This could occur when adding nodes online; for example, when going from 2 data nodes to 3 data nodes with NoOfReplicas=1, or from 4 data nodes to 6 data nodes with NoOfReplicas=2. (Bug #44301)

  • Previously, a GCP STOP event was written to the cluster log as an INFO event. Now it is logged as a WARNING event instead. (Bug #43853)

  • In some cases, OPTIMIZE TABLE on an NDB table did not free any DataMemory. (Bug #43683)

  • If the cluster crashed during the execution of a CREATE LOGFILE GROUP statement, the cluster could not be restarted afterward. (Bug #36702)

    References: See also: Bug #34102.

  • Partitioning; Disk Data: An NDB table created with a very large value for the MAX_ROWS option could—if this table was dropped and a new table with fewer partitions, but having the same table ID, was created—cause ndbd to crash when performing a system restart. This was because the server attempted to examine each partition whether or not it actually existed. (Bug #45154)

    References: See also: Bug #58638.

  • Disk Data: If the value set in the config.ini file for FileSystemPathDD, FileSystemPathDataFiles, or FileSystemPathUndoFiles was identical to the value set for FileSystemPath, that parameter was ignored when starting the data node with --initial option. As a result, the Disk Data files in the corresponding directory were not removed when performing an initial start of the affected data node or data nodes. (Bug #46243)

Changes in MySQL Cluster NDB 7.0.6 (5.1.34-ndb-7.0.6)

Functionality Added or Changed

  • The ndb_config utility program can now provide an offline dump of all MySQL Cluster configuration parameters including information such as default and permitted values, brief description, and applicable section of the config.ini file. A dump in text format is produced when running ndb_config with the new --configinfo option, and in XML format when the options --configinfo --xml are used together. For more information and examples, see ndb_config — Extract MySQL Cluster Configuration Information.

Bugs Fixed

  • Important Change; Partitioning: User-defined partitioning of an NDBCLUSTER table without any primary key sometimes failed, and could cause mysqld to crash.

    Now, if you wish to create an NDBCLUSTER table with user-defined partitioning, the table must have an explicit primary key, and all columns listed in the partitioning expression must be part of the primary key. The hidden primary key used by the NDBCLUSTER storage engine is not sufficient for this purpose. However, if the list of columns is empty (that is, the table is defined using PARTITION BY [LINEAR] KEY()), then no explicit primary key is required.

    This change does not effect the partitioning of tables using any storage engine other than NDBCLUSTER. (Bug #40709)

  • Important Change: Previously, the configuration parameter NoOfReplicas had no default value. Now the default for NoOfReplicas is 2, which is the recommended value in most settings. (Bug #44746)

  • Important Note: It was not possible to perform an online upgrade from any MySQL Cluster NDB 6.x release to MySQL Cluster NDB 7.0.5 or any to earlier MySQL Cluster NDB 7.0 release.

    With this fix, it is possible in MySQL Cluster NDB 7.0.6 and later to perform online upgrades from MySQL Cluster NDB 6.3.8 and later MySQL Cluster NDB 6.3 releases, or from MySQL Cluster NDB 7.0.5 or later MySQL Cluster NDB 7.0 releases. Online upgrades to MySQL Cluster NDB 7.0 releases previous to MySQL Cluster NDB 7.0.6 from earlier MySQL Cluster releases remain unsupported; online upgrades from MySQL Cluster NDB 7.0 releases previous to MySQL Cluster NDB 7.0.5 (including NDB 6.4.x beta releases) to later MySQL Cluster NDB 7.0 releases also remain unsupported. (Bug #44294)

  • An internal NDB API buffer was not properly initialized. (Bug #44977)

  • When a data node had written its GCI marker to the first page of a megabyte, and that node was later killed during restart after having processed that page (marker) but before completing a LCP, the data node could fail with file system errors. (Bug #44952)

    References: See also: Bug #42564, Bug #44291.

  • When restarting a data nodes, management and API nodes reconnecting to it failed to re-use existing ports that had already been dynamically allocated for communications with that data node. (Bug #44866)

  • When ndb_config could not find the file referenced by the --config-file option, it tried to read my.cnf instead, then failed with a misleading error message. (Bug #44846)

  • When a data node was down so long that its most recent local checkpoint depended on a global checkpoint that was no longer restorable, it was possible for it to be unable to use optimized node recovery when being restarted later. (Bug #44844)

    References: See also: Bug #26913.

  • Online upgrades to MySQL Cluster NDB 7.0 from a MySQL Cluster NDB 6.3 release could fail due to changes in the handling of key lengths and unique indexes during node recovery. (Bug #44827)

  • ndb_config --xml did not output any entries for the HostName parameter. In addition, the default listed for MaxNoOfFiles was outside the permitted range of values. (Bug #44749)

    References: See also: Bug #44685, Bug #44746.

  • The output of ndb_config --xml did not provide information about all sections of the configuration file. (Bug #44685)

    References: See also: Bug #44746, Bug #44749.

  • Use of __builtin_expect() had the side effect that compiler warnings about misuse of = (assignment) instead of == in comparisons were lost when building in debug mode. This is no longer employed when configuring the build with the --with-debug option. (Bug #44570)

    References: See also: Bug #44567.

  • Inspection of the code revealed that several assignment operators (=) were used in place of comparison operators (==) in DbdihMain.cpp. (Bug #44567)

    References: See also: Bug #44570.

  • When using large numbers of configuration parameters, the management server took an excessive amount of time (several minutes or more) to load these from the configuration cache when starting. This problem occurred when there were more than 32 configuration parameters specified, and became progressively worse with each additional multiple of 32 configuration parameters. (Bug #44488)

  • Building the MySQL Cluster NDB 7.0 tree failed when using the icc compiler. (Bug #44310)

  • SSL connections to SQL nodes failed on big-endian platforms. (Bug #44295)

  • Signals providing node state information (NODE_STATE_REP and CHANGE_NODE_STATE_REQ) were not propagated to all blocks of ndbmtd. This could cause the following problems:

    • Inconsistent redo logs when performing a graceful shutdown;

    • Data nodes crashing when later restarting the cluster, data nodes needing to perform node recovery during the system restart, or both.

    (Bug #44291)

    References: See also: Bug #42564.

  • An NDB internal timing function did not work correctly on Windows and could cause mysqld to fail on some AMD processors, or when running inside a virtual machine. (Bug #44276)

  • It was possible for NDB API applications to insert corrupt data into the database, which could subquently lead to data node crashes. Now, stricter checking is enforced on input data for inserts and updates. (Bug #44132)

  • ndb_restore failed when trying to restore data on a big-endian machine from a backup file created on a little-endian machine. (Bug #44069)

  • Repeated starting and stopping of data nodes could cause ndb_mgmd to fail. This issue was observed on Solaris/SPARC. (Bug #43974)

  • A number of incorrectly formatted output strings in the source code caused compiler warnings. (Bug #43878)

  • When trying to use a data node with an older version of the management server, the data node crashed on startup. (Bug #43699)

  • In some cases, data node restarts during a system restart could fail due to insufficient redo log space. (Bug #43156)

  • NDBCLUSTER did not build correctly on Solaris 9 platforms. (Bug #39080)

    References: See also: Bug #39036, Bug #39038.

  • The output of ndbd --help did not provide clear information about the program's --initial and --initial-start options. (Bug #28905)

  • It was theoretically possible for the value of a nonexistent column to be read as NULL, rather than causing an error. (Bug #27843)

  • Disk Data: During a checkpoint, restore points are created for both the on-disk and in-memory parts of a Disk Data table. Under certain rare conditions, the in-memory restore point could include or exclude a row that should have been in the snapshot. This would later lead to a crash during or following recovery.

    This issue was somewhat more likely to be encountered when using ndbmtd. (Bug #41915)

    References: See also: Bug #47832.

  • Disk Data: This fix supersedes and improves on an earlier fix made for this bug in MySQL 5.1.18. (Bug #24521)

Changes in MySQL Cluster NDB 7.0.5 (5.1.32-ndb-7.0.5)

Functionality Added or Changed

  • Introduced the HeartbeatOrder data node configuration parameter, which can be used to set the order in which heartbeats are transmitted between data nodes. This parameter can be useful in situations where multiple data nodes are running on the same host and a temporary disruption in connectivity between hosts would otherwise cause the loss of a node group, leading to failure of the cluster. (Bug #52182)

  • Two new server status variables Ndb_scan_count and Ndb_pruned_scan_count have been introduced. Ndb_scan_count gives the number of scans executed since the cluster was last started. Ndb_pruned_scan_count gives the number of scans for which NDBCLUSTER was able to use partition pruning. Together, these variables can be used to help determine in the MySQL server whether table scans are pruned by NDBCLUSTER. (Bug #44153)

Bugs Fixed

  • Important Note: Due to problem discovered after the code freeze for this release, it is not possible to perform an online upgrade from any MySQL Cluster NDB 6.x release to MySQL Cluster NDB 7.0.5 or any earlier MySQL Cluster NDB 7.0 release.

    This issue is fixed in MySQL Cluster NDB 7.0.6 and later for upgrades from MySQL Cluster NDB 6.3.8 and later MySQL Cluster NDB 6.3 releases, or from MySQL Cluster NDB 7.0.5. (Bug #44294)

  • Cluster Replication: If data node failed during an event creation operation, there was a slight risk that a surviving data node could send an invalid table reference back to NDB, causing the operation to fail with a false Error 723 (No such table). This could take place when a data node failed as a mysqld process was setting up MySQL Cluster Replication. (Bug #43754)

  • Cluster API: The following issues occurred when performing an online (rolling) upgrade of a cluster to a version of MySQL Cluster that supports configuration caching from a version that does not:

    1. When using multiple management servers, after upgrading and restarting one ndb_mgmd, any remaining management servers using the previous version of ndb_mgmd could not synchronize their configuration data.

    2. The MGM API ndb_mgm_get_configuration_nodeid() function failed to obtain configuration data.

    (Bug #43641)

  • During initial node restarts, initialization of the REDO log was always performed 1 node at a time, during start phase 4. Now this is done during start phase 2, so that the initialization can be performed in parallel, thus decreasing the time required for initial restarts involving multiple nodes. (Bug #50062)

  • If the number of fragments per table rises above a certain threshold, the DBDIH kernel block's on-disk table-definition grows large enough to occupy 2 pages. However, in MySQL Cluster NDB 7.0 (including MySQL Cluster NDB 6.4 releases), only 1 page was actually written, causing table definitions stored on disk to be incomplete.

    This issue was not observed in MySQL Cluster release series prior to MySQL Cluster NDB 7.0. (Bug #44135)

  • TransactionDeadlockDetectionTimeout values less than 100 were treated as 100. This could cause scans to time out unexpectedly. (Bug #44099)

  • The file ndberror.c contained a C++-style comment, which caused builds to fail with some C compilers. (Bug #44036)

  • A race condition could occur when a data node failed to restart just before being included in the next global checkpoint. This could cause other data nodes to fail. (Bug #43888)

  • The setting for ndb_use_transactions was ignored. This issue was only known to occur in MySQL Cluster NDB 6.4.3 and MySQL Cluster NDB 7.0.4. (Bug #43236)

  • When a data node process had been killed after allocating a node ID, but before making contact with any other data node processes, it was not possible to restart it due to a node ID allocation failure.

    This issue could effect either ndbd or ndbmtd processes. (Bug #43224)

    References: This issue is a regression of: Bug #42973.

  • ndb_restore crashed when trying to restore a backup made to a MySQL Cluster running on a platform having different endianness from that on which the original backup was taken. (Bug #39540)

  • PID files for the data and management node daemons were not removed following a normal shutdown. (Bug #37225)

  • ndb_restore --print_data did not handle DECIMAL columns correctly. (Bug #37171)

  • Invoking the management client START BACKUP command from the system shell (for example, as ndb_mgm -e "START BACKUP") did not work correctly, unless the backup ID was included when the command was invoked.

    Now, the backup ID is no longer required in such cases, and the backup ID that is automatically generated is printed to stdout, similar to how this is done when invoking START BACKUP within the management client. (Bug #31754)

  • When aborting an operation involving both an insert and a delete, the insert and delete were aborted separately. This was because the transaction coordinator did not know that the operations affected on same row, and, in the case of a committed-read (tuple or index) scan, the abort of the insert was performed first, then the row was examined after the insert was aborted but before the delete was aborted. In some cases, this would leave the row in a inconsistent state. This could occur when a local checkpoint was performed during a backup. This issue did not affect primary ley operations or scans that used locks (these are serialized).

    After this fix, for ordered indexes, all operations that follow the operation to be aborted are now also aborted.

  • Disk Data: When using multi-threaded data nodes, DROP TABLE statements on Disk Data tables could hang. (Bug #43825)

  • Disk Data: This fix completes one that was made for this issue in MySQL Cluster NDB-7.0.4, which did not rectify the problem in all cases. (Bug #43632)

  • Cluster API: If the largest offset of a RecordSpecification used for an NdbRecord object was for the NULL bits (and thus not a column), this offset was not taken into account when calculating the size used for the RecordSpecification. This meant that the space for the NULL bits could be overwritten by key or other information. (Bug #43891)

  • Cluster API: BIT columns created using the native NDB API format that were not created as nullable could still sometimes be overwritten, or cause other columns to be overwritten.

    This issue did not effect tables having BIT columns created using the mysqld format (always used by MySQL Cluster SQL nodes). (Bug #43802)

Changes in MySQL Cluster NDB 7.0.4 (5.1.32-ndb-7.0.4)

Functionality Added or Changed

Bugs Fixed

  • Cluster API: Partition pruning did not work correctly for queries involving multiple range scans.

    As part of the fix for this issue, several improvements have been made in the NDB API, including the addition of a new NdbScanOperation::getPruned() method, a new variant of NdbIndexScanOperation::setBound(), and a new PartitionSpec data structure. (Bug #37934)

  • TimeBetweenLocalCheckpoints was measured from the end of one local checkpoint to the beginning of the next, rather than from the beginning of one LCP to the beginning of the next. This meant that the time spent performing the LCP was not taken into account when determining the TimeBetweenLocalCheckpoints interval, so that LCPs were not started often enough, possibly causing data nodes to run out of redo log space prematurely. (Bug #43567)

  • The management server failed to start correctly in daemon mode. (Bug #43559)

  • Following a DROP NODEGROUP command, the output of SHOW in the ndb_mgm cliently was not updated to reflect the fact that the data nodes affected by this command were no longer part of a node group. (Bug #43413)

  • When using ndbmtd, multiple data node failures caused the remaining data nodes to fail as well. (Bug #43109)

  • It was not possible to add new data nodes to the cluster online using multi-threaded data node processes (ndbmtd). (Bug #43108)

  • Some queries using combinations of logical and comparison operators on an indexed column in the WHERE clause could fail with the error Got error 4541 'IndexBound has no bound information' from NDBCLUSTER. (Bug #42857)

  • Disk Data: When using multi-threaded data nodes, dropping a Disk Data table followed by a data node restart led to a crash. (Bug #43632)

  • Disk Data: When using ndbmtd, repeated high-volume inserts (on the order of 10000 rows inserted at a time) on a Disk Data table would eventually lead to a data node crash. (Bug #41398)

  • Disk Data: When a log file group had an undo log file whose size was too small, restarting data nodes failed with Read underflow errors.

    As a result of this fix, the minimum permitted INTIAL_SIZE for an undo log file is now 1M (1 megabyte). (Bug #29574)

  • Cluster API: The default NdbRecord structures created by NdbDictionary could have overlapping null bits and data fields. (Bug #43590)

  • Cluster API: When performing insert or write operations, NdbRecord permits key columns to be specified in both the key record and in the attribute record. Only one key column value for each key column should be sent to the NDB kernel, but this was not guaranteed. This is now ensured as follows: For insert and write operations, key column values are taken from the key record; for scan takeover update operations, key column values are taken from the attribute record. (Bug #42238)

  • Cluster API: Ordered index scans using NdbRecord formerly expressed a BoundEQ range as separate lower and upper bounds, resulting in 2 copies of the column values being sent to the NDB kernel.

    Now, when a range is specified by NdbIndexScanOperation::setBound(), the passed pointers, key lengths, and inclusive bits are compared, and only one copy of the equal key columns is sent to the kernel. This makes such operations more efficient, as half the amount of KeyInfo is now sent for a BoundEQ range as before. (Bug #38793)

Changes in MySQL Cluster NDB 6.4.3 (5.1.32-ndb-6.4.3)

Functionality Added or Changed

  • A new data node configuration parameter MaxLCPStartDelay has been introduced to facilitate parallel node recovery by causing a local checkpoint to be delayed while recovering nodes are synchronizing data dictionaries and other meta-information. For more information about this parameter, see Defining MySQL Cluster Data Nodes. (Bug #43053)

  • New options are introduced for ndb_restore for determining which tables or databases should be restored:

    • --include-tables and --include-databases can be used to restore specific tables or databases.

    • --exclude-tables and --exclude-databases can be used to exclude the specified tables or databases from being restored.

    For more information about these options, see ndb_restore — Restore a MySQL Cluster Backup. (Bug #40429)

  • Disk Data: It is now possible to specify default locations for Disk Data data files and undo log files, either together or separately, using the data node configuration parameters FileSystemPathDD, FileSystemPathDataFiles, and FileSystemPathUndoFiles. For information about these configuration parameters, see Disk Data file system parameters.

    It is also now possible to specify a log file group, tablespace, or both, that is created when the cluster is started, using the InitialLogFileGroup and InitialTablespace data node configuration parameters. For information about these configuration parameters, see Disk Data object creation parameters.

Bugs Fixed

  • Performance: Updates of the SYSTAB_0 system table to obtain a unique identifier did not use transaction hints for tables having no primary key. In such cases the NDB kernel used a cache size of 1. This meant that each insert into a table not having a primary key required an update of the corresponding SYSTAB_0 entry, creating a potential performance bottleneck.

    With this fix, inserts on NDB tables without primary keys can be under some conditions be performed up to 100% faster than previously. (Bug #39268)

  • Important Note

    It is not possible in this release to install the InnoDB plugin if InnoDB support has been compiled into mysqld. (Bug #42610)

    References: This issue is a regression of: Bug #29263.

  • Packaging: Packages for MySQL Cluster were missing the and libndbclient.a files. (Bug #42278)

  • Partitioning: Executing ALTER TABLE ... REORGANIZE PARTITION on an NDBCLUSTER table having only one partition caused mysqld to crash. (Bug #41945)

    References: See also: Bug #40389.

  • Using indexes containing variable-sized columns could lead to internal errors when the indexes were being built.

    This same issue could also cause redistribution of table data using ALTER ONLINE TABLE statements to fail with tables containing variable-sized columns. (Bug #43226)

  • Backup IDs greater than 231 were not handled correctly, causing negative values to be used in backup directory names and printouts. (Bug #43042)

  • When using ndbmtd, NDB kernel threads could hang while trying to start the data nodes with LockPagesInMainMemory set to 1. (Bug #43021)

  • When using multiple management servers and starting several API nodes (possibly including one or more SQL nodes) whose connection strings listed the management servers in different order, it was possible for 2 API nodes to be assigned the same node ID. When this happened it was possible for an API node not to get fully connected, consequently producing a number of errors whose cause was not easily recognizable. (Bug #42973)

  • When using multi-threaded data nodes, IndexMemory, MaxNoOfLocalOperations, and MaxNoOfLocalScans were effectively multiplied by the number of local query handlers in use by each ndbmtd instance. (Bug #42765)

    References: See also: Bug #42215.

  • ndb_error_reporter worked correctly only with GNU tar. (With other versions of tar, it produced empty archives.) (Bug #42753)

  • Triggers on NDBCLUSTER tables caused such tables to become locked. (Bug #42751)

    References: See also: Bug #16229, Bug #18135.

  • When performing more than 32 index or tuple scans on a single fragment, the scans could be left hanging. This caused unnecessary timeouts, and in addition could possibly lead to a hang of an LCP. (Bug #42559)

    References: This issue is a regression of: Bug #42084.

  • A data node failure that occurred between calls to NdbIndexScanOperation::readTuples(SF_OrderBy) and NdbTransaction::execute() was not correctly handled; a subsequent call to nextResult() caused a null pointer to be deferenced, leading to a segfault in mysqld. (Bug #42545)

  • If the cluster configuration cache file was larger than 32K, the management server would not start. (Bug #42543)

  • Issuing SHOW GLOBAL STATUS LIKE 'NDB%' before mysqld had connected to the cluster caused a segmentation fault. (Bug #42458)

  • When using ndbmtd for all data nodes, repeated failures of one data node during DML operations caused other data nodes to fail. (Bug #42450)

  • Data node failures that occurred before all data nodes had connected to the cluster were not handled correctly, leading to additional data node failures. (Bug #42422)

  • When using multi-threaded data nodes, their DataMemory and IndexMemory usage as reported was multiplied by the number of local query handlers (worker threads), making it appear that much more memory was being used than was actually the case. (Bug #42215)

    References: See also: Bug #42765.

  • Given a MySQL Cluster containing no data (that is, whose data nodes had all been started using --initial, and into which no data had yet been imported) and having an empty backup directory, executing START BACKUP with a user-specified backup ID caused the data nodes to crash. (Bug #41031)

  • In some cases, NDB did not check correctly whether tables had changed before trying to use the query cache. This could result in a crash of the debug MySQL server. (Bug #40464)

  • Disk Data: It was not possible to add an in-memory column online to a table that used a table-level or column-level STORAGE DISK option. The same issue prevented ALTER ONLINE TABLE ... REORGANIZE PARTITION from working on Disk Data tables. (Bug #42549)

  • Disk Data: Repeated insert and delete operations on disk-based tables could lead to failures in the NDB Tablespace Manager (TSMAN kernel block). (Bug #40344)

  • Disk Data: Creating a Disk Data tablespace with a very large extent size caused the data nodes to fail. The issue was observed when using extent sizes of 100 MB and larger. (Bug #39096)

  • Disk Data: Trying to execute a CREATE LOGFILE GROUP statement using a value greater than 150M for UNDO_BUFFER_SIZE caused data nodes to crash.

    As a result of this fix, the upper limit for UNDO_BUFFER_SIZE is now 600M; attempting to set a higher value now fails gracefully with an error. (Bug #34102)

    References: See also: Bug #36702.

  • Disk Data: When attempting to create a tablespace that already existed, the error message returned was Table or index with given name already exists. (Bug #32662)

  • Disk Data: Using a path or file name longer than 128 characters for Disk Data undo log files and tablespace data files caused a number of issues, including failures of CREATE LOGFILE GROUP, ALTER LOGFILE GROUP, CREATE TABLESPACE, and ALTER TABLESPACE statements, as well as crashes of management nodes and data nodes.

    With this fix, the maximum length for path and file names used for Disk Data undo log files and tablespace data files is now the same as the maximum for the operating system. (Bug #31769, Bug #31770, Bug #31772)

  • Disk Data: Attempting to perform a system restart of the cluster where there existed a logfile group without and undo log files caused the data nodes to crash.


    While issuing a CREATE LOGFILE GROUP statement without an ADD UNDOFILE option fails with an error in the MySQL server, this situation could arise if an SQL node failed during the execution of a valid CREATE LOGFILE GROUP statement; it is also possible to create a logfile group without any undo log files using the NDB API.

    (Bug #17614)

  • Cluster API: Some error messages from ndb_mgmd contained newline (\n) characters. This could break the MGM API protocol, which uses the newline as a line separator. (Bug #43104)

  • Cluster API: When using an ordered index scan without putting all key columns in the read mask, this invalid use of the NDB API went undetected, which resulted in the use of uninitialized memory. (Bug #42591)

Changes in MySQL Cluster NDB 6.4.2 (5.1.31-ndb-6.4.2)

Bugs Fixed

  • Connections using IPv6 were not handled correctly by mysqld. (Bug #42413)

    References: See also: Bug #42412, Bug #38247.

  • When a cluster backup failed with Error 1304 (Node node_id1: Backup request from node_id2 failed to start), no clear reason for the failure was provided.

    As part of this fix, MySQL Cluster now retries backups in the event of sequence errors. (Bug #42354)

    References: See also: Bug #22698.

  • Issuing SHOW ENGINE NDBCLUSTER STATUS on an SQL node before the management server had connected to the cluster caused mysqld to crash. (Bug #42264)

  • When using ndbmtd, setting MaxNoOfExecutionThreads to a value higher than the actual number of cores available and with insufficient SharedGlobalMemory caused the data nodes to crash.

    The fix for this issue changes the behavior of ndbmtd such that its internal job buffers no longer rely on SharedGlobalMemory. (Bug #42254)

Changes in MySQL Cluster NDB 6.4.1 (5.1.31-ndb-6.4.1)

Functionality Added or Changed

  • Important Change: Formerly, when the management server failed to create a transporter for a data node connection, net_write_timeout seconds elapsed before the data node was actually permitted to disconnect. Now in such cases the disconnection occurs immediately. (Bug #41965)

    References: See also: Bug #41713.

  • Formerly, when using MySQL Cluster Replication, records for empty epochs—that is, epochs in which no changes to NDBCLUSTER data or tables took place—were inserted into the ndb_apply_status and ndb_binlog_index tables on the slave even when --log-slave-updates was disabled. Beginning with MySQL Cluster NDB 6.2.16 and MySQL Cluster NDB 6.3.13 this was changed so that these empty epochs were no longer logged. However, it is now possible to re-enable the older behavior (and cause empty epochs to be logged) by using the --ndb-log-empty-epochs option. For more information, see Replication Slave Options and Variables.

    References: See also: Bug #37472.

Bugs Fixed

  • A maximum of 11 TUP scans were permitted in parallel. (Bug #42084)

  • The management server could hang after attempting to halt it with the STOP command in the management client. (Bug #42056)

    References: See also: Bug #40922.

  • When using ndbmtd, one thread could flood another thread, which would cause the system to stop with a job buffer full condition (currently implemented as an abort). This could be caused by committing or aborting a large transaction (50000 rows or more) on a single data node running ndbmtd. To prevent this from happening, the number of signals that can be accepted by the system threads is calculated before executing them, and only executing them if sufficient space is found. (Bug #42052)

  • MySQL Cluster would not compile when using libwrap. This issue was known to occur only in MySQL Cluster NDB 6.4.0. (Bug #41918)

  • Trying to execute an ALTER ONLINE TABLE ... ADD COLUMN statement while inserting rows into the table caused mysqld to crash. (Bug #41905)

  • When a data node connects to the management server, the node sends its node ID and transporter type; the management server then verifies that there is a transporter set up for that node and that it is in the correct state, and then sends back an acknowledgment to the connecting node. If the transporter was not in the correct state, no reply was sent back to the connecting node, which would then hang until a read timeout occurred (60 seconds). Now, if the transporter is not in the correct state, the management server acknowledges this promptly, and the node immediately disconnects. (Bug #41713)

    References: See also: Bug #41965.

  • Issuing EXIT in the management client sometimes caused the client to hang. (Bug #40922)

  • In the event that a MySQL Cluster backup failed due to file permissions issues, conflicting reports were issued in the management client. (Bug #34526)

  • If all data nodes were shut down, MySQL clients were unable to access NDBCLUSTER tables and data even after the data nodes were restarted, unless the MySQL clients themselves were restarted. (Bug #33626)

Changes in MySQL Cluster NDB 6.4.0 (5.1.30-ndb-6.4.0)

Functionality Added or Changed

  • Important Change: MySQL Cluster now caches its configuration data. This means that, by default, the management server only reads the global configuration file (usually named config.ini) the first time that it is started, and does not automatically re-read the this file when restarted. This behavior can be controlled using new management server options (--config-dir, --initial, and --reload) that have been added for this purpose. For more information, see MySQL Cluster Configuration Files, and ndb_mgmd — The MySQL Cluster Management Server Daemon.

  • Important Change: Send buffer memory is now allocated dynamically from a shared memory pool, which means that the size of the send buffer can be adjusted as necessary.

    The data node configuration parameters TotalSendBufferMemory, ReservedSendBufferMemory, and the TCP configuration paramater OverLoadLimit have been added to configure this dynamic allocation. In addition, the behavior and use of the SendBufferMemory TCP configuration parameter has changed. See Configuring MySQL Cluster Send Buffer Parameters, for more information.

  • It is now possible while in Single User Mode to restart all data nodes using ALL RESTART in the management client. Restarting of individual nodes while in Single User Mode remains not permitted. (Bug #31056)

  • It is now possible to add data nodes to a MySQL Cluster online—that is, to a running MySQL Cluster without shutting it down.

    For information about the procedure for adding data nodes online, see Adding MySQL Cluster Data Nodes Online.

  • A multi-threaded version of the MySQL Cluster data node daemon is now available. The multi-threaded ndbmtd binary is similar to ndbd and functions in much the same way, but is intended for use on machines with multiple CPU cores.

    For more information, see ndbmtd — The MySQL Cluster Data Node Daemon (Multi-Threaded).

  • It is now possible when performing a cluster backup to determine whether the backup matches the state of the data when the backup began or when it ended, using the new START BACKUP options SNAPSHOTSTART and SNAPSHOTEND in the management client. See Using The MySQL Cluster Management Client to Create a Backup, for more information.

  • Cluster API: Two new Ndb_cluster_connection methods have been added to help in diagnosing problems with NDB API client connections. The get_latest_error() method tells whether or not the latest connection attempt succeeded; if the attempt failed, get_latest_error_msg() provides an error message giving the reason.

Bugs Fixed

  • API nodes disconnected too agressively from cluster when data nodes were being restarted. This could sometimes lead to the API node being unable to access the cluster at all during a rolling restart. (Bug #41462)

  • When long signal buffer exhaustion in the ndbd process resulted in a signal being dropped, the usual handling mechanism did not take fragmented signals into account. This could result in a crash of the data node because the fragmented signal handling mechanism was not able to work with the missing fragments. (Bug #39235)

  • The failure of a master node during a DDL operation caused the cluster to be unavailable for further DDL operations until it was restarted; failures of nonmaster nodes during DLL operations caused the cluster to become completely inaccessible. (Bug #36718)

  • Status messages shown in the management client when restarting a management node were inappropriate and misleading. Now, when restarting a management node, the messages displayed are as follows, where node_id is the management node's node ID:

    ndb_mgm> node_id RESTART
    Shutting down MGM node node_id for restart
    Node node_id is being restarted

    (Bug #29275)

  • A data node failure when NoOfReplicas was greater than 2 caused all cluster SQL nodes to crash. (Bug #18621)