Changes in MySQL NDB Cluster 7.3.3 (5.6.14-ndb-7.3.3) (2013-11-18, General Availability)

• Functionality Added or Changed

• Bugs Fixed

Functionality Added or Changed

• The length of time a management node waits for a heartbeat message from another management node is now configurable using the HeartbeatIntervalMgmdMgmd management node configuration parameter added in this release. The connection is considered dead after 3 missed heartbeats. The default value is 1500 milliseconds, or a timeout of approximately 6000 ms. (Bug #17807768, Bug #16426805)

• The MySQL NDB Cluster Auto-Installer now generates a my.cnf file for each mysqld in the cluster before starting it. For more information, see Using the NDB Cluster Auto-Installer. (Bug #16994782)

• BLOB and TEXT columns are now reorganized by the ALTER ONLINE TABLE ... REORGANIZE PARTITION statement. (Bug #13714148)

Bugs Fixed

• Performance: In a number of cases found in various locations in the MySQL NDB Cluster codebase, unnecessary iterations were performed; this was caused by failing to break out of a repeating control structure after a test condition had been met. This community-contributed fix removes the unneeded repetitions by supplying the missing breaks. (Bug #16904243, Bug #69392, Bug #16904338, Bug #69394, Bug #16778417, Bug #69171, Bug #16778494, Bug #69172, Bug #16798410, Bug #69207, Bug #16801489, Bug #69215, Bug #16904266, Bug #69393)

• Packaging: Portions of the documentation specific to MySQL NDB Cluster and the NDB storage engine were not included when installing from RPMs. (Bug #16303451)

• Microsoft Windows: The Windows error ERROR_FILE_EXISTS was not recognized by NDB, which treated it as an unknown error. (Bug #16970960)

• NDB Disk Data: NDB error 899 RowId already allocated was raised due to a RowId “leak” which occurred under either of the following sets of circumstances:

  1. Insertion of a row into an in-memory table was rejected after an ordered index update failed due to insufficient DataMemory.

  2. Insertion of a row into a Disk Data table was rejected due to lack of sufficient table space.

  (Bug #13927679)

  References: See also: Bug #22494024, Bug #13990924.

• NDB Cluster APIs: The Event::setTable() method now supports a pointer or a reference to table as its required argument. If a null table pointer is used, the method now returns -1 to make it clear that this is what has occurred. (Bug #16329082)

• Trying to restore to a table having a BLOB column in a different position from that of the original one caused ndb_restore --restore-data to fail. (Bug #17395298)

• ndb_restore could abort during the last stages of a restore using attribute promotion or demotion into an existing table. This could happen if a converted attribute was nullable and the backup had been run on active database. (Bug #17275798)

• It was not possible to start MySQL NDB Cluster processes created by the Auto-Installer on a Windows host running freeSSHd. (Bug #17269626)

• The DBUTIL data node block is now less strict about the order in which it receives certain messages from other nodes. (Bug #17052422)

• ALTER ONLINE TABLE ... REORGANIZE PARTITION failed when run against a table having or using a reference to a foreign key. (Bug #17036744, Bug #69619)

• TUPKEYREQ signals are used to read data from the tuple manager block (DBTUP), and are used for all types of data access, especially for scans which read many rows. A TUPKEYREQ specifies a series of 'columns' to be read, which can be either single columns in a specific table, or pseudocolumns, two of which—READ_ALL and READ_PACKED—are aliases to read all columns in a table, or some subset of these columns. Pseudocolumns are used by modern NDB API applications as they require less space in the TUPKEYREQ to specify columns to be read, and can return the data in a more compact (packed) format.

  This fix moves the creation and initialization of on-stack Signal objects to only those pseudocolumn reads which need to EXECUTE_DIRECT to other block instances, rather than for every read. In addition, the size of an on-stack signal is now varied to suit the requirements of each pseudocolumn, so that only reads of the INDEX_STAT pseudocolumn now require initialization (and 3KB memory each time this is performed). (Bug #17009502)

• A race condition could sometimes occur when trying to lock receive threads to cores. (Bug #17009393)

• Results from joins using a WHERE with an ORDER BY ... DESC clause were not sorted properly; the DESC keyword in such cases was effectively ignored. (Bug #16999886, Bug #69528)

• RealTimeScheduler did not work correctly with data nodes running ndbmtd. (Bug #16961971)

• File system errors occurring during a local checkpoint could sometimes cause an LCP to hang with no obvious cause when they were not handled correctly. Now in such cases, such errors always cause the node to fail. Note that the LQH block always shuts down the node when a local checkpoint fails; the change here is to make likely node failure occur more quickly and to make the original file system error more visible. (Bug #16961443)

• Maintenance and checking of parent batch completion in the SPJ block of the NDB kernel was reimplemented. Among other improvements, the completion state of all ancestor nodes in the tree are now preserved. (Bug #16925513)

• Dropping a column, which was not itself a foreign key, from an NDB table having foreign keys failed with ER_TABLE_DEF_CHANGED. (Bug #16912989)

• The LCP fragment scan watchdog periodically checks for lack of progress in a fragment scan performed as part of a local checkpoint, and shuts down the node if there is no progress after a given amount of time has elapsed. This interval, formerly hard-coded as 60 seconds, can now be configured using the LcpScanProgressTimeout data node configuration parameter added in this release.

  This configuration parameter sets the maximum time the local checkpoint can be stalled before the LCP fragment scan watchdog shuts down the node. The default is 60 seconds, which provides backward compatibility with previous releases.

  You can disable the LCP fragment scan watchdog by setting this parameter to 0. (Bug #16630410)

• Added the ndb_error_reporter options --connection-timeout, which makes it possible to set a timeout for connecting to nodes, --dry-scp, which disables scp connections to remote hosts, and --skip-nodegroup, which skips all nodes in a given node group. (Bug #16602002)

  References: See also: Bug #11752792, Bug #44082.

• After issuing START BACKUP id WAIT STARTED, if id had already been used for a backup ID, an error caused by the duplicate ID occurred as expected, but following this, the START BACKUP command never completed. (Bug #16593604, Bug #68854)

• ndb_mgm treated backup IDs provided to ABORT BACKUP commands as signed values, so that backup IDs greater than 2³¹ wrapped around to negative values. This issue also affected out-of-range backup IDs, which wrapped around to negative values instead of causing errors as expected in such cases. The backup ID is now treated as an unsigned value, and ndb_mgm now performs proper range checking for backup ID values greater than MAX_BACKUPS (2³²). (Bug #16585497, Bug #68798)

• When trying to specify a backup ID greater than the maximum allowed, the value was silently truncated. (Bug #16585455, Bug #68796)

• The unexpected shutdown of another data node as a starting data node received its node ID caused the latter to hang in Start Phase 1. (Bug #16007980)

  References: See also: Bug #18993037.

• SELECT ... WHERE ... LIKE from an NDB table could return incorrect results when using engine_condition_pushdown=ON. (Bug #15923467, Bug #67724)

• The NDB receive thread waited unnecessarily for additional job buffers to become available when receiving data. This caused the receive mutex to be held during this wait, which could result in a busy wait when the receive thread was running with real-time priority.

  This fix also handles the case where a negative return value from the initial check of the job buffer by the receive thread prevented further execution of data reception, which could possibly lead to communication blockage or configured ReceiveBufferMemory underutilization. (Bug #15907515)

• When the available job buffers for a given thread fell below the critical threshold, the internal multithreading job scheduler waited for job buffers for incoming rather than outgoing signals to become available, which meant that the scheduler waited the maximum timeout (1 millisecond) before resuming execution. (Bug #15907122)

• Setting lower_case_table_names to 1 or 2 on Windows systems caused ALTER TABLE ... ADD FOREIGN KEY statements against tables with names containing uppercase letters to fail with Error 155, No such table: '(null)'. (Bug #14826778, Bug #67354)

• Under some circumstances, a race occurred where the wrong watchdog state could be reported. A new state name Packing Send Buffers is added for watchdog state number 11, previously reported as Unknown place. As part of this fix, the state numbers for states without names are always now reported in such cases. (Bug #14824490)

• When a node fails, the Distribution Handler (DBDIH kernel block) takes steps together with the Transaction Coordinator (DBTC) to make sure that all ongoing transactions involving the failed node are taken over by a surviving node and either committed or aborted. Transactions taken over which are then committed belong in the epoch that is current at the time the node failure occurs, so the surviving nodes must keep this epoch available until the transaction takeover is complete. This is needed to maintain ordering between epochs.

  A problem was encountered in the mechanism intended to keep the current epoch open which led to a race condition between this mechanism and that normally used to declare the end of an epoch. This could cause the current epoch to be closed prematurely, leading to failure of one or more surviving data nodes. (Bug #14623333, Bug #16990394)

• Exhaustion of LongMessageBuffer memory under heavy load could cause data nodes running ndbmtd to fail. (Bug #14488185)

• When using dynamic listening ports for accepting connections from API nodes, the port numbers were reported to the management server serially. This required a round trip for each API node, causing the time required for data nodes to connect to the management server to grow linearly with the number of API nodes. To correct this problem, each data node now reports all dynamic ports at once. (Bug #12593774)

• ndb_error-reporter did not support the --help option. (Bug #11756666, Bug #48606)

  References: See also: Bug #11752792, Bug #44082.

• Formerly, the node used as the coordinator or leader for distributed decision making between nodes (also known as the DICT manager—see The DBDICT Block) was indicated in the output of the ndb_mgm client SHOW command as the “master” node, although this node has no relationship to a master server in MySQL Replication. (It should also be noted that it is not necessary to know which node is the leader except when debugging NDBCLUSTER source code.) To avoid possible confusion, this label has been removed, and the leader node is now indicated in SHOW command output using an asterisk (*) character. (Bug #11746263, Bug #24880)

• The matrix of values used for thread configuration when applying the setting of the MaxNoOfExecutionThreads configuration parameter has been improved to align with support for greater numbers of LDM threads. See Multi-Threading Configuration Parameters (ndbmtd), for more information about the changes. (Bug #75220, Bug #20215689)

• Program execution failed to break out of a loop after meeting a desired condition in a number of internal methods, performing unneeded work in all cases where this occurred. (Bug #69610, Bug #69611, Bug #69736, Bug #17030606, Bug #17030614, Bug #17160263)

• ABORT BACKUP in the ndb_mgm client (see Commands in the NDB Cluster Management Client) took an excessive amount of time to return (approximately as long as the backup would have required to complete, had it not been aborted), and failed to remove the files that had been generated by the aborted backup. (Bug #68853, Bug #17719439)

• Attribute promotion and demotion when restoring data to NDB tables using ndb_restore --restore-data with the --promote-attributes and --lossy-conversions options has been improved as follows:

  • Columns of types CHAR, and VARCHAR can now be promoted to BINARY and VARBINARY, and columns of the latter two types can be demoted to one of the first two.

    Note that converted character data is not checked to conform to any character set.

  • Any of the types CHAR, VARCHAR, BINARY, and VARBINARY can now be promoted to TEXT or BLOB.

    When performing such promotions, the only other sort of type conversion that can be performed at the same time is between character types and binary types.