This section contains unified changelog information for the MySQL NDB Cluster 7.5 release series.
For changelogs covering individual MySQL NDB Cluster 7.5 releases, see NDB Cluster Release Notes.
For general information about features added in MySQL NDB Cluster 7.5, see What is New in NDB Cluster 7.5.
For an overview of features added in MySQL 5.7 that are not specific to NDB Cluster, see What Is New in MySQL 5.7. For a complete list of all bug fixes and feature changes made in MySQL 5.7 that are not specific to NDB Cluster, see the MySQL 5.7 Release Notes.
- Changes in MySQL NDB Cluster 7.5.8 (5.7.20-ndb-7.5.8)
- Changes in MySQL NDB Cluster 7.5.7 (5.7.19-ndb-7.5.7)
- Changes in MySQL NDB Cluster 7.5.6 (5.7.18-ndb-7.5.6)
- Changes in MySQL NDB Cluster 7.5.5 (5.7.17-ndb-7.5.5)
- Changes in MySQL NDB Cluster 7.5.4 (5.7.16-ndb-7.5.4)
- Changes in MySQL NDB Cluster 7.5.3 (5.7.13-ndb-7.5.3)
- Changes in MySQL NDB Cluster 7.5.2 (5.7.12-ndb-7.5.2)
- Changes in MySQL NDB Cluster 7.5.1 (5.7.11-ndb-7.5.1)
- Changes in MySQL NDB Cluster 7.5.0 (5.7.10-ndb-7.5.0)
Errors in parsing
NDB_TABLEmodifiers could cause memory leaks. (Bug #26724559)
DUMPcode 7027 to facilitate testing of issues relating to local checkpoints. For more information, see DUMP 7027. (Bug #26661468)
A previous fix intended to improve logging of node failure handling in the transaction coordinator included logging of transactions that could occur in normal operation, which made the resulting logs needlessly verbose. Such normal transactions are no longer written to the log in such cases. (Bug #26568782)
References: This issue is a regression of: Bug #26364729.
Due to a configuration file error, CPU locking capability was not available on builds for Linux platforms. (Bug #26378589)
DUMPcodes used for the
LGMANkernel block were incorrectly assigned numbers in the range used for codes belonging to
DBTUX. These have now been assigned symbolic constants and numbers in the proper range (10001, 10002, and 10003). (Bug #26365433)
Node failure handling in the
DBTCkernel block consists of a number of tasks which execute concurrently, and all of which must complete before TC node failure handling is complete. This fix extends logging coverage to record when each task completes, and which tasks remain, includes the following improvements:
Handling interactions between GCP and node failure handling interactions, in which TC takeover causes GCP participant stall at the master TC to allow it to extend the current GCI with any transactions that were taken over; the stall can begin and end in different GCP protocol states. Logging coverage is extended to cover all scenarios. Debug logging is now more consistent and understandable to users.
Logging done by the
QMGRblock as it monitors duration of node failure handling duration is done more frequently. A warning log is now generated every 30 seconds (instead of 1 minute), and this now includes
DBDIHblock debug information (formerly this was written separately, and less often).
To reduce space used,
DBTC instanceis shortened to
A new error code is added to assist testing.
NDB Cluster did not compile successfully when the build used
WITH_UNIT_TESTS=OFF. (Bug #86881, Bug #26375985)
A potential hundredfold signal fan-out when sending a
START_FRAG_REQsignal could lead to a node failure due to a job buffer full error in start phase 5 while trying to perform a local checkpoint during a restart. (Bug #86675, Bug #26263397)
References: See also: Bug #26288247, Bug #26279522.
Compilation of NDB Cluster failed when using
-DWITHOUT_SERVER=1to build only the client libraries. (Bug #85524, Bug #25741111)
Functionality Added or Changed
NDB Cluster APIs; ndbinfo Information Database: Added two tables to the
ndbinfoinformation database. The
config_nodestable provides information about nodes that are configured as part of a given NDB Cluster, such as node ID and process type. The
processestable shows information about nodes currently connected to the cluster; this information includes the process name and system process ID, and service address. For each data node and SQL node, it also shows the process ID of the node's angel process.
As part of the work done to implement the
processestable, a new
set_service_uri()method has been added to the NDB API.
NDB Cluster APIs: The system name of an NDB cluster is now visible in the mysql client as the value of the
Ndb_system_namestatus variable, and can also be obtained by NDB API application using the
Ndb_cluster_connection::get_system_name()method. The system name can be set using the
Nameparameter in the
[system]section of the cluster configuration file.
--diff-defaultoption for ndb_config. This option causes the program to print only those parameters having values that differ from their defaults. (Bug #85831, Bug #25844166)
--query-alloption to ndb_config. This option acts much like the
--queryoption except that
-a) dumps configuration information for all attributes at one time. (Bug #60095, Bug #11766869)
Packaging: Two missing dependencies were added to the
The data node package requires
The auto-installer requires
(Bug #85679, Bug #25799465)
NDB Cluster APIs: The implementation method
NdbDictionary::NdbTableImpl::getColumn(), used from many places in the NDB API where a column is referenced by name, has been made more efficient. This method used a linear search of an array of columns to find the correct column object, which could be inefficient for tables with many columns, and was detected as a significant use of CPU in customer applications. (Ideally, users should perform name-to-column object mapping, and then use column IDs or objects in method calls, but in practice this is not always done.) A less costly hash index implementation, used previously for the name lookup, is reinstated for tables having relatively many columns. (A linear search continues to be used for tables having fewer columns, where the difference in performance is neglible.) (Bug #24829435)
.logfiles contained log entries for one or more extra fragments, due to an issue with filtering out changes logged by other nodes in the same node group. This resulted in a larger
.logfile and thus use of more resources than necessary; it could also cause problems when restoring, since backups from different nodes could interfere with one another while the log was being applied. (Bug #25891014)
When making the final write to a redo log file, it is expected that the next log file is already opened for writes, but this was not always the case with a slow disk, leading to node failure. Now in such cases
NDBwaits for the next file to be opened properly before attempting to write to it. (Bug #25806659)
Data node threads can be bound to a single CPU or a set of CPUs, a set of CPUs being represented internally by
SparseBitmask. When attempting to lock to a set of CPUs, CPU usage was excessive due to the fact that the routine performing the locks used the
mt_thr_config.cpp::do_bind()method, which looks for bits that are set over the entire theoretical range of the
SparseBitmask(232-2, or 4294967294). This is fixed by using
SparseBitmask::getBitNo(), which can be used to iterate over only those bits that are actually set, instead. (Bug #25799506)
ndb_report_thresh_binlog_epoch_slipwas enabled, an event buffer status message with
report_reason=LOW/ENOUGH_FREE_EVENTBUFFERwas printed in the logs when event buffer usage was high and then decreased to a lower level. This calculation was based on total allocated event buffer memory rather than the limit set by
ndb_eventbuffer_max_alloc; it was also printed even when the event buffer had unlimited memory (
ndb_eventbuffer_max_alloc= 0, the default), which could confuse users.
This is fixed as follows:
The calculation of
ndb_eventbuffer_free_percentis now based on
ndb_eventbuffer_max_alloc, rather than the amount actually allocated.
ndb_eventbuffer_free_percentis set and
ndb_eventbuffer_max_allocis equal to 0, event buffer status messages using
report_reason=LOW/ENOUGH_FREE_EVENTBUFFERare no longer printed.
ndb_report_thresh_binlog_epoch_slipis set, an event buffer status message showing
report_reason=BUFFERED_EPOCHS_OVER_THRESHOLDis written each 10 seconds (rather than every second) whenever this is greater than the threshold.
A bulk update is executed by reading records and executing a transaction on the set of records, which is started while reading them. When transaction initialization failed, the transaction executor function was subsequently unaware that this had occurred, leading to SQL node failures. This issue is fixed by providing appropriate error handling when attempting to initialize the transaction. (Bug #25476474)
References: See also: Bug #20092754.
NoOfFragmentLogPartssuch that there were more than 4 redo log parts per local data manager led to resource exhaustion and subsequent multiple data node failures. Since this is an invalid configuration, a check has been added to detect a configuration with more than 4 redo log parts per LDM, and reject it as invalid. (Bug #25333414)
Execution of an online
ALTER TABLE ... REORGANIZE PARTITIONstatement on an
NDBtable having a primary key whose length was greater than 80 bytes led to restarting of data nodes, causing the reorganization to fail. (Bug #25152165)
In certain cases, a failed
ALTER TABLE ... ADD UNIQUE KEYstatement could lead to SQL node failure. (Bug #24444878)
References: This issue is a regression of: Bug #23089566.
Error 240 is raised when there is a mismatch between foreign key trigger columns and the values supplied to them during trigger execution, but had no error message indicating the source of the problem. (Bug #23141739)
References: See also: Bug #23068914, Bug #85857.
If the number of LDM blocks was not evenly divisible by the number of TC/SPJ blocks, SPJ requests were not equally distributed over the available SPJ instances. Now a round-robin distribution is used to distribute SPJ requests across all available SPJ instances more effectively.
As part of this work, a number of unused member variables have been removed from the class
Dbtc. (Bug #22627519)
ALTER TABLE .. MAX_ROWS=0can now be performed only by using a copying
ALTER TABLEstatement. Resetting
MAX_ROWSto 0 can no longer be performed using
ALGORITHM=INPLACE. (Bug #21960004)
During a system restart, when a node failed due to having missed sending heartbeats, all other nodes reported only that another node had failed without any additional information. Now in such cases, the fact that heartbeats were missed and the ID of the node that failed to send heartbeats is reported in both the error log and the data node log. (Bug #21576576)
The planned shutdown of an NDB Cluster having more than 10 data nodes was not always performed gracefully. (Bug #20607730)
Due to a previous issue with unclear separation between the optimize and execute phases when a query involved a
GROUP BY, the join-pushable evaluator was not sure whether its optimized query execution plan was in fact pushable. For this reason, such grouped joins were always considered not pushable. It has been determined that the separation issue has been resolved by work already done in MySQL 5.6, and so we now remove this limitation. (Bug #86623, Bug #26239591)
When deleting all rows from a table immediately followed by
DROP TABLE, it was possible that the shrinking of the
DBACChash index was not ready prior to the drop. This shrinking is a per-fragment operation that does not check the state of the table. When a table is dropped,
DBACCreleases resources, during which the description of the fragment size and page directory is not consistent; this could lead to reads of stale pages, and undefined behavior.
Inserting a great many rows followed by dropping the table should also have had such effects due to expansion of the hash index.
To fix this problem we make sure, when a fragment is about to be released, that there are no pending expansion or shrinkage operations on this fragment. (Bug #86449, Bug #26138592)
The internal function
mt.cppread three section pointers from the signal even when none was passed to it. This was mostly harmless, although unneeded. When the signal read was the last one on the last page in the job buffer, and the next page in memory was not mapped or otherwise accessible, ndbmtd failed with an error. To keep this from occurring, this function now only reads section pointers that are actually passed to it. (Bug #86354, Bug #26092639)
The ndb_show_tables program
--unqualifiedoption did not work correctly when set to 0 (false); this should disable the option and so cause fully qualified table and index names to be printed in the output. (Bug #86017, Bug #25923164)
NDBtable with foreign key constraints is created, its indexes are created first, and then, during foreign key creation, these indexes are loaded into the
NDBdictionary cache. When a
CREATE TABLEstatement failed due to an issue relating to foreign keys, the indexes already in the cache were not invalidated. This meant that any subsequent
CREATE TABLEwith any indexes having the same names as those in the failed statement produced inconsistent results. Now, in such cases, any indexes named in the failed
CREATE TABLEare immediately invalidated from the cache. (Bug #85917, Bug #25882950)
Attempting to execute
ALTER TABLE ... ADD FOREIGN KEYwhen the key to be added had the name of an existing foreign key on the same table failed with the wrong error message. (Bug #85857, Bug #23068914)
The node internal scheduler (in
mt.cpp) collects statistics about its own progress and any outstanding work it is performing. One such statistic is the number of outstanding send bytes, collected in
send_buffer::m_node_total_send_buffer_size. This information may later be used by the send thread scheduler, which uses it as a metric to tune its own send performance versus latency.
In order to reduce lock contention on the internal send buffers, they are split into two
m_sending, each protected by its own mutex, and their combined size repesented by
Investigation of the code revealed that there was no consistency as to which mutex was used to update
m_node_total_send_buffer_size, with the result that there was no consurrency protection for this value. To avoid this,
m_node_total_send_buffer_sizeis replaced with two values,
m_sending_size, which keep separate track of the sizes of the two buffers. These counters are updated under the protection of two different mutexes protecting each buffer individually, and are now added together to obtain the total size.
With concurrency control established, updates of the partial counts should now be correct, so that their combined value no longer accumulates errors over time. (Bug #85687, Bug #25800933)
TRANS_AIsignals that used the long signal format were not handled by the
DBTCkernel block. (Bug #85606, Bug #25777337)
References: See also: Bug #85519, Bug #27540805.
To prevent a scan from returning more rows, bytes, or both than the client has reserved buffers for, the
DBTUPkernel block reports the size of the
TRANSID_AIit has sent to the client in the
TUPKEYCONFsignal it sends to the requesting
DBLQHis aware of the maximum batch size available for the result set, and terminates the scan batch if this has been exceeded.
DBTUPto produce two
TRANSID_AIresults from the same row, one for the client, and one for
DBSPJ, which is needed for key lookups on the joined tables. The size of both of these were added to the read length reported by the
DBTUPblock, which caused the controlling
DBLQHblock to believe that it had consumed more of the available maximum batch size than was actually the case, leading to premature termination of the scan batch which could have a negative impact on performance of SPJ scans. To correct this, only the actual read length part of an API request is now reported in such cases. (Bug #85408, Bug #25702850)
Data node binaries for Solaris 11 built using Oracle Developer Studio 12.5 on SPARC platforms failed with bus errors. (Bug #85390, Bug #25695818)
When compiling the NDB kernel with gcc version 6.0.0 or later, it is now built using
-flifetime-dse=1. (Bug #85381, Bug #25690926)
Ubuntu 14.04 and Ubuntu 16.04 are now supported.
Functionality Added or Changed
Packaging: Yum repo packages are added for EL5, EL6, EL7, and SLES12.
Apt repo packages are added for Debian 7, Debian 8, Ubuntu 14.04, and Ubuntu 16.04
This was due to the fact that, when processing an
EXPLAINstatement, mysqld calculates the partition ID for a hash value as (
), which is correct only when the table is partitioned by
HASH, since other partitioning types use different methods of mapping hash values to partition IDs. This fix replaces the partition ID calculation performed by mysqld with an internal
NDBfunction which calculates the partition ID correctly, based on the table's partitioning type. (Bug #21068548)
References: See also: Bug #25501895, Bug #14672885.
NDB Disk Data: Stale data from NDB Disk Data tables that had been dropped could potentially be included in backups due to the fact that disk scans were enabled for these. To prevent this possibility, disk scans are now disabled—as are other types of scans—when taking a backup. (Bug #84422, Bug #25353234)
NDB Disk Data: In some cases, setting dynamic in-memory columns of an NDB Disk Data table to
NULLwas not handled correctly. (Bug #79253, Bug #22195588)
NDB Cluster APIs: When signals were sent while the client process was receiving signals such as
TC_COMMIT_ACK, these signals were temporary buffered in the send buffers of the clients which sent them. If not explicitly flushed, the signals remained in these buffers until the client woke up again and flushed its buffers. Because there was no attempt made to enforce an upper limit on how long the signal could remain unsent in the local client buffers, this could lead to timeouts and other misbehavior in the components waiting for these signals.
In addition, the fix for a previous, related issue likely made this situation worse by removing client wakeups during which the client send buffers could have been flushed.
The current fix moves responsibility for flushing messages sent by the receivers, to the receiver (
poll_ownerclient). This means that it is no longer necessary to wake up all clients merely to have them flush their buffers. Instead, the
poll_ownerclient (which is already running) performs flushing the send buffer of whatever was sent while delivering signals to the recipients. (Bug #22705935)
References: See also: Bug #18753341, Bug #23202735.
CPU usage of the data node's main thread by the
DBDIHmaster block as the end of a local checkpoint could approach 100% in certain cases where the database had a very large number of fragment replicas. This is fixed by reducing the frequency and range of fragment queue checking during an LCP. (Bug #25443080)
The ndb_print_backup_file utility failed when attempting to read from a backup file when the backup included a table having more than 500 columns. (Bug #25302901)
References: See also: Bug #25182956.
Multiple data node failures during a partial restart of the cluster could cause API nodes to fail. This was due to expansion of an internal object ID map by one thread, thus changing its location in memory, while another thread was still accessing the old location, leading to a segmentation fault in the latter thread.
unmap()functions in which this issue arose have now been made thread-safe. (Bug #25092498)
References: See also: Bug #25306089.
During the initial phase of a scan request, the
DBTCkernel block sends a series of
DIGETNODESREQsignals to the
DBDIHblock in order to obtain dictionary information for each fragment to be scanned. If
DIGETNODESREF, the error code from that signal was not read, and Error 218 Out of LongMessageBuffer was always returned instead. Now in such cases, the error code from the DIGETNODESREF signal is actually used. (Bug #85225, Bug #25642405)
There existed the possibility of a race condition between schema operations on the same database object originating from different SQL nodes; this could occur when one of the SQL nodes was late in releasing its metadata lock on the affected schema object or objects in such a fashion as to appear to the schema distribution coordinator that the lock release was acknowledged for the wrong schema change. This could result in incorrect application of the schema changes on some or all of the SQL nodes or a timeout with repeated waiting max
###sec for distributing... messages in the node logs due to failure of the distribution protocol. (Bug #85010, Bug #25557263)
References: See also: Bug #24926009.
When a foreign key was added to or dropped from an NDB table using an
ALTER TABLEstatement, the parent table's metadata was not updated, which made it possible to execute invalid alter operations on the parent afterwards.
Until you can upgrade to this release, you can work around this problem by running
SHOW CREATE TABLEon the parent immediately after adding or dropping the foreign key; this statement causes the table's metadata to be reloaded. (Bug #82989, Bug #24666177)
NDBtables with cascading foreign keys returned inconsistent results when the query cache was also enabled, due to the fact that mysqld was not aware of child table updates. This meant that results for a later
SELECTfrom the child table were fetched from the query cache, which at that point contained stale data.
This is fixed in such cases by adding all children of the parent table to an internal list to be checked by
NDBfor updates whenever the parent is updated, so that mysqld is now properly informed of any updated child tables that should be invalidated from the query cache. (Bug #81776, Bug #23553507)
Packaging: NDB Cluster Auto-Installer RPM packages for SLES 12 failed due to a dependency on
python-pycrypto. (Bug #25399608)
Packaging: The RPM installer for the MySQL NDB Cluster
auto-installerpackage had a dependency on
python-crypt. (Bug #24924607)
Microsoft Windows: Installation failed when the Auto-Installer (ndb_setup.py) was run on a Windows host that used Swedish as the system language. This was due to system messages being issued using the
cp1252character set; when these messages contained characters that did not map directly to 7-bit ASCII (such as the
Tjänsten ... startar), conversion to
UTF-8—as expected by the Auto-Installer web client—failed.
This fix has been tested only with Swedish as the system language, but should work for Windows systems set to other European languages that use the
cp1252character set. (Bug #83870, Bug #25111830)
ndb_restore did not restore tables having more than 341 columns correctly. This was due to the fact that the buffer used to hold table metadata read from
.ctlfiles was of insufficient size, so that only part of the table descriptor could be read from it in such cases. This issue is fixed by increasing the size of the buffer used by ndb_restore for file reads. (Bug #25182956)
References: See also: Bug #25302901.
No traces were written when ndbmtd received a signal in any thread other than the main thread, due to the fact that all signals were blocked for other threads. This issue is fixed by the removal of
SIGSEGVsignals from the list of signals being blocked. (Bug #25103068)
rand()function was used to produce a unique table ID and table version needed to identify a schema operation distributed between multiple SQL nodes, relying on the assumption that
rand()would never produce the same numbers on two different instances of mysqld. It was later determined that this is not the case, and that in fact it is very likely for the same random numbers to be produced on all SQL nodes.
This fix removes the usage of
rand()for producing a unique table ID or version, and instead uses a sequence in combination with the node ID of the coordinator. This guarantees uniqueness until the counter for the sequence wraps, which should be sufficient for this purpose.
The effects of this duplication could be observed as timeouts in the log (for example NDB create db: waiting max 119 sec for distributing) when restarting multiple mysqld processes simultaneously or nearly so, or when issuing the same
DROP DATABASEstatement on multiple SQL nodes. (Bug #24926009)
The ndb_show_tables utility did not display type information for hash maps or fully replicated triggers. (Bug #24383742)
Long message buffer exhaustion when firing immediate triggers could result in row ID leaks; this could later result in persistent RowId already allocated errors (
NDBError 899). (Bug #23723110)
References: See also: Bug #19506859, Bug #13927679.
The NDB Cluster Auto-Installer did not show the user how to force an exit from the application (CTRL+C). (Bug #84235, Bug #25268310)
The NDB Cluster Auto-Installer failed to exit when it was unable to start the associated service. (Bug #84234, Bug #25268278)
The NDB Cluster Auto-Installer failed when the port specified by the
--portoption (or the default port 8081) was already in use. Now in such cases, when the required port is not available, the next 20 ports are tested in sequence, with the first one available being used; only if all of these are in use does the Auto-Installer fail. (Bug #84233, Bug #25268221)
Multiples instances of the NDB Cluster Auto-Installer were not detected. This could lead to inadvertent multiple deployments on the same hosts, stray processes, and similar issues. This issue is fixed by having the Auto-Installer create a PID file (
mcc.pid), which is removed upon a successful exit. (Bug #84232, Bug #25268121)
when a parent
NDBtable in a foreign key relationship was updated, the update cascaded to a child table as expected, but the change was not cascaded to a child table of this child table (that is, to a grandchild of the original parent). This can be illustrated using the tables generated by the following
CREATE TABLE parent( id INT PRIMARY KEY AUTO_INCREMENT, col1 INT UNIQUE, col2 INT ) ENGINE NDB; CREATE TABLE child( ref1 INT UNIQUE, FOREIGN KEY fk1(ref1) REFERENCES parent(col1) ON UPDATE CASCADE ) ENGINE NDB; CREATE TABLE grandchild( ref2 INT, FOREIGN KEY fk2(ref2) REFERENCES child(ref1) ON UPDATE CASCADE ) ENGINE NDB;
childis a child of table
grandchildis a child of table
child, and a grandchild of
parent. In this scenario, a change to column
child, but it was not always propagated in turn to
grandchild. (Bug #83743, Bug #25063506)
NDBbinlog injector thread used an injector mutex to perform two important tasks:
Protect against client threads creating or dropping events whenever the injector thread waited for
Maintain access to data shared by the injector thread with client threads.
The first of these could hold the mutex for long periods of time (on the order of 10ms), while locking it again extremely quickly. This could keep it from obtaining the lock for data access (“starved”) for unnecessarily great lengths of time.
To address these problems, the injector mutex has been refactored into two—one to handle each of the two tasks just listed.
It was also found that initialization of the binlog injector thread held the injector mutex in several places unnecessarily, when only local thread data was being initialized and sent signals with condition information when nothing being waited for was updated. These unneeded actions have been removed, along with numerous previous temporary fixes for related injector mutex starvation issues. (Bug #83676, Bug #83127, Bug #25042101, Bug #24715897)
References: See also: Bug #82680, Bug #20957068, Bug #24496910.
When a data node running with
StopOnErrorset to 0 underwent an unplanned shutdown, the automatic restart performed the same type of start as the previous one. In the case where the data node had previously been started with the
--initialoption, this meant that an initial start was performed, which in cases of multiple data node failures could lead to loss of data. This issue also occurred whenever a data node shutdown led to generation of a core dump. A check is now performed to catch all such cases, and to perform a normal restart instead.
In addition, in cases where a failed data node was unable prior to shutting down to send start phase information to the angel process, the shutdown was always treated as a startup failure, also leading to an initial restart. This issue is fixed by adding a check to execute startup failure handling only if a valid start phase was received from the client. (Bug #83510, Bug #24945638)
When ndbmtd was built on Solaris/SPARC with version 5.3 of the GNU tools, data nodes using the resulting binary failed during startup. (Bug #83500, Bug #24941880)
References: See also: Bug #83517, Bug #24947597.
MySQL NDB Cluster failed to compile using GCC 6. (Bug #83308, Bug #24822203)
When a data node was restarted, the node was first stopped, and then, after a fixed wait, the management server assumed that the node had entered the
NOT_STARTEDstate, at which point, the node was sent a start signal. If the node was not ready because it had not yet completed stopping (and was therefore not actually in
NOT_STARTED), the signal was silently ignored.
To fix this issue, the management server now checks to see whether the data node has in fact reached the NOT_STARTED state before sending the start signal. The wait for the node to reach this state is split into two separate checks:
Wait for data nodes to start shutting down (maximum 12 seconds)
Wait for data nodes to complete shutting down and reach
NOT_STARTEDstate (maximum 120 seconds)
If either of these cases times out, the restart is considered failed, and an appropriate error is returned. (Bug #49464, Bug #11757421)
Functionality Added or Changed
Important Change; Packaging: Naming and organization of the RPMs provided for MySQL NDB Cluster have been changed to align with those released for the MySQL server. All MySQL NDB Cluster RPM package names are now prefixed with
mysql-cluster. Data nodes are now installed using the
data-nodepackage; management nodes are now installed from the
management-serverpackage; and SQL nodes require the
commonpackages. Important: SQL nodes must use the
mysql-clusterversion of these RPMs; the versions released for the standard MySQL server do not provide support for the
NDBstorage engine. All client programs, including both the mysql client and the ndb_mgm management client, are now included in the
For more information, see Installing NDB Cluster from RPM.
Important Change: Added the
FOR_RA_BY_LDM_X_4, which can be used to set the number of partitions used by each local data manager to two, three, and four partitions, respectively, in addition to
FOR_RA_BY_LDM, which sets this number to one.
MAX_ROWSfor setting the number of partitions used by
NDBtables is now deprecated and subject to removal in a future version of MySQL NDB Cluster.
For more information, see Setting NDB_TABLE Options. (Bug #81759, Bug #23544301)
--print-sql-logoption for the ndb_restore program included with the MySQL NDB Cluster distribution. This option causes the program to log SQL statements to
Note that each table being restored in this fashion must have an explicitly defined primary key; the hidden primary key implemented by the
NDBstorage engine is not sufficient for this purpose. (Bug #13511949)
For fully replicated tables, ndb_desc shows only nodes holding main fragment replicas for partitions; nodes with copy fragment replicas only are ignored. To make this information available in the mysql client, several new tables have been introduced in the
ndbinfoinformation database. These tables are listed here, with brief descriptions:
dict_obj_infoprovides the names and types of database (
DICT) objects in
NDB, such as tables and indexes, as well as information about parent objects where applicable
NDBtable distribution status information
table_fragmentsprovides information about the distribution of
table_infoprovides information about logging, checkpointing, storage, and other options in force for each
table_replicasprovides information about fragment replicas
For more information, see ndbinfo: The NDB Cluster Information Database. (Bug #81762, Bug #23547643)
Important Change: The default value of the
--ndb-default-column-formatserver option has been changed from
FIXED. This has been done for backwards compatibility. Only the default has been changed; setting this option to
DYNAMICcontinues to cause
DYNAMICto be used for
COLUMN_FORMATunless overridden. (Bug #24487363)
Important Change: Event buffer status reporting has been improved by altering the semantics for calculating lag or slippage. Rather than defining this lag as the number of epochs behind, lag is now taken as the number of epochs completely buffered in the event buffer, but not yet consumed by the binlog injector thread. As part of this work, the default value for the
ndb_report_thresh_binlog_epoch_slipsystem variable has been increased from 3 to 10. For more information, see the description of this variable in the documentation, as well as Event Buffer Reporting in the Cluster Log. (Bug #22916457)
References: See also: Bug #22901309.
NDB Cluster APIs: Reuse of transaction IDs could occur when
Ndbobjects were created and deleted concurrently. As part of this fix, the NDB API methods
unlock_ndb_objectsare now declared as
const. (Bug #23709232)
NDB Cluster APIs: When the management server was restarted while running an MGM API application that continuously monitored events, subsequent events were not reported to the application, with timeouts being returned indefinitely instead of an error.
This occurred because sockets for event listeners were not closed when restarting mgmd. This is fixed by ensuring that event listener sockets are closed when the management server shuts down, causing applications using functions such as
ndb_logevent_get_next()to receive a read error following the restart. (Bug #19474782)
NDB Cluster APIs: To process incoming signals, a thread which wants to act as a receiver must acquire polling rights from the transporter layer. This can be requested and assigned to a separate receiver thread, or each client thread can take the receiver role when it is waiting for a result.
When the thread acting as poll owner receives a sufficient amount of data, it releases locks on any other clients taken while delivering signals to them. This could make them runnable again, and the operating system scheduler could decide that it was time to wake them up, which happened at the expense of the poll owner threads, which were in turn excluded from the CPU while still holding polling rights on it. After this fix, polling rights are released by a thread before unlocking and signalling other threads. This makes polling rights available for other threads that are actively executing on this CPU.
This change increases concurrency when polling receiver data, which should also reduce latency for clients waiting to be woken up. (Bug #83129, Bug #24716756)
NDB Cluster APIs:
libmysqlclientexported conflicting symbols, resulting in a segmentation fault in debug builds on Linux. To fix this issue, the conflicting symbols in
libndbclient.soare no longer publicly visible. Due to this change, the version number for
libndbclient.sohas been raised from 6.0.0 to 6.1.0. (Bug #83093, Bug #24707521)
References: See also: Bug #80352, Bug #22722555.
NDB Cluster APIs: When
NDBschema object ownership checks are enabled by a given
NdbTransaction, objects used by this transaction are checked to make sure that they belong to the
NdbDictionaryowned by this connection. An attempt to create a
NdbIndexScanOperationon a table or index not belonging to the same connection fails.
This fix corrects a resource leak which occurred when the operation object to be created was allocated before checking schema object ownership and subsequently not released when the object creation failed. (Bug #81949, Bug #23623978)
References: See also: Bug #81945, Bug #23623251.
NDB Cluster APIs: NDB API objects are allocated in the context of an
Ndbobject, or of an
NdbTransactionobject which is itself owned by an
Ndbobject. When a given
Ndbobject is destroyed, all remaining
NdbTransactionobjects are terminated, and all NDB API objects related to this
Ndbobject should be released at this time as well. It was found, when there remained unclosed NdbTransaction objects when their parent
Ndbobject was destroyed, leaks of objects allocated from the
NdbTransactionobjects could occur. (However, the
NdbTransactionobjects themselves did not leak.)
While it is advisable (and, indeed, recommended) to close an
NdbTransactionexplicitly as soon as its lifetime ends, the destruction of the parent
Ndbobject should be sufficient to release whatever objects are dependent on it. Now in cases such as described previously, the
Ndbdestructor checks to ensure that all objects derived from a given
Ndbinstance are truly released. (Bug #81945, Bug #23623251)
NDB Cluster APIs: The term “fragment count type” has been superceded by “partition balance”. This change affects
NDBtables as well as in the NDB API. In
NDB_TABLEtable option syntax, the
FRAGMENT_COUNT_TYPEkeyword is replaced with
PARTITION_BALANCE. In the NDB API, the
setFragmentCountType()have been renamed to
getFragmentCountTypeString()is renamed to
getPartitionBalanceString(). In addition,
Object::FragmentCountTypehas been renamed to
PartitionBalance, and the names of its enumerated values have been updated to be consistent with the new nomenclature.
For more information on how these changes affect NDB API applications, see the indicated
Objectmember descriptions. For more information on the SQL-level changes made as part of this fix, Setting NDB_TABLE Options. (Bug #81761, Bug #23547525)
References: See also: Bug #83147, Bug #24733331.
NDB Cluster APIs: In some of the NDB API example programs included with the MySQL NDB Cluster distribution,
ndb_end()was called prior to calling the
Ndb_cluster_connectiondestructor. This caused a segmentation fault in debug builds on all platforms. The example programs affected have also been extensively revised and refactored. See NDB API Examples, for more information. (Bug #80352, Bug #22722555)
References: See also: Bug #83093, Bug #24707521.
If more than 4096 seconds elapsed while calculating an internal
NdbDuration::microSec()value, this could cause an assert warning that the calculation would overflow. We fix this to avoid any overflow or precision loss when converting from the internal “tick” format to microseconds and nanoseconds, by performing the calculation in two parts corresponding to seconds and fractions of a second. (Bug #24695026)
The serial commit protocol—which commits each operation at each replica individually and serially, and is used by the
DBTCkernel block (see The DBTC Block) for takeover and when a transaction is judged to have timed out during the
COMPLETEphase—had no support for
LATE_COMMIT, which is required for the
FULLY_REPLICATEDprotocols. (Bug #24681305)
In some cases,
ALTER TABLE ... REORGANIZE PARTITIONcould lead to an unplanned shutdown of the cluster. This was due to the fact that, for fully replicated tables. the log part ID was assumed to be the same as the partition ID. This worked when
FOR_RA_BY_LDMwas used, but not necessarily for the other partition balancing types. (Bug #24610551)
ALGORITHM=INPLACEwhen changing any of a table's
NDB_TABLEproperties (see Setting NDB_TABLE Options) caused the server to fail. (Bug #24584741)
Case-insensitivity of keywords such as
NDB_TABLEcomments was not honored. (Bug #24577931)
A number of dependencies between the binlog injector thread and the
NDButility thread—a recurring source of synchronization and other problems—were removed. The principal changes are listed here:
Moved the setup of binlog injector structures from the utility thread to the injector thread itself.
Removed sharing of some utility and injector thread structures between these threads.
Moved stopping of the utility thread from the injector thread into a common block in which other such threads are stopped.
Removed a number of hacks required by the previous design.
Removed some injector mutex locking and injector condition signaling which were made obsolete by the changes already listed.
References: See also: Bug #22204186.
A late commit
ACKsignal used for
READ_BACKUPtables caused the associated
ApiConnectionRecordto have an invalid state. (Bug #24459817)
References: See also: Bug #24444861.
Added missing error information for a failure occurring when tables on disk became full. (Bug #24425373)
When ndbmtd crashed, the resulting error log incorrectly specified the name of the trace for thread 0, appending the nonexistent suffix
_t0to the file name. (Bug #24353408)
Passing a nonexistent node ID to
CREATE NODEGROUPled to random data node failures. (Bug #23748958)
DROP TABLEfollowed by a node shutdown and subesequent master takeover—and with the containing local checkpoint not yet complete prior to the takeover—caused the LCP to be ignored, and in some cases, the data node to fail. (Bug #23735996)
References: See also: Bug #23288252.
Removed an invalid assertion to the effect that all cascading child scans are closed at the time API connection records are released following an abort of the main transaction. The assertion was invalid because closing of scans in such cases is by design asynchronous with respect to the main transaction, which means that subscans may well take some time to close after the main transaction is closed. (Bug #23709284)
Although arguments to the
DUMPcommand are 32-bit integers, ndb_mgmd used a buffer of only 10 bytes when processing them. (Bug #23708039)
READ_BACKUPsetting was not honored when performing scans on
BLOBtables. (Bug #23703536)
READ_BACKUPsetting was not applied to unique indexes. (Bug #23702848)
DBSPJread primary fragment replicas for tables with
READ_BACKUP(see Setting NDB_TABLE Options), even when a local fragment was available. (Bug #23633848)
ALL REPORT MemoryUsageproduced incorrect output when fully replicated tables were in use. (Bug #23539805)
Ordered indexes did not inherit
READ_BACKUP(see Setting NDB_TABLE Options) from an indexed table, which meant that ordered index scans continued to be routed to only to primary fragment replicas and never to backup fragment replicas.
DBDICTsets this property on ordered indexes from the table property when it distributes this information to instances of
DBSPJ. (Bug #23522027)
Updates to a table containing a virtual column could cause the binary logging thread to fail. (Bug #23514050)
A number of potential buffer overflow issues were found and fixed in the
NDBcodebase. (Bug #23152979)
During an online upgrade from a MySQL NDB Cluster 7.3 release to an NDB 7.4 (or later) release, the failures of several data nodes running the lower version during local checkpoints (LCPs), and just prior to upgrading these nodes, led to additional node failures following the upgrade. This was due to lingering elements of the
EMPTY_LCPprotocol initiated by the older nodes as part of an LCP-plus-restart sequence, and which is no longer used in NDB 7.4 and later due to LCP optimizations implemented in those versions. (Bug #23129433)
SIGNAL_DROPPED_REPhandler invoked in response to long message buffer exhaustion was defined in the
SPJkernel block, but not actually used. This meant that the default handler from
SimulatedBlockwas used instead in such cases, which shut down the data node. (Bug #23048816)
References: See also: Bug #23251145, Bug #23251423.
When a data node has insufficient redo buffer during a system restart, it does not participate in the restart until after the other nodes have started. After this, it performs a takeover of its fragments from the nodes in its node group that have already started; during this time, the cluster is already running and user activity is possible, including DML and DDL operations.
During a system restart, table creation is handled differently in the
DIHkernel block than normally, as this creation actually consists of reloading table definition data from disk on the master node. Thus,
DIHassumed that any table creation that occurred before all nodes had restarted must be related to the restart and thus always on the master node. However, during the takeover, table creation can occur on non-master nodes due to user activity; when this happened, the cluster underwent a forced shutdown.
Now an extra check is made during system restarts to detect in such cases whether the executing node is the master node, and use that information to determine whether the table creation is part of the restart proper, or is taking place during a subsequent takeover. (Bug #23028418)
ndb_restore set the
MAX_ROWSattribute for a table for which it had not been set prior to taking the backup. (Bug #22904640)
Whenever data nodes are added to or dropped from the cluster, the
NDBkernel's Event API is notified of this using a
SUB_GCP_COMPLETE_REPsignal with either the
ADD(add) flag or
SUB(drop) flag set, as well as the number of nodes to add or drop; this allows
NDBto maintain a correct count of
SUB_GCP_COMPLETE_REPsignals pending for every incomplete bucket. In addition to handling the bucket for the epoch associated with the addition or removal, it must also compensate for any later incomplete buckets associated with later epochs. Although it was possible to complete such buckets out of order, there was no handling of these, leading a stall in to event reception.
This fix adds detection and handling of such out of order bucket completion. (Bug #20402364)
References: See also: Bug #82424, Bug #24399450.
When performing online reorganization of tables, unique indexes were not included in the reorganization. (Bug #13714258)
Under very high loads, neither the receive thread nor any user thread had sufficient capacity to handle poll ownership properly. This meant that, as the load and the number of active thread increased, it became more difficult to sustain throughput. This fix attempts to increase the priority of the receive thread and retains poll ownership if successful.
This fix requires sufficient permissions to be enabled. On Linux systems, this means ensuring that either the data node binary or the user it runs as has permission to change the nice level. (Bug #83217, Bug #24761073)
When restoring a backup taken from a database containing tables that had foreign keys, ndb_restore disabled the foreign keys for data, but not for the logs. (Bug #83155, Bug #24736950)
Local reads of unique index and blob tables did not work correctly for fully replicated tables using more than one node group. (Bug #83016, Bug #24675602)
The effects of an
ALTER TABLEstatement changing a table to use
READ_BACKUPwere not preserved after a restart of the cluster. (Bug #82812, Bug #24570439)
PARTITION_BALANCEdid not work with fully replicated tables. (Bug #82801, Bug #24565265)
READ_BACKUPsettings were not propagated to internal blob tables. (Bug #82788, Bug #24558232)
The count displayed by the
c_execcolumn in the
ndbinfo.threadstattable was incomplete. (Bug #82635, Bug #24482218)
The internal function
ndbcluster_binlog_wait(), which provides a way to make sure that all events originating from a given thread arrive in the binary log, is used by
SHOW BINLOG EVENTSas well as when resetting the binary log. This function waits on an injector condition while the latest global epoch handled by
NDBis more recent than the epoch last committed in this session, which implies that this condition must be signalled whenever the binary log thread completes and updates a new latest global epoch. Inspection of the code revealed that this condition signalling was missing, and that, instead of being awakened whenever a new latest global epoch completes (~100ms), client threads waited for the maximum timeout (1 second).
This fix adds the missing injector condition signalling, while also changing it to a condition broadcast to make sure that all client threads are alerted. (Bug #82630, Bug #24481551)
Fully replicated internal foreign key or unique index triggers could fire multiple times, which led to aborted transactions for an insert or a delete operation. This happened due to redundant deferred constraint triggers firing during pre-commit. Now in such cases, we ensure that only triggers specific to unique indexes are fired in this stage. (Bug #82570, Bug #24454378)
Backups potentially could fail when using fully replicated tables due to their high usage (and subsequent exhaustion) of internal trigger resources. To compensate for this, the amount of memory reserved in the
NDBkernel for internal triggers has been increased, and is now based in part on the maximum number of tables. (Bug #82569, Bug #24454262)
References: See also: Bug #23539733.
NDBkernel, an incorrect check of state led in some cases to failure handling when no failure had actually occurred. (Bug #82568, Bug #24454093)
References: See also: Bug #23539733.
When returning from
LQHKEYREQwith failure in
LQHKEYREFin an internal trigger operation, no check was made as to whether the trigger was fully replicated, so that those triggers that were fully replicated were never handled. (Bug #82566, Bug #24453949)
References: See also: Bug #23539733.
READ_BACKUPhad not previously been set, then was set to 1 as part of an
ALTER TABLE ... ALGORITHM=INPLACEstatement, the change was not propagated to internal unique index tables or
BLOBtables. (Bug #82491, Bug #24424459)
Distribution of MySQL privileges was incomplete due to the failure of the
mysql_cluster_move_privileges()procedure to convert the
NDB. The root cause of this was an
ALTER TABLE ... ENGINE NDBstatement which sometimes failed when this table contained illegal
TIMESTAMPvalues. (Bug #82464, Bug #24430209)
The internal variable
m_max_warning_levelwas not initialized in
storage/ndb/src/kernel/blocks/thrman.cpp. This sometimes led to node failures during a restart when the uninitialized value was treated as 0. (Bug #82053, Bug #23717703)
Usually, when performing a system restart, all nodes are restored from redo logs and local checkpoints (LCPs), but in some cases some node might require a copy phase before it is finished with the system restart. When this happens, the node in question waits for all other nodes to start up completely before performing the copy phase. Notwithstanding the fact that it is thus possible to begin a local checkpoint before reaching start phase 4 in the
DBDIHblock, LCP status was initialized to
IDLEin all cases, which could lead to a node failure. Now, when performing this variant of a system restart, the LCP status is no longer initialized. (Bug #82050, Bug #23717479)
After adding a new node group online and executing
ALTER TABLE ... ALGORITHM=INPLACE REORGANIZE PARTITION, partition IDs were not set correctly for new fragments.
In a related change done as part of fixing this issue, ndb_desc
-pnow displays rows relating to partitions in order of partition ID. (Bug #82037, Bug #23710999)
STOP BACKUPit is possible sometimes that a few bytes are written to the backup data file before the backup process actually terminates. When using
ODIRECT, this resulted in the wrong error code being returned. Now in such cases, nothing is written to
O_DIRECTfiles unless the alignment is correct. (Bug #82017, Bug #23701911)
When transaction coordinator (TC) connection records were used up, it was possible to handle scans only for local checkpoints and backups, so that operations coming from the
ALTER TABLE ... REORGANIZE PARTITIONand other operations that reorganize metadata—were unnecessarily blocked. In addition, such operations were not always retried when TC records were exhausted. To fix this issue, a number of operation records are now earmarked for
DBUTILusage, as well as for LCP and backup usage so that these operations are also not negatively impacted by operations coming from
For more information, see The DBUTIL Block. (Bug #81992, Bug #23642198)
Operations performing multiple updates of the same row within the same transaction could sometimes lead to corruption of lengths of page entries. (Bug #81938, Bug #23619031)
During a node restart, a fragment can be restored using information obtained from local checkpoints (LCPs); up to 2 restorable LCPs are retained at any given time. When an LCP is reported to the
DIHkernel block as completed, but the node fails before the last global checkpoint index written into this LCP has actually completed, the latest LCP is not restorable. Although it should be possible to use the older LCP, it was instead assumed that no LCP existed for the fragment, which slowed the restart process. Now in such cases, the older, restorable LCP is used, which should help decrease long node restart times. (Bug #81894, Bug #23602217)
NDBno longer retries a global schema lock if this has failed due to a timeout (default 3000ms) and there is the potential for this lock request to participate in a metadata lock-global schema lock deadlock. Now in such cases it selects itself as a “victim”, and returns the decision to the requestor of the metadata lock, which then handles the request as a failed lock request (preferable to remaining deadlocked indefinitely), or, where a deadlock handler exists, retries the metadata lock-global schema lock. (Bug #81775, Bug #23553267)
Two issues were found in the implementation of hash maps—used by
NDBfor mapping a table row's hash value to a partition—for fully replicated tables:
Hash maps were selected based on the number of fragments rather than the number of partitions. This was previously undetected due to the fact that, for other kinds of tables, these values are always the same.
The hash map was employed as a partition-to-partition map, using the table row's hash value modulus the partition count as input.
This fix addresses both of the problems just described. (Bug #81757, Bug #23544220)
References: See also: Bug #81761, Bug #23547525, Bug #23553996.
Using mysqld together with
--ndbclusterled to problems later when attempting to use mysql_upgrade. When running with
--initialize, the server does not require
NDBsupport, and having it enabled can lead to issues with
ndbinfotables. To prevent this from happening, using the
--initializeoption now causes mysqld to ignore the
--ndbclusteroption if the latter is also specified.
This issue affects upgrades from MySQL NDB Cluster 7.5.2 or 7.5.3 only. In cases where such upgrades fail for the reasons outlined previously, you can work around the issue by deleting all
.frmfiles in the
data/ndbinfodirectory following a rolling restart of the entire cluster, then running mysql_upgrade. (Bug #81689, Bug #23518923)
References: See also: Bug #82724, Bug #24521927.
While a mysqld was waiting to connect to the management server during initialization of the
NDBhandler, it was not possible to shut down the mysqld. If the mysqld was not able to make the connection, it could become stuck at this point. This was due to an internal wait condition in the utility and index statistics threads that could go unmet indefinitely. This condition has been augmented with a maximum timeout of 1 second, which makes it more likely that these threads terminate themselves properly in such cases.
In addition, the connection thread waiting for the management server connection performed 2 sleeps in the case just described, instead of 1 sleep, as intended. (Bug #81585, Bug #23343673)
ALTER TABLE ... ALGORITHM=INPLACEon a fully replicated table did not copy the associated trigger ID, leading to a failure in the
DBDICTkernel block. (Bug #81544, Bug #23330359)
The list of deferred tree node lookup requests created when preparing to abort a
DBSPJrequest were not cleared when this was complete, which could lead to deferred operations being started even after the
DBSPJrequest aborted. (Bug #81355, Bug #23251423)
References: See also: Bug #23048816.
Error and abort handling in
Dbspj::execTRANSID_AI()was implemented such that its
abort()method was called before processing of the incoming signal was complete. Since this method sends signals to the LDM, this partly overwrote the contents of the signal which was later required by
execTRANSID_AI(). This could result in aborted
DBSPJrequests cleaning up their allocated resources too early, or not at all. (Bug #81353, Bug #23251145)
References: See also: Bug #23048816.
The read backup feature added in MySQL NDB Cluster 7.5.2 that makes it possible to read from backup replicas was not used for reads with lock, or for reads of
BLOBtables or unique key tables where locks were upgraded to reads with lock. Now the
SCAN_TABREQsignals use a flag to convey information about such locks making it possible to read from a backup replica when a read lock was upgraded due to being the read of the base table for a BLOB table, or due to being the read for a unique key. (Bug #80861, Bug #23001841)
Primary replicas of partitioned tables were not distributed evenly among node groups and local data managers.
As part of the fix for this issue, the maximum number of node groups supported for a single MySQL NDB Cluster, which was previously not determined, is now set at 48 (
MAX_NDB_NODE_GROUPS). (Bug #80845, Bug #22996305)
Several object constructors and similar functions in the
NDBcodebase did not always perform sanity checks when creating new instances. These checks are now performed under such circumstances. (Bug #77408, Bug #21286722)
An internal call to
malloc()was not checked for
NULL. The function call was replaced with a direct write. (Bug #77375, Bug #21271194)
Functionality Added or Changed
Important Change: It is now possible to set
READ_BACKUPfor an existing table online using an SQL statement such as
ALTER TABLE ... ALGORITHM=INPLACE, COMMENT="NDB_TABLE=READ_BACKUP=1". See Setting NDB_TABLE Options, for further information about the
READ_BACKUPoption. (Bug #80858, Bug #23001617)
References: See also: Bug #18435416.
Added three new tables to the
ndbinfoinformation database to provide running information about locks and lock attempts in an active MySQL NDB Cluster. These tables, with brief descriptions, are listed here:
cluster_locks: Current lock requests which are waiting for or holding locks; this information can be useful when investigating stalls and deadlocks. Analogous to
locks_per_fragment: Counts of lock claim requests, and their outcomes per fragment, as well as total time spent waiting for locks successfully and unsuccessfully. Analogous to
For more information, see ndbinfo: The NDB Cluster Information Database.
Indexes on virtual generated columns of
NDBtables are not supported.
ALTER TABLEstatements that add stored generated columns to
NDBtables cannot be performed online.
NDB Cluster APIs: The scan lock takeover issued by
NdbScanOperation::lockCurrentTuple()did not set the operation type for the takeover operation. (Bug #23314028)
During shutdown, the mysqld process could sometimes hang after logging
NDB Util: Stop...
NDB Util: Wakeup. (Bug #23343739)
References: See also: Bug #21098142.
During expansion or reduction of a hash table, allocating a new overflow page in the
DBACCkernel block caused the data node to fail when it was out of index memory. This could sometimes occur after a large table had increased or decreased very rapidly in size. (Bug #23304519)
References: This issue is a regression of: Bug #13436216.
When tracking time elapsed from sending a
SCAN_FRAGREQsignal to receiving a corresponding
SCAN_FRAGCONF, the assumption was made in the
DBTCkernel block that a
SCAN_FRAGCONFcan occur only after sending a
SCAN_NEXTREQsignal, which is not always the case: It is actually possible that a local query handler can, immediately after sending a
SCAN_FRAGCONF, send an additional
SCAN_FRAGCONFsignal upon reporting that the scan is closed. This problem is fixed by ensuring that the timer value is initialized each time before use. (Bug #81907, Bug #23605817, Bug #23306695)
Table indexes were listed in the output of ndb_desc in a nondeterministic order that could vary between platforms. Now these indexes are ordered by ID in the output. (Bug #81763, Bug #23547742)
Following a restart of the cluster, the first attempt to read from any of the
cpustat_20sectables generated a warning to the effect that columns were missing from the table. Subsequently, the
spin_timecolumns were found to be missing from each of these tables. (Bug #81681, Bug #23514557)
References: See also: Bug #23305078.
ThreadConfigparameter value with a trailing comma led to an assertion. (Bug #81588, Bug #23344374)
Functionality Added or Changed
Performance: A deficiency in event buffer memory allocation was identified as inefficient and possibly leading to undesirable results. This could happen when additional memory was allocated from the operating system to buffer received event data even when memory had already been allocated but remained unused. This is fixed by allocating the event buffer memory directly from the page allocation memory manager (
mmap()), where such functionality is offered by the operating system, allowing for direct control over the memory such that it is in fact returned to the system when released.
This remimplementation avoids the tendencies of the existing one to approach worst-case memory usage, maintainence of data structures for a worst-case event buffer event count, and useless caching of free memory in unusable positions. This work should also help minimize the runtime costs of buffering events, minimize heap fragmentation, and avoid OS-specific problems due to excessive numbers of distinct memory mappings.
In addition, the relationship between epochs and internal EventData objects is now preserved throughout the event lifecycle, reception to consumption, thus removing the need for iterating, and keeping in synch, two different lists representing the epochs and their
As part of this work, better reporting on the relevant event buffer metrics is now provided in the cluster logs.
References: See also: Bug #21651536, Bug #21660947, Bug #21661297, Bug #21673318, Bug #21689380, Bug #21809959.
NDB Cluster APIs: Added the
Ndb::setEventBufferQueueEmptyEpoch()method, which makes it possible to enable queuing of empty events (event type
TE_EMPTY). (Bug #22157845)
NDB Cluster APIs:
Ndb_free_list_tis a template used in the implementation of the NDB API to create a list keeping released API objects such as
NdbRecAttr. One drawback to this template is that released API objects are kept in the list for the lifetime of the owning
Ndbobject, such that a transient peak in the demand for any object causes an effective leak in memory that persists until this
Ndbobject itself has been released.
This work adds statistics to each
Ndb_free_listinstance which samples the usage of objects maintained by the list; now, when objects are released, they can be released into the free list, or deallocated, based on the collected usage statistics.
NDBstorage engine supports the MySQL
JSONdata type and MySQL
JSONfunctions implemented in MySQL 5.7.8 and later. This support is subject to the limitation that a single
NDBtable can have at most 3
Made the following enhancements and additions to the
ThreadConfigmultithreaded data node (ndbmtd) configuration parameter:
Added support for non-exclusive CPU locking on FreeBSD and Windows using
Added support for exclusive CPU locking on Solaris, using
cpuset_exclusive, which are added in this release.
Added thread prioritization using
thread_prio, which is added in this release.
thread_priois supported on Linux, FreeBSD, Windows, and Solaris, but the exact effects of this setting are platform-specific; see the documentation for details.
Added support for setting
realtimeon Windows platforms.
For more information, see the description of the
ThreadConfigparameter in the online documentation. (Bug #25830247)
ndb_restore now performs output logging for specific stages of its operation. (Bug #21097957)
An improvement in the hash index implementation used by MySQL NDB Cluster data nodes means that partitions may now contain more than 16 GB of data for fixed columns, and the maximum partition size for fixed column data is now increased to 128 TB. The previous limitation originated with the
DBACCblock in the
NDBkernel using only 32-bit references to the fixed-size part of a row handled in the
DBTUPblock, even though 45-bit references were already in use elsewhere in the kernel outside the
DBACCblock; all such references in
DBACCnow use 45-bit pointers instead.
As part of this work, error messages returned by the
DBACCkernel block that were overly generic have now been improved by making them more specific. (Bug #13844581, Bug #17465232)
A number of changes and improvements were made to the handling of send threads by
NDB. The changes are summarized with brief descriptions in the following list:
Decreased resource requirements for send threads, making sure that a given configuration using send threads outperforms the same configuration without send threads.
Made use of otherwise idle threads (other than receiver threads) as send threads without incurring extra CPU resources in low-load situations.
Improved response time for write transactions.
Provided for handling of bad configuration data in a more graceful manner.
Made it possible to measure CPU usage with improved real-time reporting from data nodes on their resource usage.
As part of implementing the last item of those just listed, a number of new tables providing information about CPU and thread activity by node, thread ID, and thread type have been added to the
ndbinfoinformation database. These tables are listed here:
cpustat: Provides per-second, per-thread CPU statistics
cpustat_50ms: Shows raw per-thread CPU statistics data, gathered every 50ms
cpustat_1sec: Provides raw per-thread CPU statistics data, gathered each second
cpustat_20sec: Displays raw per-thread CPU statistics data, gathered every 20 seconds
threads: Shows names and descriptions of thread types
For more information, see ndbinfo: The NDB Cluster Information Database.
Rewrote the implementation of the
NDBstorage engine's global schema lock functionality to make use of the metadata lock hooks implemented in the MySQL Server in MySQL 5.7.11.
A number of improvements provide additional read scalability for
NDBby making it possible to read tables locally. It is now possible to enable reads from any replica, rather than from the primary replica only. This is disabled by default to remain compatible with previous behavior, but can be enabled for a given SQL node using the
ndb_read_backupsystem variable added in this release.
It also becomes possible to be more flexible about the assignment of partitions by setting a fragment count type. Possible count types are one per node, one per node group, one per Local Data Manager (LDM) per node (this is the previous assignment scheme and still the default), and one per LDM per node group. This setting can be controlled for individual tables by means of a
FRAGMENT_COUNT_TYPEoption embedded in an
This also means that, when restoring table schemas, ndb_restore
--restore_metanow uses the default partitioning for the target cluster, rather than duplicating the partitioning of the cluster from which the backup was taken. This is useful when restoring to a cluster having more data nodes than the original. See Restoring to More Nodes Than the Original, for more information.
Tables using one of the two per-node-group settings for the fragment count type can also be fully replicated. This requires that the table's fragment count type is
ONE_PER_LDM_PER_NODE_GROUP, and can be enabled using the option
FULLY_REPLICATED=1within in an
NDB_TABLEcomment. The option can be enabled by default for all new
NDBtables using the
ndb_fully_replicatedsystem variable added in this release.
Settings for table-level
READ_BACKUPare also supported using the
COMMENT="NDB_TABLE=..."syntax. It is also possible (and often preferable) to set multiple options in one comment within a single
ALTER TABLEstatement. For more information and examples, see Setting NDB_TABLE Options.
This release also introduces the
ndb_data_node_neighboursystem variable, which is intended to be employed with transaction hinting (and fully-replicated tables), and to provide the current SQL node with the ID of a “nearby” data node to use. See the description of this variable in the documentation for more information.
References: See also: Bug #18435416, Bug #11762155, Bug #54717.
Incompatible Change: When the data nodes are only partially connected to the API nodes, a node used for a pushdown join may get its request from a transaction coordinator on a different node, without (yet) being connected to the API node itself. In such cases, the
NodeInfoobject for the requesting API node contained no valid info about the software version of the API node, which caused the
DBSPJblock to assume (incorrectly) when aborting to assume that the API node used
NDBversion 7.2.4 or earlier, requiring the use of a backward compatability mode to be used during query abort which sent a node failure error instead of the real error causing the abort.
Now, whenever this situation occurs, it is assumed that, if the
NDBsoftware version is not yet available, the API node version is greater than 7.2.4. (Bug #23049170)
Important Change: When started with the
--initializeoption, mysqld no longer enables the
NDBCLUSTERstorage engine plugin. This change was needed to prevent attempted initialization of system databases as distributed (rather than as specific to individual SQL nodes), which could result in a metadata lock deadlock. This fix also brings the behavior of
--initializein this regard into line with that of the discontinued
--bootstrapoption, which started a minimal mysqld instance without enabling
NDB. (Bug #22758238)
Performance: A performance problem was found in an internal polling method
do_poll()where the polling client did not check whether it had itself been woken up before completing the poll. Subsequent analysis showed that it is sufficient that only some clients in the polling queue receive data.
do_poll()can then signal these clients and give up its polling rights, even if the maximum specified wait time (10 ms) has not expired.
This change allows
do_poll()to continue polling until either the maximum specified wait time has expired, or the polling client itself has been woken up (by receiving what it was waiting for). This avoids unnecessary thread switches between client threads and thus reduces the associated overhead by as much as 10% in the API client, resulting in a significant performance improvement when client threads perform their own polling. (Bug #81229, Bug #23202735)
macOS: On OS X, ndb_config failed when an empty string was used for the
--hostoption. (Bug #80689, Bug #22908696)
Microsoft Windows: ndb_mgmd failed to start on 32-bit Windows platforms, due to an issue with calling dynamically loaded functions; such issues were also likely to occur with other
ndb_init(). It was found that all of the functions used are already supported in targeted versions of Windows, so this problem is fixed by removing the dynamic loading of these functions and using the versions provided by the Windows header files instead. (Bug #80876, Bug #23014820)
Microsoft Windows: When building MySQL NDB Cluster on Windows using more than one parallel build job it was sometimes possible for the build to fail because host_info.exe could not be installed. To fix this problem, the
install_mcctarget is now always built prior to the
host_infotarget. (Bug #80051, Bug #22566368)
Microsoft Windows: Performing
ANALYZE TABLEon a table having one or more indexes caused ndbmtd to fail with an InvalidAttrInfo error due to signal corruption. This issue occurred consistently on Windows, but could also be encountered on other platforms. (Bug #77716, Bug #21441297)
Solaris: The ndb_print_file utility failed consistently on Solaris 9 for SPARC. (Bug #80096, Bug #22579581)
NDB Disk Data: The following improvements were made to logging during restarts by data nodes using MySQL NDB Cluster Disk Data:
The total amount of undo log to be applied by the data node is now provided as the total number of pages present in the log. This is a worst case estimate.
Progress information is now provided at regular intervals (once for each 30000 records) as the undo log is applied. This information is supplied as the number of records and number of undo log pages applied so far during the current restart.
NDB Cluster APIs: Deletion of Ndb objects used a dispoportionately high amount of CPU. (Bug #22986823)
NDB Cluster APIs: Executing a transaction with an
NdbIndexOperationbased on an obsolete unique index caused the data node process to fail. Now the index is checked in such cases, and if it cannot be used the transaction fails with an appropriate error. (Bug #79494, Bug #22299443)
Reserved send buffer for the loopback transporter, introduced in MySQL NDB Cluster 7.4.8 and used by API and management nodes for administrative signals, was calculated incorrectly. (Bug #23093656, Bug #22016081)
References: This issue is a regression of: Bug #21664515.
During a node restart, re-creation of internal triggers used for verifying the referential integrity of foreign keys was not reliable, because it was possible that not all distributed TC and LDM instances agreed on all trigger identities. To fix this problem, an extra step is added to the node restart sequence, during which the trigger identities are determined by querying the current master node. (Bug #23068914)
References: See also: Bug #23221573.
Following the forced shutdown of one of the 2 data nodes in a cluster where
NoOfReplicas=2, the other data node shut down as well, due to arbitration failure. (Bug #23006431)
CREATE LOGFILE GROUPstatement which had failed due to lack of shared global memory was not performed correctly, causing node failure. In addition, the transaction in which this occurred was not rolled back correctly, also causing any subsequent
CREATE LOGFILE GROUPto fail. (Bug #22982618)
ndbinfo.tc_time_track_statstable uses histogram buckets to give a sense of the distribution of latencies. The sizes of these buckets were also reported as
HISTOGRAM BOUNDARY INFOmessages during data node startup; this printout was redundant and so has been removed. (Bug #22819868)
Online upgrades from previous versions of MySQL NDB Cluster to MySQL NDB Cluster 7.5 were not possible due to missing entries in the matrix used to test upgrade compatibility between versions. (Bug #22024947)
A failure occurred in
DBTUPin debug builds when variable-sized pages for a fragment totalled more than 4 GB. (Bug #21313546)
Restoration of metadata with ndb_restore
-moccasionally failed with the error message Failed to create index... when creating a unique index. While disgnosing this problem, it was found that the internal error PREPARE_SEIZE_ERROR (a temporary error) was reported as an unknown error. Now in such cases, ndb_restore retries the creation of the unique index, and PREPARE_SEIZE_ERROR is reported as NDB Error 748 Busy during read of event table. (Bug #21178339)
References: See also: Bug #22989944.
mysqld did not shut down cleanly when executing
ndb_index_stat. (Bug #21098142)
References: See also: Bug #23343739.
The following improvements were made to the data node error logging mechanism:
Increased the message slot size 499 bytes to 999 bytes to prevent log messages from overwriting one another or from being truncated.
Trace file namefield to the output. This field contains the trace file name (without any path) and trace file number for the thread causing the trace.
ndbmtd trace files are also now shown in the error log.
GETTABINFOREQqueue debugging were enhanced as follows:
Monitoring by a data node of the progress of
GETTABINFOREQsignals can be enabled by setting
ApiVerboseconfiguration parameter, which enables NDB API debug logging for an API node where it is set greater than or equal to 2.
DUMPcode 1229 which shows the current state of the
GETTABINFOREQqueue. (See DUMP 1229.)
See also The DBDICT Block. (Bug #20368450)
References: See also: Bug #20368354.
When a write to the
ndb_binlog_indextable failed during a MySQL Server shutdown, mysqld killed the
NDBbinary logging thread. (Bug #81166, Bug #23142945)
Memory associated with table descriptions was not freed by the internal table information method
NdbDictInterface::parseTableInfo(). (Bug #81141, Bug #23130084)
Improved memory usage by the internal
TransporterFacadeconstructor when performing mutex array initialization. (Bug #81134, Bug #23127824)
Fixed a memory leak that occurred when an error was raised in
ha_ndbcluster::get_metadata()or one of the functions which this method calls. (Bug #81045, Bug #23089566)
An internal function used to validate connections failed to update the connection count when creating a new
Ndbobject. This had the potential to create a new
Ndbobject for every operation validating the connection, which could have an impact on performance, particularly when performing schema operations. (Bug #80750, Bug #22932982)
A table scan on an
NDBtable using neither an ordered index nor any Disk Data columns normally uses an
ACCscan. If this happened while scanning an unique but unordered index which shrank (due to rows being deleted) after the scan started and then grew again (rows inserted), a single row that had been neither deleted nor inserted could be scanned twice. (Bug #80733, Bug #22926938)
Starting a backup in the ndb_mgm client after creating a large number of tables caused a forced shutdown of the cluster. (Bug #80640, Bug #228849958)
When an SQL node was started, and joined the schema distribution protocol, another SQL node, already waiting for a schema change to be distributed, timed out during that wait. This was because the code incorrectly assumed that the new SQL node would also acknowledge the schema distribution even though the new node joined too late to be a participant in it.
As part of this fix, printouts of schema distribution progress now always print the more significant part of a bitmask before the less significant; formatting of bitmasks in such printouts has also been improved. (Bug #80554, Bug #22842538)
The MySQL NDB Cluster Auto-Installer failed to work in various ways on different platforms. (Bug #79853, Bug #22502247)
The internal function
ndbrequire(), which, like
assert(), evaluates a given expression and terminates the process if the expression does not evaluate as true, now includes the failed expression in its output to the error logs. (Bug #77021, Bug #21128318)
Trying to drop a table during an ongoing backup failed with the error message Unknown table; now, it fails with Unable to alter table as backup is in progress. (Bug #47301, Bug #11755512)
References: See also: Bug #44695, Bug #11753280.
Functionality Added or Changed
Important Change: When creating or adding columns to
NDBtables, the default value used for both the
ROW_FORMAToptions is now
This change does not affect the row format or column format used by existing tables. New columns added to such tables use the new defaults, and existing columns are changed to use these as well, provided that the
ALTER TABLEstatement in question uses
ALGORITHM=COPY. Note that this cannot be done implicitly if mysqld is run with
--ndb-allow-copying-alter-table=FALSE(the default is
A new MySQL server option
--ndb-default-column-formatis added for backwards compatibility; set this to
FIXEDto force the old defaults to be used for
This behavior change is superseded in MySQL NDB Cluster 7.5.4.
Scans have been improved by replacing the
DIH_SCAN_GET_NODES_REQsignal, formerly used for communication between the
DBDIHkernel blocks in
NDB, with the
DIGETNODESREQsignal, which supports direct execution and allows for the elimination of
DIH_SCAN_GET_NODES_CONF, as well as for
EXECUTE_DIRECT. This enables enable higher scalability of data nodes when used for scan operations by decreasing the use of CPU resources for scan operations by an estimated five percent in some cases. This should also improve response times, which could help prevent issues with overload of the main threads.
As part of these changes, scans made in the
BACKUPkernel block have also been improved and made more efficient.
References: See also: Bug #80640, Bug #22884995.
The following improvements have been made to event buffer reporting in the cluster log:
Each report now identifies the API node that sent it.
The fields shown in the report have been improved and expanded. Percentages are now better specified and used only when appropriate. For improved clarity, the
latest_epochfields have been renamed to
latest_buffered_epoch, respectively. The ID of the
Ndbobject serving as the source of the report is now shown, as is the reason for making the report (as the
The frequency of unnecessary reporting has been reduced by limiting reports to those showing only significant changes in event buffer usage.
The MGM API adds a new
NDB_LE_EventBufferStatus2event type to handle the additional information provided by the new event buffer reporting. The
NDB_LE_EventBufferStatusevent type used in older versions of MySQL NDB Cluster is now deprecated, and will eventually be removed.
Important Change: The minimum value for the
BackupDataBufferSizedata node configuration parameter has been lowered from 2 MB to 512 KB. The default and maximum values for this parameter remain unchanged. (Bug #22749509)
OS X: Processing of local checkpoints was not handled correctly on Mac OS X, due to an uninitialized variable. (Bug #80236, Bug #22647462)
Microsoft Windows: Compilation of MySQL with Visual Studio 2015 failed in
ConfigInfo.cpp, due to a change in Visual Studio's handling of spaces and concatenation. (Bug #22558836, Bug #80024)
Microsoft Windows: When setting up event logging for ndb_mgmd on Windows, MySQL NDB Cluster tries to add a registry key to
HKEY_LOCAL_MACHINE, which fails if the user does not have access to the registry. In such cases ndb_mgmd logged the error Could neither create or open key, which is not accurate and which can cause confusion for users who may not realize that file logging is available and being used. Now in such cases, ndb_mgmd logs a warning Could not create or access the registry key needed for the application to log to the Windows EventLog. Run the application with sufficient privileges once to create the key, or add the key manually, or turn off logging for that application. An error (as opposed to a warning) is now reported in such cases only if there is no available output at all for ndb_mgmd event logging. (Bug #20960839)
Microsoft Windows: MySQL NDB Cluster did not compile correctly with Microsoft Visual Studio 2015, due to a change from previous versions in the VS implementation of the
_vsnprintf()function. (Bug #80276, Bug #22670525)
During node failure handling, the request structure used to drive the cleanup operation was not maintained correctly when the request was executed. This led to inconsistencies that were harmless during normal operation, but these could lead to assertion failures during node failure handling, with subsequent failure of additional nodes. (Bug #22643129)
The previous fix for a lack of mutex protection for the internal
TransporterFacade::deliver_signal()function was found to be incomplete in some cases. (Bug #22615274)
References: This issue is a regression of: Bug #77225, Bug #21185585.
When setup of the binary log as an atomic operation on one SQL node failed, this could trigger a state in other SQL nodes in which they appeared to detect the SQL node participating in schema change distribution, whereas it had not yet completed binary log setup. This could in turn cause a deadlock on the global metadata lock when the SQL node still retrying binary log setup needed this lock, while another mysqld had taken the lock for itself as part of a schema change operation. In such cases, the second SQL node waited for the first one to act on its schema distribution changes, which it was not yet able to do. (Bug #22494024)
Duplicate key errors could occur when ndb_restore was run on a backup containing a unique index. This was due to the fact that, during restoration of data, the database can pass through one or more inconsistent states prior to completion, such an inconsistent state possibly having duplicate values for a column which has a unique index. (If the restoration of data is preceded by a run with
--disable-indexesand followed by one with
--rebuild-indexes, these errors are avoided.)
Added a check for unique indexes in the backup which is performed only when restoring data, and which does not process tables that have explicitly been excluded. For each unique index found, a warning is now printed. (Bug #22329365)
NdbDictionarymetadata operations had a hard-coded 7-day timeout, which proved to be excessive for short-lived operations such as retrieval of table definitions. This could lead to unnecessary hangs in user applications which were difficult to detect and handle correctly. To help address this issue, timeout behaviour is modified so that read-only or short-duration dictionary interactions have a 2-minute timeout, while schema transactions of potentially long duration retain the existing 7-day timeout.
Such timeouts are intended as a safety net: In the event of problems, these return control to users, who can then take corrective action. Any reproducible issue with
NdbDictionarytimeouts should be reported as a bug. (Bug #20368354)
Optimization of signal sending by buffering and sending them periodically, or when the buffer became full, could cause
SUB_GCP_COMPLETE_ACKsignals to be excessively delayed. Such signals are sent for each node and epoch, with a minimum interval of
TimeBetweenEpochs; if they are not received in time, the
SUMAbuffers can overflow as a result. The overflow caused API nodes to be disconnected, leading to current transactions being aborted due to node failure. This condition made it difficult for long transactions (such as altering a very large table), to be completed. Now in such cases, the
ACKsignal is sent without being delayed. (Bug #18753341)
When setting CPU spin time, the value was needlessly cast to a boolean internally, so that setting it to any nonzero value yielded an effective value of 1. This issue, as well as the fix for it, apply both to setting the
SchedulerSpinTimerparameter and to setting
spintimeas part of a
ThreadConfigparameter value. (Bug #80237, Bug #22647476)
A logic error in an
storage/ndb/src/kernel/blocks/dbacc/DbaccMain.cpprendered useless a check for determining whether
ZREAD_ERRORshould be returned when comparing operations. This was detected when compiling with
-Werror=logical-op. (Bug #80155, Bug #22601798)
References: This issue is a regression of: Bug #21285604.
Suppressed a CMake warning that was caused by use of an incorrectly quoted variable name. (Bug #80066, Bug #22572632)
CREATE INDEXto add an index on either of two
NDBtables sharing circular foreign keys, the query succeeded but a temporary table was left on disk, breaking the foreign key constraints. This issue was also observed when attempting to create an index on a table in the middle of a chain of foreign keys—that is, a table having both parent and child keys, but on different tables. The problem did not occur when using
ALTER TABLEto perform the same index creation operation; and subsequent analysis revealed unintended differences in the way such operations were performed by
To fix this problem, we now make sure that operations performed by a
CREATE INDEXstatement are always handled internally in the same way and at the same time that the same operations are handled when performed by
DROP INDEX. (Bug #79156, Bug #22173891)
PortNumberSCI, SHM, and TCP configuration parameters, which were deprecated in MySQL 5.1.3, have now been removed and are no longer accepted in configuration files.
This change does not affect the
PortNumbermanagement node configuration parameter, whose behavior remains unaltered. (Bug #77405, Bug #21280456)
Functionality Added or Changed
Important Change: Previously, the
NDBscheduler always optimized for speed against throughput in a predetermined manner (this was hard coded); this balance can now be set using the
SchedulerResponsivenessdata node configuration parameter. This parameter accepts an integer in the range of 0-10 inclusive, with 5 as the default. Higher values provide better response times relative to throughput. Lower values provide increased throughput, but impose longer response times. (Bug #78531, Bug #21889312)
Important Change: A number of MySQL NDB Cluster data node configuration parameters were deprecated in earlier versions of MySQL NDB Cluster, and have been removed with this release. These parameters include
Disklessinstead), as well as
DiskCheckpointSpeedInRestart. The archaic and unused
ByteOrdercomputer configuration parameter has also been removed, as well as the unused
MaxNoOfSavedEventsmanagement node confugration parameter. These parameters are no longer supported; most of them already did not have (or no longer had) any effect. Trying to use any of these parameters in a MySQL NDB Cluster configuration file now results in an error.
For more information, see What is New in NDB Cluster 7.5. (Bug #77404, Bug #21280428)
Important Change: The
ndbinfodatabase can now provide default and current information about MySQL NDB Cluster node configuration parameters as a result of the following changes:
config_paramstable has been enhanced with additional columns providing information about each configuration parameter, including its type, default, and maximum and minimum values (where applicable).
config_valuestable has been added. A row in this table shows the current value of a parameter on a given node.
You can obtain values of MySQL NDB Cluster configuration parameters by name using a join on these two tables such as the one shown here:
SELECT p.param_name AS Name, v.node_id AS Node, p.param_type AS Type, p.param_default AS 'Default', v.config_value AS Current FROM config_params p JOIN config_values v ON p.param_number = v.config_param WHERE p. param_name IN ('NodeId', 'HostName','DataMemory', 'IndexMemory');
(Bug #71587, Bug #18183958)
Important Change: The
ExecuteOnComputerconfiguration parameter for management, data, and API nodes is now deprecated, and is subject to removal in a future MySQL NDB Cluster release. For all types of MySQL NDB Cluster nodes, you should now use the
HostNameparameter exclusively for identifying hosts in the cluster configuration file.
Deprecated MySQL NDB Cluster node configuration parameters are now indicated as such by ndb_config
--xml. For each parameter currently deprecated, the corresponding
<param/>tag in the XML output now includes the attribute
deprecated="true". (Bug #21127135)
--ndb-cluster-connection-pool-nodeidsoption for mysqld, which can be used to specify a list of nodes by node ID for connection pooling. The number of node IDs in the list must equal the value set for
--ndb-cluster-connection-pool. (Bug #19521789)
PROMPTcommand in the ndb_mgm client. This command has the syntax
PROMPT, which sets the client's prompt to
string. Issuing the command without an argument causes the prompt to be reset to the default (
ndb_mgm>). See Commands in the NDB Cluster Management Client, for more information. (Bug #18421338)
--databaseoption has not been specified for ndb_show_tables, and no tables are found in the
TEST_DBdatabase, an appropriate warning message is now issued. (Bug #50633, Bug #11758430)
NDBstorage engine now uses the improved records-per-key interface for index statistics introduced for the optimizer in MySQL 5.7. Some improvements due to this change are listed here:
The optimizer can now choose better execution plans for queries on
NDBtables in many cases where a less optimal join index or table join order would previously have been chosen.
EXPLAINnow provides more accurate row estimates than previously.
Improved cardinality estimates can be obtained from
Incompatible Change; NDB Cluster APIs: The
pollEvents2()method now returns -1, indicating an error, whenever a negative value is used for the time argument. (Bug #20762291)
Important Change; NDB Cluster APIs:
Ndb::pollEvents()is now compatible with the
TE_OUT_OF_MEMORYevent types introduced in MySQL NDB Cluster 7.4.3. For detailed information about this change, see the description of this method in the MySQL NDB Cluster API Developer Guide. (Bug #20646496)
The behavior of
Ndb::pollEvents()has also been modified such that it now returns NDB_FAILURE_GCI (equal to
~(Uint64) 0) when a cluster failure has been detected. (Bug #18753887)
Important Change; NDB Cluster APIs: To release the memory used for dropped event operations, the event API formerly depended on
nextEvent()to consume all events possibly referring to the dropped events. This dependency between
dropEventOperation()and the first two methods required the entire event buffer to be read before attempting to release event operation memory (that is, until successive calls to
nextEvent()returned no more events).
A related cleanup issue arose following the reset of the event buffer (when all event operations had previously been dropped), and the event buffer was truncated by the first
createEventOperation()call subsequent to the reset.
To fix these problems, the event buffer is now cleared when the last event operation is dropped, rather than waiting for a subsequent create operation which might or might not occur. Memory taken up by dropped event operations is also now released when the event queue has been cleared, which removes the hidden requirement for consuming all events to free up memory. In addition, event operation memory is now released as soon as all events referring to the operation have been consumed, rather than waiting for the entire event buffer to be consumed. (Bug #78145, Bug #21661297)
Important Change; NDB Cluster APIs: The MGM API error-handling functions
ndb_mgm_get_latest_error_desc()each failed when used with a
NULLhandle. You should note that, although these functions are now null-safe, values returned in this case are arbitrary and not meaningful. (Bug #78130, Bug #21651706)
Important Change; NDB Cluster APIs: The following NDB API methods were not actually implemented and have been removed from the sources:
Important Change: The options controlling behavior of
NDBprograms with regard to the number and timing of successive attempts to connect to a management server have changed as listed here:
The minimum value for the
--connect-retry-delayoption common to all
NDBprograms has been changed from 0 to 1; this means that all
NDBprograms now wait at least 1 second between successive connection attempts, and it is no longer possible to set a waiting time equal to 0.
The semantics for the
--connect-retriesoption have changed slightly, such that the value of this option now sets the number of times an
NDBprogram tries to connect to a management server. Setting this option to 0 now causes the program to attempt the connection indefinitely, until it either succeeds or is terminated by other means (such as kill).
In addition, the default for the
--connect-retriesoption for the ndb_mgm client has been changed from 3 to 12, so that the minimum, maximum, and default values for this option when used with ndb_mgm are now exactly the same as for all other
--try-reconnectoption, although deprecated in MySQL NDB Cluster 7.4, continues to be supported as a synonym for ndb_mgm
--connect-retriesto provide backwards compatibility. The default value for
--try-reconnecthas also been changed from 3 to 12, respectively, so that this option continues to behave in the exactly in the same way as
Important Change: In previous versions of MySQL NDB Cluster, other DDL operations could not be part of
ALTER ONLINE TABLE ... RENAME .... (This was disallowed by the fix for BUG#16021021.) MySQL NDB Cluster 7.5 makes the following changes:
Support for the
OFFLINEkeywords, which was deprecated in MySQL NDB Cluster 7.3, is now removed, and use of these now causes a syntax error; the
NDBstorage engine now accepts only
ALGORITHM = DEFAULT,
ALGORITHM = COPY, and
ALGORITHM = INPLACEto specify whether the
ALTERoperation is copying or in-place, just as in the standard MySQL Server.
ALTER TABLE ... ALGORITHM=COPYING RENAME.
(Bug #20804269, Bug #76543, Bug #20479917, Bug #75797)
References: See also: Bug #16021021.
NDB Disk Data: A unique index on a column of an
NDBtable is implemented with an associated internal ordered index, used for scanning. While dropping an index, this ordered index was dropped first, followed by the drop of the unique index itself. This meant that, when the drop was rejected due to (for example) a constraint violation, the statement was rejected but the associated ordered index remained deleted, so that any subsequent operation using a scan on this table failed. We fix this problem by causing the unique index to be removed first, before removing the ordered index; removal of the related ordered index is no longer performed when removal of a unique index fails. (Bug #78306, Bug #21777589)
NDB Cluster APIs: The binary log injector did not work correctly with
TE_INCONSISTENTevent type handling by
Ndb::nextEvent(). (Bug #22135541)
References: See also: Bug #20646496.
NDB Cluster APIs: While executing
dropEvent(), if the coordinator
DBDICTfailed after the subscription manager (
SUMAblock) had removed all subscriptions but before the coordinator had deleted the event from the system table, the dropped event remained in the table, causing any subsequent drop or create event with the same name to fail with
NDBerror 1419 Subscription already dropped or error 746 Event name already exists. This occurred even when calling
dropEvent()with a nonzero force argument.
Now in such cases, error 1419 is ignored, and
DBDICTdeletes the event from the table. (Bug #21554676)
NDB Cluster APIs: Creation and destruction of
Ndb_cluster_connectionobjects by multiple threads could make use of the same application lock, which in some cases led to failures in the global dictionary cache. To alleviate this problem, the creation and destruction of several internal NDB API objects have been serialized. (Bug #20636124)
NDB Cluster APIs: When an
Ndbobject created prior to a failure of the cluster was reused, the event queue of this object could still contain data node events originating from before the failure. These events could reference “old” epochs (from before the failure occurred), which in turn could violate the assumption made by the
nextEvent()method that epoch numbers always increase. This issue is addressed by explicitly clearing the event queue in such cases. (Bug #18411034)
References: See also: Bug #20888668.
NDB Cluster APIs:
pollEvents2()were slow to receive events, being dependent on other client threads or blocks to perform polling of transporters on their behalf. This fix allows a client thread to perform its own transporter polling when it has to wait in either of these methods.
Introduction of transporter polling also revealed a problem with missing mutex protection in the
ndbcluster_binloghandler, which has been added as part of this fix. (Bug #79311, Bug #20957068, Bug #22224571)
NDB Cluster APIs: After the initial restart of a node following a cluster failure, the cluster failure event added as part of the restart process was deleted when an event that existed prior to the restart was later deleted. This meant that, in such cases, an Event API client had no way of knowing that failure handling was needed. In addition, the GCI used for the final cleanup of deleted event operations, performed by
nextEvent()when these methods have consumed all available events, was lost. (Bug #78143, Bug #21660947)
A serious regression was inadvertently introduced in MySQL NDB Cluster 7.4.8 whereby local checkpoints and thus restarts often took much longer than expected. This occurred due to the fact that the setting for
MaxDiskWriteSpeedOwnRestartwas ignored during restarts and the value of
MaxDiskWriteSpeedOtherNodeRestart, which is much lower by default than the default for
MaxDiskWriteSpeedOwnRestart, was used instead. This issue affected restart times and performance only and did not have any impact on normal operations. (Bug #22582233)
The epoch for the latest restorable checkpoint provided in the cluster log as part of its reporting for
EventBufferStatusevents (see NDB Cluster: Messages in the Cluster Log) was not well defined and thus unreliable; depending on various factors, the reported epoch could be the one currently being consumed, the one most recently consumed, or the next one queued for consumption.
This fix ensures that the latest restorable global checkpoint is always regarded as the one that was most recently completely consumed by the user, and thus that it was the latest restorable global checkpoint that existed at the time the report was generated. (Bug #22378288)
--ndb-allow-copying-alter-tableoption for mysqld. Setting this option (or the equivalent system variable
ALTER TABLEstatements from performing copying operations. The default value is
ON. (Bug #22187649)
References: See also: Bug #17400320.
Attempting to create an
NDBtable having greater than the maximum supported combined width for all
BITcolumns (4096) caused data node failure when these columns were defined with
COLUMN_FORMAT DYNAMIC. (Bug #21889267)
Creating a table with the maxmimum supported number of columns (512) all using
COLUMN_FORMAT DYNAMICled to data node failures. (Bug #21863798)
In a MySQL NDB Cluster with multiple LDM instances, all instances wrote to the node log, even inactive instances on other nodes. During restarts, this caused the log to be filled with messages from other nodes, such as the messages shown here:
2015-06-24 00:20:16 [ndbd] INFO -- We are adjusting Max Disk Write Speed, a restart is ongoing now ... 2015-06-24 01:08:02 [ndbd] INFO -- We are adjusting Max Disk Write Speed, no restarts ongoing anymore
Now this logging is performed only by the active LDM instance. (Bug #21362380)
Backup block states were reported incorrectly during backups. (Bug #21360188)
References: See also: Bug #20204854, Bug #21372136.
For a timeout in
GET_TABINFOREQwhile executing a
CREATE INDEXstatement, mysqld returned Error 4243 (Index not found) instead of the expected Error 4008 (Receive from NDB failed).
The fix for this bug also fixes similar timeout issues for a number of other signals that are sent the
DBDICTkernel block as part of DDL operations, including
LIST_TABLES_REQ, as well as several internal functions used in handling
NDBschema operations. (Bug #21277472)
References: See also: Bug #20617891, Bug #20368354, Bug #19821115.
Previously, multiple send threads could be invoked for handling sends to the same node; these threads then competed for the same send lock. While the send lock blocked the additional send threads, work threads could be passed to other nodes.
This issue is fixed by ensuring that new send threads are not activated while there is already an active send thread assigned to the same node. In addition, a node already having an active send thread assigned to it is no longer visible to other, already active, send threads; that is, such a node is longer added to the node list when a send thread is currently assigned to it. (Bug #20954804, Bug #76821)
Queueing of pending operations when the redo log was overloaded (
DefaultOperationRedoProblemActionAPI node configuration parameter) could lead to timeouts when data nodes ran out of redo log space (P_TAIL_PROBLEM errors). Now when the redo log is full, the node aborts requests instead of queuing them. (Bug #20782580)
References: See also: Bug #20481140.
NDBevent buffer can be used with an
Ndbobject to subscribe to table-level row change event streams. Users subscribe to an existing event; this causes the data nodes to start sending event data signals (
SUB_TABLE_DATA) and epoch completion signals (
SUB_GCP_COMPLETE) to the
SUB_GCP_COMPLETE_REPsignals can arrive for execution in concurrent receiver thread before completion of the internal method call used to start a subscription.
SUB_GCP_COMPLETE_REPsignals depends on the total number of
SUMAbuckets (sub data streams), but this may not yet have been set, leading to the present issue, when the counter used for tracking the
TOTAL_BUCKETS_INIT) was found to be set to erroneous values. Now
TOTAL_BUCKETS_INITis tested to be sure it has been set correctly before it is used. (Bug #20575424, Bug #76255)
References: See also: Bug #20561446, Bug #21616263.
NDBstatistics queries could be delayed by the error delay set for
ndb_index_stat_option(default 60 seconds) when the index that was queried had been marked with internal error. The same underlying issue could also cause
ANALYZE TABLEto hang when executed against an
NDBtable having multiple indexes where an internal error occured on one or more but not all indexes.
Now in such cases, any existing statistics are returned immediately, without waiting for any additonal statistics to be discovered. (Bug #20553313, Bug #20707694, Bug #76325)
Memory allocated when obtaining a list of tables or databases was not freed afterward. (Bug #20234681, Bug #74510)
References: See also: Bug #18592390, Bug #72322.
BackupDiskWriteSpeedPctdata node parameter. Setting this parameter causes the data node to reserve a percentage of its maximum write speed (as determined by the value of
MaxDiskWriteSpeed) for use in local checkpoints while performing a backup.
BackupDiskWriteSpeedPctis interpreted as a percentage which can be set between 0 and 90 inclusive, with a default value of 50. (Bug #20204854)
References: See also: Bug #21372136.
After restoring the database schema from backup using ndb_restore, auto-discovery of restored tables in transactions having multiple statements did not work correctly, resulting in Deadlock found when trying to get lock; try restarting transaction errors.
This issue was encountered both in the mysql client, as well as when such transactions were executed by application programs using Connector/J and possibly other MySQL APIs.
Prior to upgrading, this issue can be worked around by executing
SELECT TABLE_NAME, TABLE_SCHEMA FROM INFORMATION_SCHEMA.TABLES WHERE ENGINE = 'NDBCLUSTER'on all SQL nodes following the restore operation, before executing any other statements. (Bug #18075170)
STOP -fto force a node shutdown even when it triggered a complete shutdown of the cluster, it was possible to lose data when a sufficient number of nodes were shut down, triggering a cluster shutodwn, and the timing was such that
SUMAhandovers had been made to nodes already in the process of shutting down. (Bug #17772138)
When using a sufficiently large value for
TransactionDeadlockDetectionTimeoutand the default value for
ORDER BY transidwith multiple concurrent conflicting or deadlocked transactions, each transaction having several pending operations, caused the SQL node where the query was run to fail. (Bug #16731538, Bug #67596)
ndbinfo.config_paramstable is now read-only. (Bug #11762750, Bug #55383)
NDBfailed during a node restart due to the status of the current local checkpoint being set but not as active, even though it could have other states under such conditions. (Bug #78780, Bug #21973758)
ndbmtd checked for signals being sent only after a full cycle in
run_job_buffers, which is performed for all job buffer inputs. Now this is done as part of
run_job_buffersitself, which avoids executing for extended periods of time without sending to other nodes or flushing signals to other threads. (Bug #78530, Bug #21889088)
When attempting to enable index statistics, creation of the required system tables, events and event subscriptions often fails when multiple mysqld processes using index statistics are started concurrently in conjunction with starting, restarting, or stopping the cluster, or with node failure handling. This is normally recoverable, since the affected mysqld process or processes can (and do) retry these operations shortly thereafter. For this reason, such failures are no longer logged as warnings, but merely as informational events. (Bug #77760, Bug #21462846)
It was possible to end up with a lock on the send buffer mutex when send buffers became a limiting resource, due either to insufficient send buffer resource configuration, problems with slow or failing communications such that all send buffers became exhausted, or slow receivers failing to consume what was sent. In this situation worker threads failed to allocate send buffer memory for signals, and attempted to force a send in order to free up space, while at the same time the send thread was busy trying to send to the same node or nodes. All of these threads competed for taking the send buffer mutex, which resulted in the lock already described, reported by the watchdog as
Stuck in Send. This fix is made in two parts, listed here:
The send thread no longer holds the global send thread mutex while getting the send buffer mutex; it now releases the global mutex prior to locking the send buffer mutex. This keeps worker threads from getting stuck in send in such cases.
Locking of the send buffer mutex done by the send threads now uses a try-lock. If the try-lock fails, the node to make the send to is reinserted at the end of the list of send nodes in order to be retried later. This removes the
Stuck in Sendcondition for the send threads.
(Bug #77081, Bug #21109605)