NDB Cluster APIs; ndbinfo Information Database: Added two tables to the
ndbinfoinformation database. The
config_nodestable provides information about nodes that are configured as part of a given NDB Cluster, such as node ID and process type. The
processestable shows information about nodes currently connected to the cluster; this information includes the process name and system process ID, and service address. For each data node and SQL node, it also shows the process ID of the node's angel process.
As part of the work done to implement the
processestable, a new
set_service_uri()method has been added to the NDB API.
NDB Cluster APIs: The system name of an NDB cluster is now visible in the mysql client as the value of the
Ndb_system_namestatus variable, and can also be obtained by NDB API application using the
Ndb_cluster_connection::get_system_name()method. The system name can be set using the
Nameparameter in the
[system]section of the cluster configuration file. (WL #10321)
--diff-defaultoption for ndb_config. This option causes the program to print only those parameters having values that differ from their defaults. (Bug #85831, Bug #25844166)
--query-alloption to ndb_config. This option acts much like the
--queryoption except that
-a) dumps configuration information for all attributes at one time. (Bug #60095, Bug #11766869)
Packaging: Two missing dependencies were added to the
The data node package requires
The auto-installer requires
(Bug #85679, Bug #25799465)
NDB Cluster APIs: The implementation method
NdbDictionary::NdbTableImpl::getColumn(), used from many places in the NDB API where a column is referenced by name, has been made more efficient. This method used a linear search of an array of columns to find the correct column object, which could be inefficient for tables with many columns, and was detected as a significant use of CPU in customer applications. (Ideally, users should perform name-to-column object mapping, and then use column IDs or objects in method calls, but in practice this is not always done.) A less costly hash index implementation, used previously for the name lookup, is reinstated for tables having relatively many columns. (A linear search continues to be used for tables having fewer columns, where the difference in performance is neglible.) (Bug #24829435)
.logfiles contained log entries for one or more extra fragments, due to an issue with filtering out changes logged by other nodes in the same node group. This resulted in a larger
.logfile and thus use of more resources than necessary; it could also cause problems when restoring, since backups from different nodes could interfere with one another while the log was being applied. (Bug #25891014)
When making the final write to a redo log file, it is expected that the next log file is already opened for writes, but this was not always the case with a slow disk, leading to node failure. Now in such cases
NDBwaits for the next file to be opened properly before attempting to write to it. (Bug #25806659)
Data node threads can be bound to a single CPU or a set of CPUs, a set of CPUs being represented internally by
SparseBitmask. When attempting to lock to a set of CPUs, CPU usage was excessive due to the fact that the routine performing the locks used the
mt_thr_config.cpp::do_bind()method, which looks for bits that are set over the entire theoretical range of the
SparseBitmask(232-2, or 4294967294). This is fixed by using
SparseBitmask::getBitNo(), which can be used to iterate over only those bits that are actually set, instead. (Bug #25799506)
ndb_report_thresh_binlog_epoch_slipwas enabled, an event buffer status message with
report_reason=LOW/ENOUGH_FREE_EVENTBUFFERwas printed in the logs when event buffer usage was high and then decreased to a lower level. This calculation was based on total allocated event buffer memory rather than the limit set by
ndb_eventbuffer_max_alloc; it was also printed even when the event buffer had unlimited memory (
ndb_eventbuffer_max_alloc= 0, the default), which could confuse users.
This is fixed as follows:
The calculation of
ndb_eventbuffer_free_percentis now based on
ndb_eventbuffer_max_alloc, rather than the amount actually allocated.
ndb_eventbuffer_free_percentis set and
ndb_eventbuffer_max_allocis equal to 0, event buffer status messages using
report_reason=LOW/ENOUGH_FREE_EVENTBUFFERare no longer printed.
ndb_report_thresh_binlog_epoch_slipis set, an event buffer status message showing
report_reason=BUFFERED_EPOCHS_OVER_THRESHOLDis written each 10 seconds (rather than every second) whenever this is greater than the threshold.
A bulk update is executed by reading records and executing a transaction on the set of records, which is started while reading them. When transaction initialization failed, the transaction executor function was subsequently unaware that this had occurred, leading to SQL node failures. This issue is fixed by providing appropriate error handling when attempting to initialize the transaction. (Bug #25476474)
References: See also: Bug #20092754.
NoOfFragmentLogPartssuch that there were more than 4 redo log parts per local data manager led to resource exhaustion and subsequent multiple data node failures. Since this is an invalid configuration, a check has been added to detect a configuration with more than 4 redo log parts per LDM, and reject it as invalid. (Bug #25333414)
Execution of an online
ALTER TABLE ... REORGANIZE PARTITIONstatement on an
NDBtable having a primary key whose length was greater than 80 bytes led to restarting of data nodes, causing the reorganization to fail. (Bug #25152165)
In certain cases, a failed
ALTER TABLE ... ADD UNIQUE KEYstatement could lead to SQL node failure. (Bug #24444878)
References: This issue is a regression of: Bug #23089566.
Error 240 is raised when there is a mismatch between foreign key trigger columns and the values supplied to them during trigger execution, but had no error message indicating the source of the problem. (Bug #23141739)
References: See also: Bug #23068914, Bug #85857.
If the number of LDM blocks was not evenly divisible by the number of TC/SPJ blocks, SPJ requests were not equally distributed over the available SPJ instances. Now a round-robin distribution is used to distribute SPJ requests across all available SPJ instances more effectively.
As part of this work, a number of unused member variables have been removed from the class
Dbtc. (Bug #22627519)
ALTER TABLE .. MAX_ROWS=0can now be performed only by using a copying
ALTER TABLEstatement. Resetting
MAX_ROWSto 0 can no longer be performed using
ALGORITHM=INPLACE. (Bug #21960004)
During a system restart, when a node failed due to having missed sending heartbeats, all other nodes reported only that another node had failed without any additional information. Now in such cases, the fact that heartbeats were missed and the ID of the node that failed to send heartbeats is reported in both the error log and the data node log. (Bug #21576576)
The planned shutdown of an NDB Cluster having more than 10 data nodes was not always performed gracefully. (Bug #20607730)
Due to a previous issue with unclear separation between the optimize and execute phases when a query involved a
GROUP BY, the join-pushable evaluator was not sure whether its optimized query execution plan was in fact pushable. For this reason, such grouped joins were always considered not pushable. It has been determined that the separation issue has been resolved by work already done in MySQL 5.6, and so we now remove this limitation. (Bug #86623, Bug #26239591)
When deleting all rows from a table immediately followed by
DROP TABLE, it was possible that the shrinking of the
DBACChash index was not ready prior to the drop. This shrinking is a per-fragment operation that does not check the state of the table. When a table is dropped,
DBACCreleases resources, during which the description of the fragment size and page directory is not consistent; this could lead to reads of stale pages, and undefined behavior.
Inserting a great many rows followed by dropping the table should also have had such effects due to expansion of the hash index.
To fix this problem we make sure, when a fragment is about to be released, that there are no pending expansion or shrinkage operations on this fragment. (Bug #86449, Bug #26138592)
The internal function
mt.cppread three section pointers from the signal even when none was passed to it. This was mostly harmless, although unneeded. When the signal read was the last one on the last page in the job buffer, and the next page in memory was not mapped or otherwise accessible, ndbmtd failed with an error. To keep this from occurring, this function now only reads section pointers that are actually passed to it. (Bug #86354, Bug #26092639)
The ndb_show_tables program
--unqualifiedoption did not work correctly when set to 0 (false); this should disable the option and so cause fully qualified table and index names to be printed in the output. (Bug #86017, Bug #25923164)
NDBtable with foreign key constraints is created, its indexes are created first, and then, during foreign key creation, these indexes are loaded into the
NDBdictionary cache. When a
CREATE TABLEstatement failed due to an issue relating to foreign keys, the indexes already in the cache were not invalidated. This meant that any subsequent
CREATE TABLEwith any indexes having the same names as those in the failed statement produced inconsistent results. Now, in such cases, any indexes named in the failed
CREATE TABLEare immediately invalidated from the cache. (Bug #85917, Bug #25882950)
Attempting to execute
ALTER TABLE ... ADD FOREIGN KEYwhen the key to be added had the name of an existing foreign key on the same table failed with the wrong error message. (Bug #85857, Bug #23068914)
The node internal scheduler (in
mt.cpp) collects statistics about its own progress and any outstanding work it is performing. One such statistic is the number of outstanding send bytes, collected in
send_buffer::m_node_total_send_buffer_size. This information may later be used by the send thread scheduler, which uses it as a metric to tune its own send performance versus latency.
In order to reduce lock contention on the internal send buffers, they are split into two
m_sending, each protected by its own mutex, and their combined size repesented by
Investigation of the code revealed that there was no consistency as to which mutex was used to update
m_node_total_send_buffer_size, with the result that there was no consurrency protection for this value. To avoid this,
m_node_total_send_buffer_sizeis replaced with two values,
m_sending_size, which keep separate track of the sizes of the two buffers. These counters are updated under the protection of two different mutexes protecting each buffer individually, and are now added together to obtain the total size.
With concurrency control established, updates of the partial counts should now be correct, so that their combined value no longer accumulates errors over time. (Bug #85687, Bug #25800933)
TRANS_AIsignals that used the long signal format were not handled by the
DBTCkernel block. (Bug #85606, Bug #25777337)
References: See also: Bug #85519, Bug #27540805.
To prevent a scan from returning more rows, bytes, or both than the client has reserved buffers for, the
DBTUPkernel block reports the size of the
TRANSID_AIit has sent to the client in the
TUPKEYCONFsignal it sends to the requesting
DBLQHis aware of the maximum batch size available for the result set, and terminates the scan batch if this has been exceeded.
DBTUPto produce two
TRANSID_AIresults from the same row, one for the client, and one for
DBSPJ, which is needed for key lookups on the joined tables. The size of both of these were added to the read length reported by the
DBTUPblock, which caused the controlling
DBLQHblock to believe that it had consumed more of the available maximum batch size than was actually the case, leading to premature termination of the scan batch which could have a negative impact on performance of SPJ scans. To correct this, only the actual read length part of an API request is now reported in such cases. (Bug #85408, Bug #25702850)
Data node binaries for Solaris 11 built using Oracle Developer Studio 12.5 on SPARC platforms failed with bus errors. (Bug #85390, Bug #25695818)
When compiling the NDB kernel with gcc version 6.0.0 or later, it is now built using
-flifetime-dse=1. (Bug #85381, Bug #25690926)