Documentation Home
MySQL NDB Cluster 7.5
Related Documentation Download this Excerpt
PDF (US Ltr) - 4.1Mb
PDF (A4) - 4.1Mb


MySQL NDB Cluster 7.5  /  ...  /  NDB Cluster: Messages in the Cluster Log

6.2.1 NDB Cluster: Messages in the Cluster Log

The following table lists the most common NDB cluster log messages. For information about the cluster log, log events, and event types, see Section 6.3, “Event Reports Generated in NDB Cluster”. These log messages also correspond to log event types in the MGM API; see The Ndb_logevent_type Type, for related information of interest to Cluster API developers.

Table 6.1 Common NDB cluster log messages

Log Message Description Event Name Event Type Priority Severity
Node mgm_node_id: Node data_node_id Connected The data node having node ID node_id has connected to the management server (node mgm_node_id). Connected Connection 8 INFO
Node mgm_node_id: Node data_node_id Disconnected The data node having node ID data_node_id has disconnected from the management server (node mgm_node_id). Disconnected Connection 8 ALERT
Node data_node_id: Communication to Node api_node_id closed The API node or SQL node having node ID api_node_id is no longer communicating with data node data_node_id. CommunicationClosed Connection 8 INFO
Node data_node_id: Communication to Node api_node_id opened The API node or SQL node having node ID api_node_id is now communicating with data node data_node_id. CommunicationOpened Connection 8 INFO
Node mgm_node_id: Node api_node_id: API version The API node having node ID api_node_id has connected to management node mgm_node_id using NDB API version version (generally the same as the MySQL version number). ConnectedApiVersion Connection 8 INFO
Node node_id: Global checkpoint gci started A global checkpoint with the ID gci has been started; node node_id is the master responsible for this global checkpoint. GlobalCheckpointStarted Checkpoint 9 INFO
Node node_id: Global checkpoint gci completed The global checkpoint having the ID gci has been completed; node node_id was the master responsible for this global checkpoint. GlobalCheckpointCompleted Checkpoint 10 INFO
Node node_id: Local checkpoint lcp started. Keep GCI = current_gci oldest restorable GCI = old_gci The local checkpoint having sequence ID lcp has been started on node node_id. The most recent GCI that can be used has the index current_gci, and the oldest GCI from which the cluster can be restored has the index old_gci. LocalCheckpointStarted Checkpoint 7 INFO
Node node_id: Local checkpoint lcp completed The local checkpoint having sequence ID lcp on node node_id has been completed. LocalCheckpointCompleted Checkpoint 8 INFO
Node node_id: Local Checkpoint stopped in CALCULATED_KEEP_GCI The node was unable to determine the most recent usable GCI. LCPStoppedInCalcKeepGci Checkpoint 0 ALERT
Node node_id: Table ID = table_id, fragment ID = fragment_id has completed LCP on Node node_id maxGciStarted: started_gci maxGciCompleted: completed_gci A table fragment has been checkpointed to disk on node node_id. The GCI in progress has the index started_gci, and the most recent GCI to have been completed has the index completed_gci. LCPFragmentCompleted Checkpoint 11 INFO
Node node_id: ACC Blocked num_1 and TUP Blocked num_2 times last second Undo logging is blocked because the log buffer is close to overflowing. UndoLogBlocked Checkpoint 7 INFO
Node node_id: Start initiated version Data node node_id, running NDB version version, is beginning its startup process. NDBStartStarted StartUp 1 INFO
Node node_id: Started version Data node node_id, running NDB version version, has started successfully. NDBStartCompleted StartUp 1 INFO
Node node_id: STTORRY received after restart finished The node has received a signal indicating that a cluster restart has completed. STTORRYRecieved StartUp 15 INFO
Node node_id: Start phase phase completed (type) The node has completed start phase phase of a type start. For a listing of start phases, see Section 6.4, “Summary of NDB Cluster Start Phases”. (type is one of initial, system, node, initial node, or <Unknown>.) StartPhaseCompleted StartUp 4 INFO
Node node_id: CM_REGCONF president = president_id, own Node = own_id, our dynamic id = dynamic_id Node president_id has been selected as president. own_id and dynamic_id should always be the same as the ID (node_id) of the reporting node. CM_REGCONF StartUp 3 INFO
Node node_id: CM_REGREF from Node president_id to our Node node_id. Cause = cause The reporting node (ID node_id) was unable to accept node president_id as president. The cause of the problem is given as one of Busy, Election with wait = false, Not president, Election without selecting new candidate, or No such cause. CM_REGREF StartUp 8 INFO
Node node_id: We are Node own_id with dynamic ID dynamic_id, our left neighbor is Node id_1, our right is Node id_2 The node has discovered its neighboring nodes in the cluster (node id_1 and node id_2). node_id, own_id, and dynamic_id should always be the same; if they are not, this indicates a serious misconfiguration of the cluster nodes. FIND_NEIGHBOURS StartUp 8 INFO
Node node_id: type shutdown initiated The node has received a shutdown signal. The type of shutdown is either Cluster or Node. NDBStopStarted StartUp 1 INFO
Node node_id: Node shutdown completed [, action] [Initiated by signal signal.] The node has been shut down. This report may include an action, which if present is one of restarting, no start, or initial. The report may also include a reference to an NDB Protocol signal; for possible signals, refer to Operations and Signals. NDBStopCompleted StartUp 1 INFO
Node node_id: Forced node shutdown completed [, action]. [Occurred during startphase start_phase.] [ Initiated by signal.] [Caused by error error_code: 'error_message(error_classification). error_status'. [(extra info extra_code)]] The node has been forcibly shut down. The action (one of restarting, no start, or initial) subsequently being taken, if any, is also reported. If the shutdown occurred while the node was starting, the report includes the start_phase during which the node failed. If this was a result of a signal sent to the node, this information is also provided (see Operations and Signals, for more information). If the error causing the failure is known, this is also included; for more information about NDB error messages and classifications, see NDB Cluster API Errors. NDBStopForced StartUp 1 ALERT
Node node_id: Node shutdown aborted The node shutdown process was aborted by the user. NDBStopAborted StartUp 1 INFO
Node node_id: StartLog: [GCI Keep: keep_pos LastCompleted: last_pos NewestRestorable: restore_pos] This reports global checkpoints referenced during a node start. The redo log prior to keep_pos is dropped. last_pos is the last global checkpoint in which data node the participated; restore_pos is the global checkpoint which is actually used to restore all data nodes. StartREDOLog StartUp 4 INFO
startup_message [Listed separately; see below.] There are a number of possible startup messages that can be logged under different circumstances. These are listed separately; see Section 6.2.2, “NDB Cluster Log Startup Messages”. StartReport StartUp 4 INFO
Node node_id: Node restart completed copy of dictionary information Copying of data dictionary information to the restarted node has been completed. NR_CopyDict NodeRestart 8 INFO
Node node_id: Node restart completed copy of distribution information Copying of data distribution information to the restarted node has been completed. NR_CopyDistr NodeRestart 8 INFO
Node node_id: Node restart starting to copy the fragments to Node node_id Copy of fragments to starting data node node_id has begun NR_CopyFragsStarted NodeRestart 8 INFO
Node node_id: Table ID = table_id, fragment ID = fragment_id have been copied to Node node_id Fragment fragment_id from table table_id has been copied to data node node_id NR_CopyFragDone NodeRestart 10 INFO
Node node_id: Node restart completed copying the fragments to Node node_id Copying of all table fragments to restarting data node node_id has been completed NR_CopyFragsCompleted NodeRestart 8 INFO
Node node_id: Node node1_id completed failure of Node node2_id Data node node1_id has detected the failure of data node node2_id NodeFailCompleted NodeRestart 8 ALERT
All nodes completed failure of Node node_id All (remaining) data nodes have detected the failure of data node node_id NodeFailCompleted NodeRestart 8 ALERT
Node failure of node_idblock completed The failure of data node node_id has been detected in the blockNDB kernel block, where block is 1 of DBTC, DBDICT, DBDIH, or DBLQH; for more information, see NDB Kernel Blocks NodeFailCompleted NodeRestart 8 ALERT
Node mgm_node_id: Node data_node_id has failed. The Node state at failure was state_code A data node has failed. Its state at the time of failure is described by an arbitration state code state_code: possible state code values can be found in the file include/kernel/signaldata/ArbitSignalData.hpp. NODE_FAILREP NodeRestart 8 ALERT
President restarts arbitration thread [state=state_code] or Prepare arbitrator node node_id [ticket=ticket_id] or Receive arbitrator node node_id [ticket=ticket_id] or Started arbitrator node node_id [ticket=ticket_id] or Lost arbitrator node node_id - process failure [state=state_code] or Lost arbitrator node node_id - process exit [state=state_code] or Lost arbitrator node node_id - error_message [state=state_code] This is a report on the current state and progress of arbitration in the cluster. node_id is the node ID of the management node or SQL node selected as the arbitrator. state_code is an arbitration state code, as found in include/kernel/signaldata/ArbitSignalData.hpp. When an error has occurred, an error_message, also defined in ArbitSignalData.hpp, is provided. ticket_id is a unique identifier handed out by the arbitrator when it is selected to all the nodes that participated in its selection; this is used to ensure that each node requesting arbitration was one of the nodes that took part in the selection process. ArbitState NodeRestart 6 INFO
Arbitration check lost - less than 1/2 nodes left or Arbitration check won - all node groups and more than 1/2 nodes left or Arbitration check won - node group majority or Arbitration check lost - missing node group or Network partitioning - arbitration required or Arbitration won - positive reply from node node_id or Arbitration lost - negative reply from node node_id or Network partitioning - no arbitrator available or Network partitioning - no arbitrator configured or Arbitration failure - error_message [state=state_code] This message reports on the result of arbitration. In the event of arbitration failure, an error_message and an arbitration state_code are provided; definitions for both of these are found in include/kernel/signaldata/ArbitSignalData.hpp. ArbitResult NodeRestart 2 ALERT
Node node_id: GCP Take over started This node is attempting to assume responsibility for the next global checkpoint (that is, it is becoming the master node) GCP_TakeoverStarted NodeRestart 7 INFO
Node node_id: GCP Take over completed This node has become the master, and has assumed responsibility for the next global checkpoint GCP_TakeoverCompleted NodeRestart 7 INFO
Node node_id: LCP Take over started This node is attempting to assume responsibility for the next set of local checkpoints (that is, it is becoming the master node) LCP_TakeoverStarted NodeRestart 7 INFO
Node node_id: LCP Take over completed This node has become the master, and has assumed responsibility for the next set of local checkpoints LCP_TakeoverCompleted NodeRestart 7 INFO
Node node_id: Trans. Count = transactions, Commit Count = commits, Read Count = reads, Simple Read Count = simple_reads, Write Count = writes, AttrInfo Count = AttrInfo_objects, Concurrent Operations = concurrent_operations, Abort Count = aborts, Scans = scans, Range scans = range_scans This report of transaction activity is given approximately once every 10 seconds TransReportCounters Statistic 8 INFO
Node node_id: Operations=operations Number of operations performed by this node, provided approximately once every 10 seconds OperationReportCounters Statistic 8 INFO
Node node_id: Table with ID = table_id created A table having the table ID shown has been created TableCreated Statistic 7 INFO
Node node_id: Mean loop Counter in doJob last 8192 times = count JobStatistic Statistic 9 INFO
Mean send size to Node = node_id last 4096 sends = bytes bytes This node is sending an average of bytes bytes per send to node node_id SendBytesStatistic Statistic 9 INFO
Mean receive size to Node = node_id last 4096 sends = bytes bytes This node is receiving an average of bytes of data each time it receives data from node node_id ReceiveBytesStatistic Statistic 9 INFO
Node node_id: Data usage is data_memory_percentage% (data_pages_used 32K pages of total data_pages_total) / Node node_id: Index usage is index_memory_percentage% (index_pages_used 8K pages of total index_pages_total) This report is generated when a DUMP 1000 command is issued in the cluster management client MemoryUsage Statistic 5 INFO
Node node1_id: Transporter to node node2_id reported error error_code: error_message A transporter error occurred while communicating with node node2_id; for a listing of transporter error codes and messages, see NDB Transporter Errors, in MySQL NDB Cluster Internals Manual TransporterError Error 2 ERROR
Node node1_id: Transporter to node node2_id reported error error_code: error_message A warning of a potential transporter problem while communicating with node node2_id; for a listing of transporter error codes and messages, see NDB Transporter Errors, for more information TransporterWarning Error 8 WARNING
Node node1_id: Node node2_id missed heartbeat heartbeat_id This node missed a heartbeat from node node2_id MissedHeartbeat Error 8 WARNING
Node node1_id: Node node2_id declared dead due to missed heartbeat This node has missed at least 3 heartbeats from node node2_id, and so has declared that node dead DeadDueToHeartbeat Error 8 ALERT
Node node1_id: Node Sent Heartbeat to node = node2_id This node has sent a heartbeat to node node2_id SentHeartbeat Info 12 INFO
(NDB 7.5.0 and earlier:) Node node_id: Event buffer status: used=bytes_used (percent_used%) alloc=bytes_allocated (percent_available%) max=bytes_available apply_epoch=latest_restorable_epoch latest_epoch=latest_epoch This report is seen during heavy event buffer usage, for example, when many updates are being applied in a relatively short period of time; the report shows the number of bytes and the percentage of event buffer memory used, the bytes allocated and percentage still available, and the latest and latest restorable epochs EventBufferStatus Info 7 INFO
(NDB 7.5.1 and later:) Node node_id: Event buffer status (object_id): used=bytes_used (percent_used% of alloc) alloc=bytes_allocated max=bytes_available latest_consumed_epoch=latest_consumed_epoch latest_buffered_epoch=latest_buffered_epoch report_reason=report_reason This report is seen during heavy event buffer usage, for example, when many updates are being applied in a relatively short period of time; the report shows the number of bytes and the percentage of event buffer memory used, the bytes allocated and percentage still available, and the latest buffered and consumed epochs; for more information, see Section 6.2.3, “Event Buffer Reporting in the Cluster Log” EventBufferStatus2 Info 7 INFO
Node node_id: Entering single user mode, Node node_id: Entered single user mode Node API_node_id has exclusive access, Node node_id: Entering single user mode These reports are written to the cluster log when entering and exiting single user mode; API_node_id is the node ID of the API or SQL having exclusive access to the cluster (for more information, see Section 6.6, “NDB Cluster Single User Mode”); the message Unknown single user report API_node_id indicates an error has taken place and should never be seen in normal operation SingleUser Info 7 INFO
Node node_id: Backup backup_id started from node mgm_node_id A backup has been started using the management node having mgm_node_id; this message is also displayed in the cluster management client when the START BACKUP command is issued; for more information, see Section 6.8.2, “Using The NDB Cluster Management Client to Create a Backup” BackupStarted Backup 7 INFO
Node node_id: Backup backup_id started from node mgm_node_id completed. StartGCP: start_gcp StopGCP: stop_gcp #Records: records #LogRecords: log_records Data: data_bytes bytes Log: log_bytes bytes The backup having the ID backup_id has been completed; for more information, see Section 6.8.2, “Using The NDB Cluster Management Client to Create a Backup” BackupCompleted Backup 7 INFO
Node node_id: Backup request from mgm_node_id failed to start. Error: error_code The backup failed to start; for error codes, see MGM API Errors BackupFailedToStart Backup 7 ALERT
Node node_id: Backup backup_id started from mgm_node_id has been aborted. Error: error_code The backup was terminated after starting, possibly due to user intervention BackupAborted Backup 7 ALERT
Log Message Description Event Name Event Type Priority Severity