5.1 ndbd — The NDB Cluster Data Node Daemon

The ndbd binary provides the single-threaded version of the process that is used to handle all the data in tables employing the NDBCLUSTER storage engine. This data node process enables a data node to accomplish distributed transaction handling, node recovery, checkpointing to disk, online backup, and related tasks. In NDB 8.0.38 and later, when started, ndbd logs a warning similar to that shown here:

2024-05-28 13:32:16 [ndbd] WARNING  -- Running ndbd with a single thread of
signal execution.  For multi-threaded signal execution run the ndbmtd binary.

ndbmtd is the multi-threaded version of this binary.

In an NDB Cluster, a set of ndbd processes cooperate in handling data. These processes can execute on the same computer (host) or on different computers. The correspondences between data nodes and Cluster hosts is completely configurable.

Options that can be used with ndbd are shown in the following table. Additional descriptions follow the table.

Table 5.1 Command-line options used with the program ndbd

Format	Description	Added, Deprecated, or Removed
--bind-address=name	Local bind address	(Supported in all NDB releases based on MySQL 8.0)
--character-sets-dir=path	Directory containing character sets	(Supported in all NDB releases based on MySQL 8.0)
--connect-delay=#	Obsolete synonym for --connect-retry-delay, which should be used instead of this option	REMOVED: NDB 8.0.28
--connect-retries=#	Set the number of times to retry a connection before giving up; 0 means 1 attempt only (and no retries); -1 means continue retrying indefinitely	(Supported in all NDB releases based on MySQL 8.0)
--connect-retry-delay=#	Time to wait between attempts to contact a management server, in seconds; 0 means do not wait between attempts	(Supported in all NDB releases based on MySQL 8.0)
--connect-string=connection_string, -c connection_string	Same as --ndb-connectstring	(Supported in all NDB releases based on MySQL 8.0)
--core-file	Write core file on error; used in debugging	(Supported in all NDB releases based on MySQL 8.0)
--daemon, -d	Start ndbd as daemon (default); override with --nodaemon	(Supported in all NDB releases based on MySQL 8.0)
--defaults-extra-file=path	Read given file after global files are read	(Supported in all NDB releases based on MySQL 8.0)
--defaults-file=path	Read default options from given file only	(Supported in all NDB releases based on MySQL 8.0)
--defaults-group-suffix=string	Also read groups with concat(group, suffix)	(Supported in all NDB releases based on MySQL 8.0)
--filesystem-password=password	Password for node file system encryption; can be passed from stdin, tty, or my.cnf file	ADDED: NDB 8.0.31
--filesystem-password-from-stdin={TRUE|FALSE}	Get password for node file system encryption, passed from stdin	ADDED: NDB 8.0.31
--foreground	Run ndbd in foreground, provided for debugging purposes (implies --nodaemon)	(Supported in all NDB releases based on MySQL 8.0)
--help, -?	Display help text and exit	(Supported in all NDB releases based on MySQL 8.0)
--initial	Perform initial start of ndbd, including file system cleanup; consult documentation before using this option	(Supported in all NDB releases based on MySQL 8.0)
--initial-start	Perform partial initial start (requires --nowait-nodes)	(Supported in all NDB releases based on MySQL 8.0)
--install[=name]	Used to install data node process as Windows service; does not apply on other platforms	(Supported in all NDB releases based on MySQL 8.0)
--logbuffer-size=#	Control size of log buffer; for use when debugging with many log messages being generated; default is sufficient for normal operations	(Supported in all NDB releases based on MySQL 8.0)
--login-path=path	Read given path from login file	(Supported in all NDB releases based on MySQL 8.0)
--ndb-connectstring=connection_string, -c connection_string	Set connect string for connecting to ndb_mgmd. Syntax: "[nodeid=id;][host=]hostname[:port]". Overrides entries in NDB_CONNECTSTRING and my.cnf	(Supported in all NDB releases based on MySQL 8.0)
--ndb-mgmd-host=connection_string, -c connection_string	Same as --ndb-connectstring	(Supported in all NDB releases based on MySQL 8.0)
--ndb-nodeid=#	Set node ID for this node, overriding any ID set by --ndb-connectstring	(Supported in all NDB releases based on MySQL 8.0)
--nodaemon	Do not start ndbd as daemon; provided for testing purposes	(Supported in all NDB releases based on MySQL 8.0)
--no-defaults	Do not read default options from any option file other than login file	(Supported in all NDB releases based on MySQL 8.0)
--nostart, -n	Do not start ndbd immediately; ndbd waits for command to start from ndb_mgm	(Supported in all NDB releases based on MySQL 8.0)
--nowait-nodes=list	Do not wait for these data nodes to start (takes comma-separated list of node IDs); requires --ndb-nodeid	(Supported in all NDB releases based on MySQL 8.0)
--ndb-optimized-node-selection	Enable optimizations for selection of nodes for transactions. Enabled by default; use --skip-ndb-optimized-node-selection to disable	REMOVED: 8.0.31
--print-defaults	Print program argument list and exit	(Supported in all NDB releases based on MySQL 8.0)
--remove[=name]	Used to remove data node process that was previously installed as Windows service; does not apply on other platforms	(Supported in all NDB releases based on MySQL 8.0)
--usage, -?	Display help text and exit; same as --help	(Supported in all NDB releases based on MySQL 8.0)
--verbose, -v	Write extra debugging information to node log	(Supported in all NDB releases based on MySQL 8.0)
--version, -V	Display version information and exit	(Supported in all NDB releases based on MySQL 8.0)

Note

All of these options also apply to the multithreaded version of this program (ndbmtd) and you may substitute “ndbmtd” for “ndbd” wherever the latter occurs in this section.

• --bind-address

  Command-Line Format	--bind-address=name
  Type	String
  Default Value

  Causes ndbd to bind to a specific network interface (host name or IP address). This option has no default value.

• --character-sets-dir

  Command-Line Format	--character-sets-dir=path

  Directory containing character sets.

• --connect-delay=#

  Command-Line Format	--connect-delay=#
  Deprecated	Yes (removed in 8.0.28-ndb-8.0.28)
  Type	Numeric
  Default Value	5
  Minimum Value	0
  Maximum Value	3600

  Determines the time to wait between attempts to contact a management server when starting (the number of attempts is controlled by the --connect-retries option). The default is 5 seconds.

  This option is deprecated, and is subject to removal in a future release of NDB Cluster. Use --connect-retry-delay instead.

• --connect-retries=#

  Command-Line Format	--connect-retries=#
  Type	Numeric
  Default Value	12
  Minimum Value (≥ 8.0.28-ndb-8.0.28)	-1
  Minimum Value	-1
  Minimum Value	-1
  Minimum Value (≤ 8.0.27-ndb-8.0.27)	0
  Maximum Value	65535

  Set the number of times to retry a connection before giving up; 0 means 1 attempt only (and no retries). The default is 12 attempts. The time to wait between attempts is controlled by the --connect-retry-delay option.

  Beginning with NDB 8.0.28, you can set this option to -1, in which case, the data node process continues indefinitely to try to connect.

• --connect-retry-delay=#

  Command-Line Format	--connect-retry-delay=#
  Type	Numeric
  Default Value	5
  Minimum Value	0
  Maximum Value	4294967295

  Determines the time to wait between attempts to contact a management server when starting (the time between attempts is controlled by the --connect-retries option). The default is 5 seconds.

  This option takes the place of the --connect-delay option, which is now deprecated and subject to removal in a future release of NDB Cluster.

  The short form -r for this option is deprecated as of NDB 8.0.28, and subject to removal in a future release of NDB Cluster. Use the long form instead.

• --connect-string

  Command-Line Format	--connect-string=connection_string
  Type	String
  Default Value	[none]

  Same as --ndb-connectstring.

• --core-file

  Command-Line Format	--core-file

  Write core file on error; used in debugging.

• --daemon, -d

  Command-Line Format	--daemon

  Instructs ndbd or ndbmtd to execute as a daemon process. This is the default behavior. --nodaemon can be used to prevent the process from running as a daemon.

  This option has no effect when running ndbd or ndbmtd on Windows platforms.

• --defaults-extra-file

  Command-Line Format	--defaults-extra-file=path
  Type	String
  Default Value	[none]

  Read given file after global files are read.

• --defaults-file

  Command-Line Format	--defaults-file=path
  Type	String
  Default Value	[none]

  Read default options from given file only.

• --defaults-group-suffix

  Command-Line Format	--defaults-group-suffix=string
  Type	String
  Default Value	[none]

  Also read groups with concat(group, suffix).

• --filesystem-password

  Command-Line Format	--filesystem-password=password
  Introduced	8.0.31-ndb-8.0.31

  Pass the filesystem encryption and decryption password to the data node process using stdin, tty, or the my.cnf file.

  Requires EncryptedFileSystem = 1.

  For more information, see Section 6.14, “File System Encryption for NDB Cluster”.

• --filesystem-password-from-stdin

  Command-Line Format	--filesystem-password-from-stdin={TRUE|FALSE}
  Introduced	8.0.31-ndb-8.0.31

  Pass the filesystem encryption and decryption password to the data node process from stdin (only).

  Requires EncryptedFileSystem = 1.

  For more information, see Section 6.14, “File System Encryption for NDB Cluster”.

• --foreground

  Command-Line Format	--foreground

  Causes ndbd or ndbmtd to execute as a foreground process, primarily for debugging purposes. This option implies the --nodaemon option.

  This option has no effect when running ndbd or ndbmtd on Windows platforms.

• --help

  Command-Line Format	--help

  Display help text and exit.

• --initial

  Command-Line Format	--initial

  Instructs ndbd to perform an initial start. An initial start erases any files created for recovery purposes by earlier instances of ndbd. It also re-creates recovery log files. On some operating systems, this process can take a substantial amount of time.

  An --initial start is to be used only when starting the ndbd process under very special circumstances; this is because this option causes all files to be removed from the NDB Cluster file system and all redo log files to be re-created. These circumstances are listed here:

  • When performing a software upgrade which has changed the contents of any files.

  • When restarting the node with a new version of ndbd.

  • As a measure of last resort when for some reason the node restart or system restart repeatedly fails. In this case, be aware that this node can no longer be used to restore data due to the destruction of the data files.

  Warning

  To avoid the possibility of eventual data loss, it is recommended that you not use the --initial option together with StopOnError = 0. Instead, set StopOnError to 0 in config.ini only after the cluster has been started, then restart the data nodes normally—that is, without the --initial option. See the description of the StopOnError parameter for a detailed explanation of this issue. (Bug #24945638)

  Use of this option prevents the StartPartialTimeout and StartPartitionedTimeout configuration parameters from having any effect.

  Important

  This option does not affect backup files that have already been created by the affected node.

  Prior to NDB 8.0.21, the --initial option also did not affect any Disk Data files. In NDB 8.0.21 and later, when used to perform an initial restart of the cluster, the option causes the removal of all data files associated with Disk Data tablespaces and undo log files associated with log file groups that existed previously on this data node (see Section 6.11, “NDB Cluster Disk Data Tables”).

  This option also has no effect on recovery of data by a data node that is just starting (or restarting) from data nodes that are already running (unless they also were started with --initial, as part of an initial restart). This recovery of data occurs automatically, and requires no user intervention in an NDB Cluster that is running normally.

  It is permissible to use this option when starting the cluster for the very first time (that is, before any data node files have been created); however, it is not necessary to do so.

• --initial-start

  Command-Line Format	--initial-start

  This option is used when performing a partial initial start of the cluster. Each node should be started with this option, as well as --nowait-nodes.

  Suppose that you have a 4-node cluster whose data nodes have the IDs 2, 3, 4, and 5, and you wish to perform a partial initial start using only nodes 2, 4, and 5—that is, omitting node 3:

  $> ndbd --ndb-nodeid=2 --nowait-nodes=3 --initial-start
  $> ndbd --ndb-nodeid=4 --nowait-nodes=3 --initial-start
  $> ndbd --ndb-nodeid=5 --nowait-nodes=3 --initial-start

  When using this option, you must also specify the node ID for the data node being started with the --ndb-nodeid option.

  Important

  Do not confuse this option with the --nowait-nodes option for ndb_mgmd, which can be used to enable a cluster configured with multiple management servers to be started without all management servers being online.

• --install[=name]

  Command-Line Format	--install[=name]
  Platform Specific	Windows
  Type	String
  Default Value	ndbd

  Causes ndbd to be installed as a Windows service. Optionally, you can specify a name for the service; if not set, the service name defaults to ndbd. Although it is preferable to specify other ndbd program options in a my.ini or my.cnf configuration file, it is possible to use together with --install. However, in such cases, the --install option must be specified first, before any other options are given, for the Windows service installation to succeed.

  It is generally not advisable to use this option together with the --initial option, since this causes the data node file system to be wiped and rebuilt every time the service is stopped and started. Extreme care should also be taken if you intend to use any of the other ndbd options that affect the starting of data nodes—including --initial-start, --nostart, and --nowait-nodes—together with --install, and you should make absolutely certain you fully understand and allow for any possible consequences of doing so.

  The --install option has no effect on non-Windows platforms.

• --logbuffer-size=#

  Command-Line Format	--logbuffer-size=#
  Type	Integer
  Default Value	32768
  Minimum Value	2048
  Maximum Value	4294967295

  Sets the size of the data node log buffer. When debugging with high amounts of extra logging, it is possible for the log buffer to run out of space if there are too many log messages, in which case some log messages can be lost. This should not occur during normal operations.

• --login-path

  Command-Line Format	--login-path=path
  Type	String
  Default Value	[none]

  Read given path from login file.

• --ndb-connectstring

  Command-Line Format	--ndb-connectstring=connection_string
  Type	String
  Default Value	[none]

  Set connect string for connecting to ndb_mgmd. Syntax: "[nodeid=id;][host=]hostname[:port]". Overrides entries in NDB_CONNECTSTRING and my.cnf.

• --ndb-mgmd-host

  Command-Line Format	--ndb-mgmd-host=connection_string
  Type	String
  Default Value	[none]

  Same as --ndb-connectstring.

• --ndb-nodeid

  Command-Line Format	--ndb-nodeid=#
  Type	Integer
  Default Value	[none]

  Set node ID for this node, overriding any ID set by --ndb-connectstring.

• --ndb-optimized-node-selection

  Command-Line Format	--ndb-optimized-node-selection
  Removed	8.0.31

  Enable optimizations for selection of nodes for transactions. Enabled by default; use --skip-ndb-optimized-node-selection to disable.

• --nodaemon

  Command-Line Format	--nodaemon

  Prevents ndbd or ndbmtd from executing as a daemon process. This option overrides the --daemon option. This is useful for redirecting output to the screen when debugging the binary.

  The default behavior for ndbd and ndbmtd on Windows is to run in the foreground, making this option unnecessary on Windows platforms, where it has no effect.

• --no-defaults

  Command-Line Format	--no-defaults

  Do not read default options from any option file other than login file.

• --nostart, -n

  Command-Line Format	--nostart

  Instructs ndbd not to start automatically. When this option is used, ndbd connects to the management server, obtains configuration data from it, and initializes communication objects. However, it does not actually start the execution engine until specifically requested to do so by the management server. This can be accomplished by issuing the proper START command in the management client (see Section 6.1, “Commands in the NDB Cluster Management Client”).

• --nowait-nodes=node_id_1[, node_id_2[, ...]]

  Command-Line Format	--nowait-nodes=list
  Type	String
  Default Value

  This option takes a list of data nodes for which the cluster does not wait, prior to starting.

  This can be used to start the cluster in a partitioned state. For example, to start the cluster with only half of the data nodes (nodes 2, 3, 4, and 5) running in a 4-node cluster, you can start each ndbd process with --nowait-nodes=3,5. In this case, the cluster starts as soon as nodes 2 and 4 connect, and does not wait StartPartitionedTimeout milliseconds for nodes 3 and 5 to connect as it would otherwise.

  If you wanted to start up the same cluster as in the previous example without one ndbd (say, for example, that the host machine for node 3 has suffered a hardware failure) then start nodes 2, 4, and 5 with --nowait-nodes=3. Then the cluster starts as soon as nodes 2, 4, and 5 connect, and does not wait for node 3 to start.

• --print-defaults

  Command-Line Format	--print-defaults

  Print program argument list and exit.

• --remove[=name]

  Command-Line Format	--remove[=name]
  Platform Specific	Windows
  Type	String
  Default Value	ndbd

  Causes an ndbd process that was previously installed as a Windows service to be removed. Optionally, you can specify a name for the service to be uninstalled; if not set, the service name defaults to ndbd.

  The --remove option has no effect on non-Windows platforms.

• --usage

  Command-Line Format	--usage

  Display help text and exit; same as --help.

• --verbose, -v

  Causes extra debug output to be written to the node log.

  You can also use NODELOG DEBUG ON and NODELOG DEBUG OFF to enable and disable this extra logging while the data node is running.

• --version

  Command-Line Format	--version

  Display version information and exit.

ndbd generates a set of log files which are placed in the directory specified by DataDir in the config.ini configuration file.

These log files are listed below. node_id is and represents the node's unique identifier. For example, ndb_2_error.log is the error log generated by the data node whose node ID is 2.

• ndb_node_id_error.log is a file containing records of all crashes which the referenced ndbd process has encountered. Each record in this file contains a brief error string and a reference to a trace file for this crash. A typical entry in this file might appear as shown here:

  Date/Time: Saturday 30 July 2004 - 00:20:01
  Type of error: error
  Message: Internal program error (failed ndbrequire)
  Fault ID: 2341
  Problem data: DbtupFixAlloc.cpp
  Object of reference: DBTUP (Line: 173)
  ProgramName: NDB Kernel
  ProcessID: 14909
  TraceFile: ndb_2_trace.log.2
  ***EOM***

  Listings of possible ndbd exit codes and messages generated when a data node process shuts down prematurely can be found in Data Node Error Messages.

  Important

  The last entry in the error log file is not necessarily the newest one (nor is it likely to be). Entries in the error log are not listed in chronological order; rather, they correspond to the order of the trace files as determined in the ndb_node_id_trace.log.next file (see below). Error log entries are thus overwritten in a cyclical and not sequential fashion.

• ndb_node_id_trace.log.trace_id is a trace file describing exactly what happened just before the error occurred. This information is useful for analysis by the NDB Cluster development team.

  It is possible to configure the number of these trace files that are created before old files are overwritten. trace_id is a number which is incremented for each successive trace file.

• ndb_node_id_trace.log.next is the file that keeps track of the next trace file number to be assigned.

• ndb_node_id_out.log is a file containing any data output by the ndbd process. This file is created only if ndbd is started as a daemon, which is the default behavior.

• ndb_node_id.pid is a file containing the process ID of the ndbd process when started as a daemon. It also functions as a lock file to avoid the starting of nodes with the same identifier.

• ndb_node_id_signal.log is a file used only in debug versions of ndbd, where it is possible to trace all incoming, outgoing, and internal messages with their data in the ndbd process.

It is recommended not to use a directory mounted through NFS because in some environments this can cause problems whereby the lock on the .pid file remains in effect even after the process has terminated.

To start ndbd, it may also be necessary to specify the host name of the management server and the port on which it is listening. Optionally, one may also specify the node ID that the process is to use.

$> ndbd --connect-string="nodeid=2;host=ndb_mgmd.mysql.com:1186"

See Section 4.3.3, “NDB Cluster Connection Strings”, for additional information about this issue. For more information about data node configuration parameters, see Section 4.3.6, “Defining NDB Cluster Data Nodes”.

When ndbd starts, it actually initiates two processes. The first of these is called the “angel process”; its only job is to discover when the execution process has been completed, and then to restart the ndbd process if it is configured to do so. Thus, if you attempt to kill ndbd using the Unix kill command, it is necessary to kill both processes, beginning with the angel process. The preferred method of terminating an ndbd process is to use the management client and stop the process from there.

The execution process uses one thread for reading, writing, and scanning data, as well as all other activities. This thread is implemented asynchronously so that it can easily handle thousands of concurrent actions. In addition, a watch-dog thread supervises the execution thread to make sure that it does not hang in an endless loop. A pool of threads handles file I/O, with each thread able to handle one open file. Threads can also be used for transporter connections by the transporters in the ndbd process. In a multi-processor system performing a large number of operations (including updates), the ndbd process can consume up to 2 CPUs if permitted to do so.

For a machine with many CPUs it is possible to use several ndbd processes which belong to different node groups; however, such a configuration is still considered experimental and is not supported for MySQL 8.0 in a production setting. See Section 2.7, “Known Limitations of NDB Cluster”.