This section discusses how to perform a rolling restart of a MySQL Cluster installation, so called because it involves stopping and starting (or restarting) each node in turn, so that the cluster itself remains operational. This is often done as part of a rolling upgrade or rolling downgrade, where high availability of the cluster is mandatory and no downtime of the cluster as a whole is permissible. Where we refer to upgrades, the information provided here also generally applies to downgrades as well.
There are a number of reasons why a rolling restart might be desirable. These are described in the next few paragraphs.
Configuration change. To make a change in the cluster's configuration, such as adding an SQL node to the cluster, or setting a configuration parameter to a new value.
MySQL Cluster software upgrade or downgrade. To upgrade the cluster to a newer version of the MySQL Cluster software (or to downgrade it to an older version). This is usually referred to as a “rolling upgrade” (or “rolling downgrade”, when reverting to an older version of MySQL Cluster).
Change on node host. To make changes in the hardware or operating system on which one or more MySQL Cluster node processes are running.
System reset (cluster reset). To reset the cluster because it has reached an undesirable state. In such cases it is often desirable to reload the data and metadata of one or more data nodes. This can be done any of three ways:
Start each data node process (ndbd) with
the --initial option, which
forces the data node to clear its file system and to reload
all MySQL Cluster data and metadata from the other data nodes.
Create a backup using the ndb_mgm client
BACKUP command prior to performing the
restart. Following the upgrade, restore the node or nodes
using ndb_restore.
See Section 17.5.3, “Online Backup of MySQL Cluster”, and Section 17.4.14, “ndb_restore — Restore a MySQL Cluster Backup”, for more information.
Use mysqldump to create a backup prior to
the upgrade; afterward, restore the dump using
LOAD DATA
INFILE.
Resource Recovery.
To free memory previously allocated to a table by successive
INSERT and
DELETE operations, for re-use by
other MySQL Cluster tables.
The process for performing a rolling restart may be generalized as follows:
The specifics for implementing a given rolling upgrade depend upon the changes being made. A more detailed view of the process is presented here:
In the previous diagram, the Stop
and Start steps indicate that the
process must be stopped completely using a shell command (such as
kill on most Unix systems) or the management
client STOP command, then started again from a
system shell by invoking the ndbd or
ndb_mgmd executable as appropriate.
Restart indicates that the
process may be restarted using the ndb_mgm
management client RESTART command (see
Section 17.5.2, “Commands in the MySQL Cluster Management Client”).
When performing an upgrade or downgrade of the MySQL Cluster software, you must upgrade or downgrade the management nodes first, then the data nodes, and finally the SQL nodes. Doing so in any other order may leave the cluster in an unusable state.

User Comments
I am not sure if the following is documented, so I am putting in here. We used Server version: 5.0.45 MySQL Community Server (GPL)
It appears that ndbd rolling restarts wait for transactions to commit, but eventually time out and close the uncommited connection.
I think this is good behavior, it cleans up stalled transactions that would probably fail anyway, and then starts the node.
This came out of a lot of interesting scenarios spanning rolling restarts. We only start and stop ndbd. We have 2 ndb notes containing all the data, node 6 and 7.
I used two terminals, indicated below by prompts ndb_mgm> and mysql>. Read above how "restart -n" is like stop, but it can be started later from the ndb_mgm. Here is how I did the experiment:
ndb_mgm> 7 start
ndb_mgm> 6 restart -n
### This effectively stops node 6
ndb_mgm> show
### Observe node 6 is down, 7 is up
mysql> start transaction;
mysql> insert ...;
ndb_mgm> 6 start
ndb_mgm> show
### Node 6 six will stay in a "Starting" state for several minutes.
### if you commit the transaction, 6 will immediately start.
### if you wait until node 6 starts, 5 or 10 minutes, the following happens in the mysql client terminal:
mysql> select * from ...;
ERROR 2006 (HY000): MySQL server has gone away
No connection. Trying to reconnect...
Connection id: 165066
Current database: *** NONE ***
### Observe that the insert transaction was rolled back.
### That's it.
Speculative conclusion: rolling restarts wait for commits, but eventually time out and close the uncommitted connection.
This fails to mention what you must do if you make a change to the number of replicas. You update can be applied 'online'.
Add your own comment.