This section discusses how to perform a rolling restart of a MySQL Cluster installation, so called because it involves stopping and starting (or restarting) each node in turn, so that the cluster itself remains operational. This is often done as part of a rolling upgrade or rolling downgrade, where high availability of the cluster is mandatory and no downtime of the cluster as a whole is permissible. Where we refer to upgrades, the information provided here also generally applies to downgrades as well.
There are a number of reasons why a rolling restart might be desirable:
Cluster software upgrade/downgrade. To upgrade the cluster to a newer version of the MySQL Cluster software (or to downgrade it to an older version). This is usually referred to as a “rolling upgrade” (or “rolling downgrade”, when reverting to an older version of MySQL Cluster).
Change on node host. To make changes in the hardware or operating system on which one or more cluster nodes are running.
Cluster reset. To reset the cluster because it has reached an undesirable state. In such cases it is often desirable to reload the data and metadata of one or more data nodes. This can be done 1 of 3 ways:
Start each data node process
(ndbd, or possibly
ndbmtd in MySQL Cluster NDB 7.0
and later) with the
--initial option, which
forces the data node to clear its filesystem and
reload data and metadata from the other data nodes.
See Section 15.5.3, “Online Backup of MySQL Cluster”, and Section 15.4.15, “ndb_restore — Restore a MySQL Cluster Backup”, for more information.
Use mysqldump (see
Section 4.5.4, “mysqldump — A Database Backup Program”) to create a backup
prior to the upgrade; afterward, restore the dump
The process for performing a rolling restart may be generalized as follows:
The specifics for implementing a particular rolling upgrade depend upon the actual changes being made. A more detailed view of the process is presented here:
In the previous diagram, the
Start steps indicate that the
process must be stopped completely using a shell command (such
as kill on most Unix systems) or the
STOP command, then started
again from a system shell by invoking the
ndbd or ndb_mgmd
executable as appropriate.
Restart indicates the process
may be restarted using the ndb_mgm management
When performing an upgrade or downgrade of the cluster software, you must upgrade or downgrade the management nodes first, then the data nodes, and finally the SQL nodes. Doing so in any other order may leave the cluster in an unusable state.