The first task when preparing to import a standalone MySQL Cluster into MySQL Cluster Manager is to create a “target” cluster, Once this is done, we modify the target's configuration until it matches that of the “wild” cluster that we want to import. At a later point in the example, we also show how to test the configuration in a dry run before attempting to perform the actual import.
To create and then configure the target cluster, follow the steps listed here:
Install MySQL Cluster Manager and start mcmd on all hosts; we assume that you have installed MySQL Cluster Manager to the recommended location, in this case the directory
/opt/mcm-1.3.6
. (See Section 2.3, “MySQL Cluster Manager Installation”, for more information.) Once you have done this, you can start the mcm client (see Section 3.3, “Starting the MySQL Cluster Manager Client”) on any one of these hosts to perform the next few steps.Create a MySQL Cluster Manager site encompassing all four of these hosts, using the
create site
command, as shown here:mcm> create site --hosts=alpha,beta,gamma,delta newsite; +---------------------------+ | Command result | +---------------------------+ | Site created successfully | +---------------------------+ 1 row in set (0.15 sec)
We have named this site
newsite
. You should be able to see it listed in the output of thelist sites
command, similar to what is shown here:mcm> list sites; +---------+------+-------+------------------------+ | Site | Port | Local | Hosts | +---------+------+-------+------------------------+ | newsite | 1862 | Local | alpha,beta,gamma,delta | +---------+------+-------+------------------------+ 1 row in set (0.01 sec)
Add a MySQL Cluster Manager package referencing the MySQL Cluster binaries, using the
add package
command; this command's--basedir
option can be used to point to the correct location. The command shown here creates such a package, namednewpackage
:mcm> add package --basedir=/usr/local/mysql newpackage; +----------------------------+ | Command result | +----------------------------+ | Package added successfully | +----------------------------+ 1 row in set (0.70 sec)
You do not need to include the
bin
directory containing the MySQL Cluster executables in the--basedir
path. Since the executables are in/usr/local/mysql/bin
, it is sufficient to specify/usr/local/mysql
; MySQL Cluster Manager automatically checks for the binaries in abin
directory within the one specified by--basedir
.Create the target cluster including at least some of the same processes and hosts used by the standalone cluster. Do not include any processes or hosts that are not part of this cluster. In order to prevent potentially disruptive process or cluster operations from interfering by accident with the import process, it is strongly recommended that you create the cluster for import, using the
--import
option for thecreate cluster
command.You must also take care to preserve the correct node ID (as listed in the
config.ini
file shown previously) for each node. In MySQL Cluster Manager 1.3.1 and later, using the--import
option allows you to specify node IDs under 49 for nodes other than data nodes, which is otherwise prevented when usingcreate cluster
(the restriction has been lifted since MySQL Cluster Manager 1.3.4).The following command creates the cluster
newcluster
for import, and includes the management and data nodes, but not the SQL or “free” API node (which we add in the next step):mcm> create cluster --import --package=newpackage \ --processhosts=ndb_mgmd:50@alpha,ndbd:1@beta,ndbd:2@gamma \ newcluster; +------------------------------+ | Command result | +------------------------------+ | Cluster created successfully | +------------------------------+ 1 row in set (0.96 sec)
You can verify that the cluster was created correctly by checking the output of
show status
with the--process
(-r
) option, like this:mcm> show status -r newcluster; +--------+----------+-------+--------+-----------+------------+ | NodeId | Process | Host | Status | Nodegroup | Package | +--------+----------+-------+--------+-----------+------------+ | 50 | ndb_mgmd | alpha | import | | newpackage | | 5 | ndbd | beta | import | n/a | newpackage | | 6 | ndbd | gamma | import | n/a | newpackage | +--------+----------+-------+--------+-----------+------------+ 3 rows in set (0.01 sec)
If necessary, add any remaining processes and hosts from the “wild” cluster not included in the previous step using one or more
add process
commands. We have not yet accounted for 2 of the nodes from the wild cluster: the SQL node with node ID 100, on hostdelta
; and the API node which has node ID 101, and is not bound to any specific host. You can use the following command to add both of these processes tonewcluster
:mcm> add process --processhosts=mysqld:100@delta,ndbapi:101@* newcluster; +----------------------------+ | Command result | +----------------------------+ | Process added successfully | +----------------------------+ 1 row in set (0.41 sec)
Once again checking the output from
show status
-r
, we see that themysqld
andndbapi
processes were added as expected:mcm> show status -r newcluster; +--------+----------+-------+--------+-----------+------------+ | NodeId | Process | Host | Status | Nodegroup | Package | +--------+----------+-------+--------+-----------+------------+ | 50 | ndb_mgmd | alpha | import | | newpackage | | 5 | ndbd | beta | import | n/a | newpackage | | 6 | ndbd | gamma | import | n/a | newpackage | | 100 | mysqld | delta | import | | newpackage | | 101 | ndbapi | * | import | | | +--------+----------+-------+--------+-----------+------------+ 5 rows in set (0.08 sec)
You can also see that, since
newcluster
was created using thecreate cluster
command's--import
option, the status of all processes in this cluster—including those we just added—isimport
. This means we cannot yet startnewcluster
or any of its processes, as shown here:mcm> start process 50 newcluster; ERROR 5317 (00MGR): Unable to perform operation on cluster created for import mcm> start cluster newcluster; ERROR 5317 (00MGR): Unable to perform operation on cluster created for import
The
import
status and its effects onnewcluster
and its cluster processes persist until we have completed importing another cluster intonewcluster
.Following the execution of the
add process
command shown previously, the targetnewcluster
cluster now has the same processes, with the same node IDs, and on the same hosts as the original standalone cluster. We are ready to proceed to the next step.Duplicate the wild cluster's configuration attributes in the target cluster. In MySQL Cluster Manager 1.3.1 and later, you can handle most of these using the
import config
command. Test out first the effects of the command by running it with the--dryrun
option (the step only works if you have created the mcmd user on the cluster's mysqld nodes):mcm> import config --dryrun newcluster; +---------------------------------------------------------------------------+ | Command result | +---------------------------------------------------------------------------+ | Import checks passed. Please check log for settings that will be applied. | +---------------------------------------------------------------------------+ 1 row in set (5.36 sec)
ImportantBefore executing this command it is necessary to set any non-default ports for
ndb_mgmd
andmysqld
processes using theset
command in the mcm client.As indicated by the output from
import config --dryrun
, you can see the configuration attributes and values that would be copied tonewcluster
by the unimpeded command in the agent log file (mcmd.log
), which by default is created in the MySQL Cluster Manager installation directory. If you open this file in a text editor, you can locate a series ofset
commands that would accomplish this task, similar to what is shown here in emphasized texts:2014-03-14 16:05:11.896: (message) [T0x1ad12a0 CMGR ]: Got new message mgr_import_configvalues {84880f7a 35 0} 2014-03-14 16:05:11.896: (message) [T0x1ad12a0 CMGR ]: Got new message mgr_import_configvalues {84880f7a 36 0} 2014-03-14 16:05:11.896: (message) [T0x1ad12a0 CMGR ]: Got new message mgr_import_configvalues {84880f7a 37 0} 2014-03-14 16:05:13.698: (message) [T0x7f4fb80171a0 RECFG]: All utility process have finished 2014-03-14 16:05:13.698: (message) [T0x7f4fb80171a0 RECFG]: Process started : /usr/local/mysql/bin/mysqld --no-defaults --help --verbose 2014-03-14 16:05:13.698: (message) [T0x7f4fb80171a0 RECFG]: Spawning mysqld --nodefaults --help --verbose asynchronously 2014-03-14 16:05:13.904: (message) [T0x7f4fb80171a0 RECFG]: Successfully pulled default configuration from mysqld 100 2014-03-14 16:05:13.905: (warning) [T0x7f4fb80171a0 RECFG]: Failed to remove evsource! 2014-03-14 16:05:15.719: (message) [T0x7f4fb80171a0 RECFG]: All utility process have finished 2014-03-14 16:05:15.725: (message) [T0x7f4fb80171a0 RECFG]: Applying mysqld configuration to cluster... 2014-03-14 16:05:16.186: (message) [T0x1ad12a0 CMGR ]: Got new message mgr_import_configvalues {84880f7a 38 0} 2014-03-14 16:05:16.187: (message) [T0x1ad12a0 CMGR ]: Got new message x_trans {84880f7a 39 0} 2014-03-14 16:05:16.286: (message) [T0x1ad12a0 CMGR ]: Got new message x_trans {84880f7a 40 0} 2014-03-14 16:05:16.286: (message) [T0x7f4fb80171a0 RECFG]: The following will be applied to the current cluster config: set DataDir:ndb_mgmd:50="" newcluster set IndexMemory:ndbd:5=1073741824 newcluster set DataMemory:ndbd:5=1073741824 newcluster set DataDir:ndbd:5=/usr/local/mysql/mysql-cluster/data newcluster set ThreadConfig:ndbd:5="" newcluster set IndexMemory:ndbd:6=1073741824 newcluster set DataMemory:ndbd:6=1073741824 newcluster set DataDir:ndbd:6=/usr/local/mysql/mysql-cluster/data newcluster set ThreadConfig:ndbd:6="" newcluster set basedir:mysqld:100=/usr/local/mysql newcluster set character_sets_dir:mysqld:100=/usr/local/mysql/share/charsets newcluster set datadir:mysqld:100=/usr/local/mysql/data newcluster set general_log_file:mysqld:100=/usr/local/mysql/data/delta.log newcluster set lc_messages_dir:mysqld:100=/usr/local/mysql/share newcluster set log_error:mysqld:100=/usr/local/mysql/data/delta.err newcluster set ndb_connectstring:mysqld:100=alpha newcluster set ndb_mgmd_host:mysqld:100=alpha newcluster set optimizer_trace:mysqld:100=enabled=off,one_line=off newcluster set pid_file:mysqld:100=/usr/local/mysql/data/delta.pid newcluster set plugin_dir:mysqld:100=/usr/local/mysql/lib/plugin newcluster set report_port:mysqld:100=3306 newcluster set slow_query_log_file:mysqld:100=/usr/local/mysql/data/delta-slow.log newcluster set sql_mode:mysqld:100=STRICT_TRANS_TABLES,NO_ENGINE_SUBSTITUTION newcluster
Options used at the command line instead of in a configuration file to start a node of the standalone cluster are not imported into the target cluster by the
import config
command; moreover, they will cause one of the following to happen when theimport config
--dryrun
is run:For some options, MySQL Cluster Manager will issue a warning that “Option
<param>
may be removed on next restart of process<type>
<nodeid>
,” meaning that those options will not be imported into the target cluster, and thus will not be applied when those nodes are restarted after the import. Here are the lists of such options for each node type:For ndb_mgmd nodes: -
--configdir
,--initial
,--log-name
,--reload
,--verbose
For ndbd and ndbmtd nodes:
--connect-retries
,--connect-delay
,--daemon=false
,--nodaemon
,--verbose
For mysqld nodes:
--ndbcluster
, the--ndbinfo-*
options,--verbose
,--datadir
,--defaults-group-suffix
When a warning is received for any of these options but you want its value to be included as part of the configuration for your target cluster, you will need to set the option value for your target cluster manually using the set command before you perform the final
import cluster
step.For some other options, while their values will also not be imported into the target cluster, unlike options described in (i) above, no warnings will be issued for them. Here are lists of such options for each node type:
For ndb_mgmd nodes:
--config-cache
,--daemon
,--ndb-nodeid
,--nodaemon=false
,--config-file
,--skip-config-cache
For ndbd and ndbmtd nodes:
--daemon
,--foreground
,--initial
,--ndb-connectstring
,--connect-string
,--ndb-mgmd-host
,--ndb-nodeid
,--nodaemon=false
For mysqld nodes:
--ndb-connectstring
,--ndb-mgmd-host
,--ndb-nodeid
,--defaults-file
,--no-defaults
,--basedir
For options that belong to neither of the groups in (i) or (ii) above, having started the standalone cluster's nodes with them at the command line will cause the
import config
--dryrun
command to fail with an error, complaining that the options are unsupported. You have to restart the wild cluster's nodes without those options, and then retry theimport config
--dryrun
command.
Assuming that the dry run was successful, you should now be able to import the wild cluster's configuration into newcluster, with the command and a result similar to what is shown here:
mcm> import config newcluster; +------------------------------------------------------------------------------------------------------------------+ | Command result | +------------------------------------------------------------------------------------------------------------------+ | Configuration imported successfully. Please manually verify plugin options, abstraction level and default values | +------------------------------------------------------------------------------------------------------------------+
You should check the log from the dry run and the resulting configuration of
newcluster
carefully against the configuration of the wild cluster. If you find any inconsistencies, you must correct these innewcluster
using the appropriateset
commands afterwards.
Manual configuration import (MySQL Cluster Manager 1.3.0).
In MySQL Cluster Manager 1.3.0, which does not support the
import config
command, it
is necessary to copy the wild cluster's configuration
manually, using set
commands in the mcm client (once you have
obtained the values of any attributes that differ from their
defaults). The remainder of this section applies primarily
to MySQL Cluster Manager 1.3.0 and the process described here is generally
not needed in MySQL Cluster Manager 1.3.1 and later.
MySQL Cluster global configuration data is stored in a file
that is usually (but not always) named
config.ini
. Its location on a management
node host is arbitrary (there is no default location for it),
but if this is not already known, you can determine it by
checking—for example, on a typical Linux
system—the output of ps
for the --config-file
option
value that the management node was started with, shown with
emphasized text in the output:
$> ps ax | grep ndb_mgmd
18851 ? Ssl 0:00 ./ndb_mgmd --config-file=/var/lib/mysql-cluster/config.ini
18922 pts/4 S+ 0:00 grep --color=auto ndb_mgmd
This file uses INI
format to store global
configuration information, and is thus easy to read, or to
parse with a script. We start the setup of the target
cluster' configuration by checking each section of this
file in turn. The first section is repeated here:
[ndbd default]
DataMemory= 16G
IndexMemory= 12G
NoOfReplicas= 2
The [ndbd default]
heading indicates that
all attributes defined in this section apply to all cluster
data nodes. We can set all three attributes listed in this
section of the file for all data nodes in
newcluster
, using the
set
command shown here:
mcm> set DataMemory:ndbd=16G,IndexMemory:ndbd=12G,NoOfReplicas:ndbd=2 newcluster;
+-----------------------------------+
| Command result |
+-----------------------------------+
| Cluster reconfigured successfully |
+-----------------------------------+
1 row in set (0.36 sec)
You can verify that the desired changes have taken effect
using the get
command, as
shown here:
mcm> get DataMemory:ndbd,IndexMemory:ndbd,NoOfReplicas:ndbd newcluster;
+--------------+-------+----------+---------+----------+---------+---------+---------+
| Name | Value | Process1 | NodeId1 | Process2 | NodeId2 | Level | Comment |
+--------------+-------+----------+---------+----------+---------+---------+---------+
| DataMemory | 16G | ndbd | 5 | | | Process | |
| IndexMemory | 12G | ndbd | 5 | | | Process | |
| NoOfReplicas | 2 | ndbd | 5 | | | Process | |
| DataMemory | 16G | ndbd | 6 | | | Process | |
| IndexMemory | 12G | ndbd | 6 | | | Process | |
| NoOfReplicas | 2 | ndbd | 6 | | | Process | |
+--------------+-------+----------+---------+----------+---------+---------+---------+
6 rows in set (0.07 sec)
The next section in the file is shown here:
[ndb_mgmd]
HostName=alpha
NodeId=1
This section of the file applies to the management node. We
set its NodeId
and
HostName
attributes
previously, when we created newcluster
. No
further changes are required at this time.
The next two sections of the config.ini
file, shown here, contain configuration values specific to
each of the data nodes:
[ndbd]
NodeId=5
HostName=beta
DataDir=/var/lib/mysql-cluster
[ndbd]
NodeId=6
HostName=gamma
DataDir=/var/lib/mysql-cluster
As was the case for the management node, we already provided
the correct node IDs and host names for the data nodes when we
created newcluster
, so only the
DataDir
attribute
remains to be set. We can accomplish this by executing the
following command in the mcm client:
mcm> set DataDir:ndbd:5=/var/lib/mysql-cluster,DataDir:ndbd:6=/var/lib/mysql-cluster \
newcluster;
+-----------------------------------+
| Command result |
+-----------------------------------+
| Cluster reconfigured successfully |
+-----------------------------------+
1 row in set (0.42 sec)
You may have noticed that we could have set the
DataDir
attribute on
the process level using the shorter and simpler command
set DataDir:ndbd=/var/lib/mysql-cluster
newcluster
, but since this attribute was defined
individually for each data node in the original configuration,
we match this scope in the new configuration by setting this
attribute for each ndbd instance instead.
Once again, we check the result using the
mcm client
get
command, as shown here:
mcm> get DataDir:ndbd newcluster;
+---------+------------------------+----------+---------+----------+---------+-------+---------+
| Name | Value | Process1 | NodeId1 | Process2 | NodeId2 | Level | Comment |
+---------+------------------------+----------+---------+----------+---------+-------+---------+
| DataDir | /var/lib/mysql-cluster | ndbd | 5 | | | | |
| DataDir | /var/lib/mysql-cluster | ndbd | 6 | | | | |
+---------+------------------------+----------+---------+----------+---------+-------+---------+
2 rows in set (0.01 sec)
Configuration attributes for the SQL node are contained the next section of the file, shown here:
[mysqld]
NodeId=100
HostName=delta
The NodeId
and
HostName
attributes
were already set when we added the mysqld
process to newcluster
, so no additional
set
commands are required at
this point. Keep in mind that there may be
additional local configuration values for this
mysqld that must be accounted for in the
configuration we are creating for
newcluster
; we discuss how to
determine these values on the SQL node later in this section.
The remaining section of the file, shown here, contains a section defining attributes for a “free” API node that is not required to connect from any particular host:
[ndbapi]
NodeId=101
We have already set the NodeId
and there is
no need for a HostName
for a free process.
There are no other attributes that need to be set for this
node.
For more information about the MySQL
config.ini
global configuration file, see
NDB Cluster Configuration Files, and
NDB Cluster Configuration: Basic Example.
As mentioned earlier in this section, each
mysqld process (SQL node) may have, in
addition to any attributes set in
config.ini
, its own configuration data in
the form of system variables which are specific to that
mysqld. These can be set in two ways:
Because the initial values of many options can be changed at
runtime, it is recommended that—rather than attempt to
read the my.cnf
or
my.ini
file—you check values for
all system variables on each SQL node “live” in
the mysql client by examining the output of
the SHOW VARIABLES
statement,
and execute set
commands
setting each of these values where it differs from the default
for that variable on that SQL node.
The mcm client can execute a script file
containing client commands. The contents of such a script,
named my-commands.mcm
, which contains all
commands we executed to create and configure
newcluster
, are shown here:
create cluster --import --package=newpackage --processhosts=ndb_mgmd:50@alpha,ndbd:5@beta,ndbd:6@gamma newcluster;
add process --processhosts=mysqld:100@delta,ndbapi:101@* newcluster;
set DataMemory:ndbd=16G,IndexMemory:ndbd=12G,NoOfReplicas:ndbd=2 newcluster;
set DataDir:ndbd:5=/var/lib/mysql-cluster,DataDir:ndbd:6=/var/lib/mysql-cluster newcluster;
You can run such a script by invoking the client from the command line with a redirection operator, like this:
$> mcm < my-commands.mcm
The name of the script file is completely arbitrary. It must
contain valid mcm client commands or
comments only. (A comment is delimited by a
#
character, and extends from the point in
the line where this is found to the end of the line.) Any
valid mcm client command can be used in
such a file. mcm must be able to read the
file, but the file need not be executable, or readable by any
other users.