The NDB Cluster restoration program is implemented as a separate
command-line utility ndb_restore, which can
normally be found in the MySQL bin
directory. This program reads the files created as a result of
the backup and inserts the stored information into the database.
Beginning with NDB 7.5.15 and 7.6.11, this program no longer
prints NDBT_ProgramExit: ...
when it
finishes its run. Applications depending on this behavior
should be modified accordingly when upgrading from earlier
releases.
ndb_restore must be executed once for each of
the backup files that were created by the
START BACKUP
command used to
create the backup (see
Section 6.8.2, “Using The NDB Cluster Management Client to Create a Backup”).
This is equal to the number of data nodes in the cluster at the
time that the backup was created.
Before using ndb_restore, it is recommended that the cluster be running in single user mode, unless you are restoring multiple data nodes in parallel. See Section 6.6, “NDB Cluster Single User Mode”, for more information.
Options that can be used with ndb_restore are shown in the following table. Additional descriptions follow the table.
Table 5.17 Command-line options used with the program ndb_restore
Format | Description | Added, Deprecated, or Removed |
---|---|---|
Allow changes to set of columns making up table's primary key | ADDED: NDB 7.6.14 |
|
Append data to tab-delimited file | (Supported in all NDB releases based on MySQL 5.7) |
|
Path to backup files directory | (Supported in all NDB releases based on MySQL 5.7) |
|
Restore from backup having this ID | (Supported in all NDB releases based on MySQL 5.7) |
|
Directory containing character sets | (Supported in all NDB releases based on MySQL 5.7) |
|
Alias for --connectstring | (Supported in all NDB releases based on MySQL 5.7) |
|
Number of times to retry connection before giving up | (Supported in all NDB releases based on MySQL 5.7) |
|
Number of seconds to wait between attempts to contact management server | (Supported in all NDB releases based on MySQL 5.7) |
|
Same as --ndb-connectstring | (Supported in all NDB releases based on MySQL 5.7) |
|
Write core file on error; used in debugging | (Supported in all NDB releases based on MySQL 5.7) |
|
Read given file after global files are read | (Supported in all NDB releases based on MySQL 5.7) |
|
Read default options from given file only | (Supported in all NDB releases based on MySQL 5.7) |
|
Also read groups with concat(group, suffix) | (Supported in all NDB releases based on MySQL 5.7) |
|
Causes indexes from backup to be ignored; may decrease time needed to restore data | (Supported in all NDB releases based on MySQL 5.7) |
|
Do not ignore system table during restore; experimental only; not for production use | (Supported in all NDB releases based on MySQL 5.7) |
|
List of one or more databases to exclude (includes those not named) | (Supported in all NDB releases based on MySQL 5.7) |
|
Do not restore any intermediate tables (having names prefixed with '#sql-') that were left over from copying ALTER TABLE operations; specify FALSE to restore such tables | (Supported in all NDB releases based on MySQL 5.7) |
|
Causes columns from backup version of table that are missing from version of table in database to be ignored | (Supported in all NDB releases based on MySQL 5.7) |
|
Causes tables from backup that are missing from database to be ignored | (Supported in all NDB releases based on MySQL 5.7) |
|
List of one or more tables to exclude (includes those in same database that are not named); each table reference must include database name | (Supported in all NDB releases based on MySQL 5.7) |
|
Fields are enclosed by this character | (Supported in all NDB releases based on MySQL 5.7) |
|
Fields are optionally enclosed by this character | (Supported in all NDB releases based on MySQL 5.7) |
|
Fields are terminated by this character | (Supported in all NDB releases based on MySQL 5.7) |
|
Display help text and exit | (Supported in all NDB releases based on MySQL 5.7) |
|
Print binary types in hexadecimal format | (Supported in all NDB releases based on MySQL 5.7) |
|
Ignore log entries containing updates to columns now included in extended primary key | ADDED: NDB 7.6.14 |
|
List of one or more databases to restore (excludes those not named) | (Supported in all NDB releases based on MySQL 5.7) |
|
List of one or more tables to restore (excludes those in same database that are not named); each table reference must include database name | (Supported in all NDB releases based on MySQL 5.7) |
|
Lines are terminated by this character | (Supported in all NDB releases based on MySQL 5.7) |
|
Read given path from login file | (Supported in all NDB releases based on MySQL 5.7) |
|
Allow lossy conversions of column values (type demotions or changes in sign) when restoring data from backup | (Supported in all NDB releases based on MySQL 5.7) |
|
If mysqld is connected and using binary logging, do not log restored data | (Supported in all NDB releases based on MySQL 5.7) |
|
Do not read default options from any option file other than login file | (Supported in all NDB releases based on MySQL 5.7) |
|
Do not restore objects relating to Disk Data | (Supported in all NDB releases based on MySQL 5.7) |
|
Do not upgrade array type for varsize attributes which do not already resize VAR data, and do not change column attributes | (Supported in all NDB releases based on MySQL 5.7) |
|
Set connect string for connecting to ndb_mgmd. Syntax: "[nodeid=id;][host=]hostname[:port]". Overrides entries in NDB_CONNECTSTRING and my.cnf | (Supported in all NDB releases based on MySQL 5.7) |
|
Same as --ndb-connectstring | (Supported in all NDB releases based on MySQL 5.7) |
|
Specify node group map; unused, unsupported | (Supported in all NDB releases based on MySQL 5.7) |
|
Set node ID for this node, overriding any ID set by --ndb-connectstring | (Supported in all NDB releases based on MySQL 5.7) |
|
Enable optimizations for selection of nodes for transactions. Enabled by default; use --skip-ndb-optimized-node-selection to disable | (Supported in all NDB releases based on MySQL 5.7) |
|
ID of node where backup was taken | (Supported in all NDB releases based on MySQL 5.7) |
|
Number of slices to apply when restoring by slice | ADDED: NDB 7.6.13 |
|
Number of parallel transactions to use while restoring data | (Supported in all NDB releases based on MySQL 5.7) |
|
Allow preservation of trailing spaces (including padding) when promoting fixed-width string types to variable-width types | (Supported in all NDB releases based on MySQL 5.7) |
|
Print metadata, data, and log to stdout (equivalent to --print-meta --print-data --print-log) | (Supported in all NDB releases based on MySQL 5.7) |
|
Print data to stdout | (Supported in all NDB releases based on MySQL 5.7) |
|
Print program argument list and exit | (Supported in all NDB releases based on MySQL 5.7) |
|
Print log to stdout | (Supported in all NDB releases based on MySQL 5.7) |
|
Print metadata to stdout | (Supported in all NDB releases based on MySQL 5.7) |
|
Write SQL log to stdout | ADDED: NDB 7.5.4 |
|
Print status of restore each given number of seconds | (Supported in all NDB releases based on MySQL 5.7) |
|
Allow attributes to be promoted when restoring data from backup | (Supported in all NDB releases based on MySQL 5.7) |
|
Causes multithreaded rebuilding of ordered indexes found in backup; number of threads used is determined by setting BuildIndexThreads | (Supported in all NDB releases based on MySQL 5.7) |
|
Apply offset to value of specified column using indicated function and arguments. Format is [db].[tbl].[col]:[fn]:[args]; see documentation for details | ADDED: NDB 7.6.14 |
|
Restore table data and logs into NDB Cluster using NDB API | (Supported in all NDB releases based on MySQL 5.7) |
|
Restore epoch info into status table; useful on replica cluster for starting replication; updates or inserts row in mysql.ndb_apply_status with ID 0 | (Supported in all NDB releases based on MySQL 5.7) |
|
Restore metadata to NDB Cluster using NDB API | (Supported in all NDB releases based on MySQL 5.7) |
|
Restore MySQL privilege tables that were previously moved to NDB | (Supported in all NDB releases based on MySQL 5.7) |
|
Restore to differently named database; format is olddb,newdb | (Supported in all NDB releases based on MySQL 5.7) |
|
Ignore missing blob tables in backup file | (Supported in all NDB releases based on MySQL 5.7) |
|
Skip table structure check during restore | (Supported in all NDB releases based on MySQL 5.7) |
|
Causes schema objects not recognized by ndb_restore to be ignored when restoring backup made from newer NDB version to older version | (Supported in all NDB releases based on MySQL 5.7) |
|
Slice ID, when restoring by slices | ADDED: NDB 7.6.13 |
|
Creates a tab-separated .txt file for each table in path provided | (Supported in all NDB releases based on MySQL 5.7) |
|
Prefix all info, error, and debug log messages with timestamps | ADDED: NDB 7.5.30, 5.7.41-ndb-7.6.26 |
|
Display help text and exit; same as --help | (Supported in all NDB releases based on MySQL 5.7) |
|
Level of verbosity in output | (Supported in all NDB releases based on MySQL 5.7) |
|
Display version information and exit | (Supported in all NDB releases based on MySQL 5.7) |
-
Command-Line Format --allow-pk-changes[=0|1]
Introduced 5.7.29-ndb-7.6.14 Type Integer Default Value 0
Minimum Value 0
Maximum Value 1
When this option is set to
1
, ndb_restore allows the primary keys in a table definition to differ from that of the same table in the backup. This may be desirable when backing up and restoring between different schema versions with primary key changes on one or more tables, and it appears that performing the restore operation using ndb_restore is simpler or mor efficient than issuing manyALTER TABLE
statements after restoring table schemas and data.The following changes in primary key definitions are supported by
--allow-pk-changes
:Extending the primary key: A non-nullable column that exists in the table schema in the backup becomes part of the table's primary key in the database.
ImportantWhen extending a table's primary key, any columns which become part of primary key must not be updated while the backup is being taken; any such updates discovered by ndb_restore cause the restore operation to fail, even when no change in value takes place. In some cases, it may be possible to override this behavior using the
--ignore-extended-pk-updates
option; see the description of this option for more information.Contracting the primary key (1): A column that is already part of the table's primary key in the backup schema is no longer part of the primary key, but remains in the table.
Contracting the primary key (2): A column that is already part of the table's primary key in the backup schema is removed from the table entirely.
These differences can be combined with other schema differences supported by ndb_restore, including changes to blob and text columns requiring the use of staging tables.
Basic steps in a typical scenario using primary key schema changes are listed here:
Restore table schemas using ndb_restore
--restore-meta
Alter schema to that desired, or create it
Back up the desired schema
Run ndb_restore
--disable-indexes
using the backup from the previous step, to drop indexes and constraintsRun ndb_restore
--allow-pk-changes
(possibly along with--ignore-extended-pk-updates
,--disable-indexes
, and possibly other options as needed) to restore all dataRun ndb_restore
--rebuild-indexes
using the backup made with the desired schema, to rebuild indexes and constraints
When extending the primary key, it may be necessary for ndb_restore to use a temporary secondary unique index during the restore operation to map from the old primary key to the new one. Such an index is created only when necessary to apply events from the backup log to a table which has an extended primary key. This index is named
NDB$RESTORE_PK_MAPPING
, and is created on each table requiring it; it can be shared, if necessary, by multiple instances of ndb_restore instances running in parallel. (Running ndb_restore--rebuild-indexes
at the end of the restore process causes this index to be dropped.) -
Command-Line Format --append
When used with the
--tab
and--print-data
options, this causes the data to be appended to any existing files having the same names. -
Command-Line Format --backup-path=path
Type Directory name Default Value ./
The path to the backup directory is required; this is supplied to ndb_restore using the
--backup-path
option, and must include the subdirectory corresponding to the ID backup of the backup to be restored. For example, if the data node'sDataDir
is/var/lib/mysql-cluster
, then the backup directory is/var/lib/mysql-cluster/BACKUP
, and the backup files for the backup with the ID 3 can be found in/var/lib/mysql-cluster/BACKUP/BACKUP-3
. The path may be absolute or relative to the directory in which the ndb_restore executable is located, and may be optionally prefixed withbackup-path=
.It is possible to restore a backup to a database with a different configuration than it was created from. For example, suppose that a backup with backup ID
12
, created in a cluster with two storage nodes having the node IDs2
and3
, is to be restored to a cluster with four nodes. Then ndb_restore must be run twice—once for each storage node in the cluster where the backup was taken. However, ndb_restore cannot always restore backups made from a cluster running one version of MySQL to a cluster running a different MySQL version. See Section 3.7, “Upgrading and Downgrading NDB Cluster”, for more information.ImportantIt is not possible to restore a backup made from a newer version of NDB Cluster using an older version of ndb_restore. You can restore a backup made from a newer version of MySQL to an older cluster, but you must use a copy of ndb_restore from the newer NDB Cluster version to do so.
For example, to restore a cluster backup taken from a cluster running NDB Cluster 7.5.35 to a cluster running NDB Cluster 7.4.39, you must use the ndb_restore that comes with the NDB Cluster 7.5.35 distribution.
For more rapid restoration, the data may be restored in parallel, provided that there is a sufficient number of cluster connections available. That is, when restoring to multiple nodes in parallel, you must have an
[api]
or[mysqld]
section in the clusterconfig.ini
file available for each concurrent ndb_restore process. However, the data files must always be applied before the logs. -
Command-Line Format --backupid=#
Type Numeric Default Value none
This option is used to specify the ID or sequence number of the backup, and is the same number shown by the management client in the
Backup
message displayed upon completion of a backup. (See Section 6.8.2, “Using The NDB Cluster Management Client to Create a Backup”.)backup_id
completedImportantWhen restoring cluster backups, you must be sure to restore all data nodes from backups having the same backup ID. Using files from different backups can at best result in restoring the cluster to an inconsistent state, and may fail altogether.
In NDB 7.5.13 and later, and in NDB 7.6.9 and later, this option is required.
-
Command-Line Format --character-sets-dir=path
Directory containing character sets.
-
Command-Line Format --connect=connection_string
Type String Default Value localhost:1186
Alias for
--ndb-connectstring
. -
Command-Line Format --connect-retries=#
Type Integer Default Value 12
Minimum Value 0
Maximum Value 12
Number of times to retry connection before giving up.
-
Command-Line Format --connect-retry-delay=#
Type Integer Default Value 5
Minimum Value 0
Maximum Value 5
Number of seconds to wait between attempts to contact management server.
-
Command-Line Format --connect-string=connection_string
Type String Default Value [none]
Same as
--ndb-connectstring
. -
Command-Line Format --core-file
Write core file on error; used in debugging.
-
Command-Line Format --defaults-extra-file=path
Type String Default Value [none]
Read given file after global files are read.
-
Command-Line Format --defaults-file=path
Type String Default Value [none]
Read default options from given file only.
-
Command-Line Format --defaults-group-suffix=string
Type String Default Value [none]
Also read groups with concat(group, suffix).
-
Command-Line Format --disable-indexes
Disable restoration of indexes during restoration of the data from a native
NDB
backup. Afterwards, you can restore indexes for all tables at once with multithreaded building of indexes using--rebuild-indexes
, which should be faster than rebuilding indexes concurrently for very large tables.Beginning with NDB 7.5.24 and NDB 7.6.20, this option also drops any foreign keys specified in the backup.
-
Command-Line Format --dont-ignore-systab-0
Normally, when restoring table data and metadata, ndb_restore ignores the copy of the
NDB
system table that is present in the backup.--dont-ignore-systab-0
causes the system table to be restored. This option is intended for experimental and development use only, and is not recommended in a production environment. -
Command-Line Format --exclude-databases=list
Type String Default Value Comma-delimited list of one or more databases which should not be restored.
This option is often used in combination with
--exclude-tables
; see that option's description for further information and examples. --exclude-intermediate-sql-tables[
=TRUE|FALSE]
Command-Line Format --exclude-intermediate-sql-tables[=TRUE|FALSE]
Type Boolean Default Value TRUE
When performing copying
ALTER TABLE
operations, mysqld creates intermediate tables (whose names are prefixed with#sql-
). WhenTRUE
, the--exclude-intermediate-sql-tables
option keeps ndb_restore from restoring such tables that may have been left over from these operations. This option isTRUE
by default.-
Command-Line Format --exclude-missing-columns
It is possible to restore only selected table columns using this option, which causes ndb_restore to ignore any columns missing from tables being restored as compared to the versions of those tables found in the backup. This option applies to all tables being restored. If you wish to apply this option only to selected tables or databases, you can use it in combination with one or more of the
--include-*
or--exclude-*
options described elsewhere in this section to do so, then restore data to the remaining tables using a complementary set of these options. -
Command-Line Format --exclude-missing-tables
It is possible to restore only selected tables using this option, which causes ndb_restore to ignore any tables from the backup that are not found in the target database.
-
Command-Line Format --exclude-tables=list
Type String Default Value List of one or more tables to exclude; each table reference must include the database name. Often used together with
--exclude-databases
.When
--exclude-databases
or--exclude-tables
is used, only those databases or tables named by the option are excluded; all other databases and tables are restored by ndb_restore.This table shows several invocations of ndb_restore usng
--exclude-*
options (other options possibly required have been omitted for clarity), and the effects these options have on restoring from an NDB Cluster backup:Table 5.18 Several invocations of ndb_restore using --exclude-* options, and the effects these options have on restoring from an NDB Cluster backup.
Option Result --exclude-databases=db1
All tables in all databases except db1
are restored; no tables indb1
are restored--exclude-databases=db1,db2
(or--exclude-databases=db1
--exclude-databases=db2
)All tables in all databases except db1
anddb2
are restored; no tables indb1
ordb2
are restored--exclude-tables=db1.t1
All tables except t1
in databasedb1
are restored; all other tables indb1
are restored; all tables in all other databases are restored--exclude-tables=db1.t2,db2.t1
(or--exclude-tables=db1.t2
--exclude-tables=db2.t1)
All tables in database db1
except fort2
and all tables in databasedb2
except for tablet1
are restored; no other tables indb1
ordb2
are restored; all tables in all other databases are restoredYou can use these two options together. For example, the following causes all tables in all databases except for databases
db1
anddb2
, and tablest1
andt2
in databasedb3
, to be restored:$> ndb_restore [...] --exclude-databases=db1,db2 --exclude-tables=db3.t1,db3.t2
(Again, we have omitted other possibly necessary options in the interest of clarity and brevity from the example just shown.)
You can use
--include-*
and--exclude-*
options together, subject to the following rules:The actions of all
--include-*
and--exclude-*
options are cumulative.All
--include-*
and--exclude-*
options are evaluated in the order passed to ndb_restore, from right to left.In the event of conflicting options, the first (rightmost) option takes precedence. In other words, the first option (going from right to left) that matches against a given database or table “wins”.
For example, the following set of options causes ndb_restore to restore all tables from database
db1
exceptdb1.t1
, while restoring no other tables from any other databases:--include-databases=db1 --exclude-tables=db1.t1
However, reversing the order of the options just given simply causes all tables from database
db1
to be restored (includingdb1.t1
, but no tables from any other database), because the--include-databases
option, being farthest to the right, is the first match against databasedb1
and thus takes precedence over any other option that matchesdb1
or any tables indb1
:--exclude-tables=db1.t1 --include-databases=db1
-
Command-Line Format --fields-enclosed-by=char
Type String Default Value Each column value is enclosed by the string passed to this option (regardless of data type; see the description of
--fields-optionally-enclosed-by
). --fields-optionally-enclosed-by
Command-Line Format --fields-optionally-enclosed-by
Type String Default Value The string passed to this option is used to enclose column values containing character data (such as
CHAR
,VARCHAR
,BINARY
,TEXT
, orENUM
).-
Command-Line Format --fields-terminated-by=char
Type String Default Value \t (tab)
The string passed to this option is used to separate column values. The default value is a tab character (
\t
). -
Command-Line Format --help
Display help text and exit.
-
Command-Line Format --hex
If this option is used, all binary values are output in hexadecimal format.
-
Command-Line Format --ignore-extended-pk-updates[=0|1]
Introduced 5.7.29-ndb-7.6.14 Type Integer Default Value 0
Minimum Value 0
Maximum Value 1
When using the
--allow-pk-changes
option, columns which become part of a table's primary key must not be updated while the backup is being taken; such columns should keep the same values from the time values are inserted into them until the rows containing the values are deleted. If ndb_restore encounters updates to these columns when restoring a backup, the restore fails. Because some applications may set values for all columns when updating a row, even when some column values are not changed, the backup may include log events appearing to update columns which are not in fact modified. In such cases you can set--ignore-extended-pk-updates
to1
, forcing ndb_restore to ignore such updates.ImportantWhen causing these updates to be ignored, the user is responsible for ensuring that there are no updates to the values of any columns that become part of the primary key.
For more information, see the description of
--allow-pk-changes
. -
Command-Line Format --include-databases=list
Type String Default Value Comma-delimited list of one or more databases to restore. Often used together with
--include-tables
; see the description of that option for further information and examples. -
Command-Line Format --include-tables=list
Type String Default Value Comma-delimited list of tables to restore; each table reference must include the database name.
When
--include-databases
or--include-tables
is used, only those databases or tables named by the option are restored; all other databases and tables are excluded by ndb_restore, and are not restored.The following table shows several invocations of ndb_restore using
--include-*
options (other options possibly required have been omitted for clarity), and the effects these have on restoring from an NDB Cluster backup:Table 5.19 Several invocations of ndb_restore using --include-* options, and their effects on restoring from an NDB Cluster backup.
Option Result --include-databases=db1
Only tables in database db1
are restored; all tables in all other databases are ignored--include-databases=db1,db2
(or--include-databases=db1
--include-databases=db2
)Only tables in databases db1
anddb2
are restored; all tables in all other databases are ignored--include-tables=db1.t1
Only table t1
in databasedb1
is restored; no other tables indb1
or in any other database are restored--include-tables=db1.t2,db2.t1
(or--include-tables=db1.t2
--include-tables=db2.t1
)Only the table t2
in databasedb1
and the tablet1
in databasedb2
are restored; no other tables indb1
,db2
, or any other database are restoredYou can also use these two options together. For example, the following causes all tables in databases
db1
anddb2
, together with the tablest1
andt2
in databasedb3
, to be restored (and no other databases or tables):$> ndb_restore [...] --include-databases=db1,db2 --include-tables=db3.t1,db3.t2
(Again we have omitted other, possibly required, options in the example just shown.)
It also possible to restore only selected databases, or selected tables from a single database, without any
--include-*
(or--exclude-*
) options, using the syntax shown here:ndb_restore other_options db_name,[db_name[,...] | tbl_name[,tbl_name][,...]]
In other words, you can specify either of the following to be restored:
All tables from one or more databases
One or more tables from a single database
-
Command-Line Format --lines-terminated-by=char
Type String Default Value \n (linebreak)
Specifies the string used to end each line of output. The default is a linefeed character (
\n
). -
Command-Line Format --login-path=path
Type String Default Value [none]
Read given path from login file.
-
Command-Line Format --lossy-conversions
This option is intended to complement the
--promote-attributes
option. Using--lossy-conversions
allows lossy conversions of column values (type demotions or changes in sign) when restoring data from backup. With some exceptions, the rules governing demotion are the same as for MySQL replication; see Replication of Columns Having Different Data Types, for information about specific type conversions currently supported by attribute demotion.Beginning with NDB 7.5.23 and NDB 7.6.19, this option also makes it possible to restore a
NULL
column asNOT NULL
. The column must not contain anyNULL
entries; otherwise ndb_restore stops with an error.ndb_restore reports any truncation of data that it performs during lossy conversions once per attribute and column.
-
Command-Line Format --ndb-connectstring=connection_string
Type String Default Value [none]
Set connect string for connecting to ndb_mgmd. Syntax: "[nodeid=id;][host=]hostname[:port]". Overrides entries in NDB_CONNECTSTRING and my.cnf.
-
Command-Line Format --ndb-mgmd-host=connection_string
Type String Default Value [none]
Same as
--ndb-connectstring
. -
Command-Line Format --ndb-nodegroup-map=map
Intended for restoring a backup taken from one node group to a different node group, but never completely implemented; unsupported.
-
Command-Line Format --ndb-nodeid=#
Type Integer Default Value [none]
Set node ID for this node, overriding any ID set by
--ndb-connectstring
. --ndb-optimized-node-selection
Command-Line Format --ndb-optimized-node-selection
Enable optimizations for selection of nodes for transactions. Enabled by default; use
--skip-ndb-optimized-node-selection
to disable.-
Command-Line Format --no-binlog
This option prevents any connected SQL nodes from writing data restored by ndb_restore to their binary logs.
-
Command-Line Format --no-defaults
Do not read default options from any option file other than login file.
-
Command-Line Format --no-restore-disk-objects
This option stops ndb_restore from restoring any NDB Cluster Disk Data objects, such as tablespaces and log file groups; see Section 6.11, “NDB Cluster Disk Data Tables”, for more information about these.
-
Command-Line Format --no-upgrade
When using ndb_restore to restore a backup,
VARCHAR
columns created using the old fixed format are resized and recreated using the variable-width format now employed. This behavior can be overridden by specifying--no-upgrade
. -
Command-Line Format --nodeid=#
Type Numeric Default Value none
Specify the node ID of the data node on which the backup was taken.
When restoring to a cluster with different number of data nodes from that where the backup was taken, this information helps identify the correct set or sets of files to be restored to a given node. (In such cases, multiple files usually need to be restored to a single data node.) See Section 5.24.2, “Restoring to a different number of data nodes”, for additional information and examples.
In NDB 7.5.13 and later, and in NDB 7.6.9 and later, this option is required.
-
Command-Line Format --num-slices=#
Introduced 5.7.29-ndb-7.6.13 Type Integer Default Value 1
Minimum Value 1
Maximum Value 1024
When restoring a backup by slices, this option sets the number of slices into which to divide the backup. This allows multiple instances of ndb_restore to restore disjoint subsets in parallel, potentially reducing the amount of time required to perform the restore operation.
A slice is a subset of the data in a given backup; that is, it is a set of fragments having the same slice ID, specified using the
--slice-id
option. The two options must always be used together, and the value set by--slice-id
must always be less than the number of slices.ndb_restore encounters fragments and assigns each one a fragment counter. When restoring by slices, a slice ID is assigned to each fragment; this slice ID is in the range 0 to 1 less than the number of slices. For a table that is not a
BLOB
table, the slice to which a given fragment belongs is determined using the formula shown here:[slice_ID] = [fragment_counter] % [number_of_slices]
For a
BLOB
table, a fragment counter is not used; the fragment number is used instead, along with the ID of the main table for theBLOB
table (recall thatNDB
storesBLOB
values in a separate table internally). In this case, the slice ID for a given fragment is calculated as shown here:[slice_ID] = ([main_table_ID] + [fragment_ID]) % [number_of_slices]
Thus, restoring by
N
slices means runningN
instances of ndb_restore, all with--num-slices=
(along with any other necessary options) and one each withN
--slice-id=1
,--slice-id=2
,--slice-id=3
, and so on throughslice-id=
.N
-1 -
Command-Line Format --parallelism=#
Type Numeric Default Value 128
Minimum Value 1
Maximum Value 1024
ndb_restore uses single-row transactions to apply many rows concurrently. This parameter determines the number of parallel transactions (concurrent rows) that an instance of ndb_restore tries to use. By default, this is 128; the minimum is 1, and the maximum is 1024.
The work of performing the inserts is parallelized across the threads in the data nodes involved. This mechanism is employed for restoring bulk data from the
.Data
file—that is, the fuzzy snapshot of the data; it is not used for building or rebuilding indexes. The change log is applied serially; index drops and builds are DDL operations and handled separately. There is no thread-level parallelism on the client side of the restore. --preserve-trailing-spaces
,-P
Command-Line Format --preserve-trailing-spaces
Cause trailing spaces to be preserved when promoting a fixed-width character data type to its variable-width equivalent—that is, when promoting a
CHAR
column value toVARCHAR
, or aBINARY
column value toVARBINARY
. Otherwise, any trailing spaces are dropped from such column values when they are inserted into the new columns.-
Command-Line Format --print
Causes ndb_restore to print all data, metadata, and logs to
stdout
. Equivalent to using the--print-data
,--print-meta
, and--print-log
options together.NoteUse of
--print
or any of the--print_*
options is in effect performing a dry run. Including one or more of these options causes any output to be redirected tostdout
; in such cases, ndb_restore makes no attempt to restore data or metadata to an NDB Cluster. -
Command-Line Format --print-data
Cause ndb_restore to direct its output to
stdout
. Often used together with one or more of--tab
,--fields-enclosed-by
,--fields-optionally-enclosed-by
,--fields-terminated-by
,--hex
, and--append
.TEXT
andBLOB
column values are always truncated. Such values are truncated to the first 256 bytes in the output. This cannot currently be overridden when using--print-data
. -
Command-Line Format --print-defaults
Print program argument list and exit.
-
Command-Line Format --print-log
Cause ndb_restore to output its log to
stdout
. -
Command-Line Format --print-meta
Print all metadata to
stdout
. -
Command-Line Format --print-sql-log
Introduced 5.7.16-ndb-7.5.4 Log SQL statements to
stdout
. Use the option to enable; normally this behavior is disabled. The option checks before attempting to log whether all the tables being restored have explicitly defined primary keys; queries on a table having only the hidden primary key implemented byNDB
cannot be converted to valid SQL.This option does not work with tables having
BLOB
columns.The
--print-sql-log
option was added in NDB 7.5.4. (Bug #13511949) -
Command-Line Format --progress-frequency=#
Type Numeric Default Value 0
Minimum Value 0
Maximum Value 65535
Print a status report each
N
seconds while the backup is in progress. 0 (the default) causes no status reports to be printed. The maximum is 65535. -
Command-Line Format --promote-attributes
ndb_restore supports limited attribute promotion in much the same way that it is supported by MySQL replication; that is, data backed up from a column of a given type can generally be restored to a column using a “larger, similar” type. For example, data from a
CHAR(20)
column can be restored to a column declared asVARCHAR(20)
,VARCHAR(30)
, orCHAR(30)
; data from aMEDIUMINT
column can be restored to a column of typeINT
orBIGINT
. See Replication of Columns Having Different Data Types, for a table of type conversions currently supported by attribute promotion.Beginning with NDB 7.5.23 and NDB 7.6.19, this option also makes it possible to restore a
NOT NULL
column asNULL
.Attribute promotion by ndb_restore must be enabled explicitly, as follows:
Prepare the table to which the backup is to be restored. ndb_restore cannot be used to re-create the table with a different definition from the original; this means that you must either create the table manually, or alter the columns which you wish to promote using
ALTER TABLE
after restoring the table metadata but before restoring the data.Invoke ndb_restore with the
--promote-attributes
option (short form-A
) when restoring the table data. Attribute promotion does not occur if this option is not used; instead, the restore operation fails with an error.
When converting between character data types and
TEXT
orBLOB
, only conversions between character types (CHAR
andVARCHAR
) and binary types (BINARY
andVARBINARY
) can be performed at the same time. For example, you cannot promote anINT
column toBIGINT
while promoting aVARCHAR
column toTEXT
in the same invocation of ndb_restore.Converting between
TEXT
columns using different character sets is not supported, and is expressly disallowed.When performing conversions of character or binary types to
TEXT
orBLOB
with ndb_restore, you may notice that it creates and uses one or more staging tables named
. These tables are not needed afterwards, and are normally deleted by ndb_restore following a successful restoration.table_name
$STnode_id
-
Command-Line Format --rebuild-indexes
Enable multithreaded rebuilding of the ordered indexes while restoring a native
NDB
backup. The number of threads used for building ordered indexes by ndb_restore with this option is controlled by theBuildIndexThreads
data node configuration parameter and the number of LDMs.It is necessary to use this option only for the first run of ndb_restore; this causes all ordered indexes to be rebuilt without using
--rebuild-indexes
again when restoring subsequent nodes. You should use this option prior to inserting new rows into the database; otherwise, it is possible for a row to be inserted that later causes a unique constraint violation when trying to rebuild the indexes.Building of ordered indices is parallelized with the number of LDMs by default. Offline index builds performed during node and system restarts can be made faster using the
BuildIndexThreads
data node configuration parameter; this parameter has no effect on dropping and rebuilding of indexes by ndb_restore, which is performed online.Rebuilding of unique indexes uses disk write bandwidth for redo logging and local checkpointing. An insufficient amount of this bandwith can lead to redo buffer overload or log overload errors. In such cases you can run ndb_restore
--rebuild-indexes
again; the process resumes at the point where the error occurred. You can also do this when you have encountered temporary errors. You can repeat execution of ndb_restore--rebuild-indexes
indefinitely; you may be able to stop such errors by reducing the value of--parallelism
. If the problem is insufficient space, you can increase the size of the redo log (FragmentLogFileSize
node configuration parameter), or you can increase the speed at which LCPs are performed (MaxDiskWriteSpeed
and related parameters), in order to free space more quickly. --remap-column=
db
.tbl
.col
:fn
:args
Command-Line Format --remap-column=string
Introduced 5.7.29-ndb-7.6.14 Type String Default Value [none]
When used together with
--restore-data
, this option applies a function to the value of the indicated column. Values in the argument string are listed here:db
: Database name, following any renames performed by--rewrite-database
.tbl
: Table name.col
: Name of the column to be updated. This column must be of typeINT
orBIGINT
. The column can also be but is not required to beUNSIGNED
.fn
: Function name; currently, the only supported name isoffset
.args
: Arguments supplied to the function. Currently, only a single argument, the size of the offset to be added by theoffset
function, is supported. Negative values are supported. The size of the argument cannot exceed that of the signed variant of the column's type; for example, ifcol
is anINT
column, then the allowed range of the argument passed to theoffset
function is-2147483648
to2147483647
(see Integer Types (Exact Value) - INTEGER, INT, SMALLINT, TINYINT, MEDIUMINT, BIGINT).If applying the offset value to the column would cause an overflow or underflow, the restore operation fails. This could happen, for example, if the column is a
BIGINT
, and the option attempts to apply an offset value of 8 on a row in which the column value is 4294967291, since4294967291 + 8 = 4294967299 > 4294967295
.
This option can be useful when you wish to merge data stored in multiple source instances of NDB Cluster (all using the same schema) into a single destination NDB Cluster, using NDB native backup (see Section 6.8.2, “Using The NDB Cluster Management Client to Create a Backup”) and ndb_restore to merge the data, where primary and unique key values are overlapping between source clusters, and it is necessary as part of the process to remap these values to ranges that do not overlap. It may also be necessary to preserve other relationships between tables. To fulfill such requirements, it is possible to use the option multiple times in the same invocation of ndb_restore to remap columns of different tables, as shown here:
$> ndb_restore --restore-data --remap-column=hr.employee.id:offset:1000 \ --remap-column=hr.manager.id:offset:1000 --remap-column=hr.firstaiders.id:offset:1000
(Other options not shown here may also be used.)
--remap-column
can also be used to update multiple columns of the same table. Combinations of multiple tables and columns are possible. Different offset values can also be used for different columns of the same table, like this:$> ndb_restore --restore-data --remap-column=hr.employee.salary:offset:10000 \ --remap-column=hr.employee.hours:offset:-10
When source backups contain duplicate tables which should not be merged, you can handle this by using
--exclude-tables
,--exclude-databases
, or by some other means in your application.Information about the structure and other characteristics of tables to be merged can obtained using
SHOW CREATE TABLE
; the ndb_desc tool; andMAX()
,MIN()
,LAST_INSERT_ID()
, and other MySQL functions.Replication of changes from merged to unmerged tables, or from unmerged to merged tables, in separate instances of NDB Cluster is not supported.
-
Command-Line Format --restore-data
Output
NDB
table data and logs. -
Command-Line Format --restore-epoch
Add (or restore) epoch information to the cluster replication status table. This is useful for starting replication on an NDB replica cluster. When this option is used, the row in the
mysql.ndb_apply_status
having0
in theid
column is updated if it already exists; such a row is inserted if it does not already exist. (See Section 7.9, “NDB Cluster Backups With NDB Cluster Replication”.) -
Command-Line Format --restore-meta
This option causes ndb_restore to print
NDB
table metadata.The first time you run the ndb_restore restoration program, you also need to restore the metadata. In other words, you must re-create the database tables—this can be done by running it with the
--restore-meta
(-m
) option. Restoring the metadata need be done only on a single data node; this is sufficient to restore it to the entire cluster.In older versions of NDB Cluster, tables whose schemas were restored using this option used the same number of partitions as they did on the original cluster, even if it had a differing number of data nodes from the new cluster. In NDB 7.5.2 and later, when restoring metadata, this is no longer an issue; ndb_restore now uses the default number of partitions for the target cluster, unless the number of local data manager threads is also changed from what it was for data nodes in the original cluster.
NoteThe cluster should have an empty database when starting to restore a backup. (In other words, you should start the data nodes with
--initial
prior to performing the restore.) -
Command-Line Format --restore-privilege-tables
ndb_restore does not by default restore distributed MySQL privilege tables. This option causes ndb_restore to restore the privilege tables.
This works only if the privilege tables were converted to
NDB
before the backup was taken. For more information, see Section 6.13, “Distributed Privileges Using Shared Grant Tables”. --rewrite-database
=olddb,newdb
Command-Line Format --rewrite-database=string
Type String Default Value none
This option makes it possible to restore to a database having a different name from that used in the backup. For example, if a backup is made of a database named
products
, you can restore the data it contains to a database namedinventory
, use this option as shown here (omitting any other options that might be required):$> ndb_restore --rewrite-database=product,inventory
The option can be employed multiple times in a single invocation of ndb_restore. Thus it is possible to restore simultaneously from a database named
db1
to a database nameddb2
and from a database nameddb3
to one nameddb4
using--rewrite-database=db1,db2 --rewrite-database=db3,db4
. Other ndb_restore options may be used between multiple occurrences of--rewrite-database
.In the event of conflicts between multiple
--rewrite-database
options, the last--rewrite-database
option used, reading from left to right, is the one that takes effect. For example, if--rewrite-database=db1,db2 --rewrite-database=db1,db3
is used, only--rewrite-database=db1,db3
is honored, and--rewrite-database=db1,db2
is ignored. It is also possible to restore from multiple databases to a single database, so that--rewrite-database=db1,db3 --rewrite-database=db2,db3
restores all tables and data from databasesdb1
anddb2
into databasedb3
.ImportantWhen restoring from multiple backup databases into a single target database using
--rewrite-database
, no check is made for collisions between table or other object names, and the order in which rows are restored is not guaranteed. This means that it is possible in such cases for rows to be overwritten and updates to be lost.-
Command-Line Format --skip-broken-objects
This option causes ndb_restore to ignore corrupt tables while reading a native
NDB
backup, and to continue restoring any remaining tables (that are not also corrupted). Currently, the--skip-broken-objects
option works only in the case of missing blob parts tables. -
Command-Line Format --skip-table-check
It is possible to restore data without restoring table metadata. By default when doing this, ndb_restore fails with an error if a mismatch is found between the table data and the table schema; this option overrides that behavior.
Some of the restrictions on mismatches in column definitions when restoring data using ndb_restore are relaxed; when one of these types of mismatches is encountered, ndb_restore does not stop with an error as it did previously, but rather accepts the data and inserts it into the target table while issuing a warning to the user that this is being done. This behavior occurs whether or not either of the options
--skip-table-check
or--promote-attributes
is in use. These differences in column definitions are of the following types:Different
COLUMN_FORMAT
settings (FIXED
,DYNAMIC
,DEFAULT
)Different
STORAGE
settings (MEMORY
,DISK
)Different default values
Different distribution key settings
-
Command-Line Format --skip-unknown-objects
This option causes ndb_restore to ignore any schema objects it does not recognize while reading a native
NDB
backup. This can be used for restoring a backup made from a cluster running (for example) NDB 7.6 to a cluster running NDB Cluster 7.5. -
Command-Line Format --slice-id=#
Introduced 5.7.29-ndb-7.6.13 Type Integer Default Value 0
Minimum Value 0
Maximum Value 1023
When restoring by slices, this is the ID of the slice to restore. This option is always used together with
--num-slices
, and its value must be always less than that of--num-slices
.For more information, see the description of the
--num-slices
elsewhere in this section. -
Command-Line Format --tab=path
Type Directory name Causes
--print-data
to create dump files, one per table, each named
. It requires as its argument the path to the directory where the files should be saved; usetbl_name
.txt.
for the current directory. -
Command-Line Format --timestamp-printouts{=true|false}
Introduced 5.7.41-ndb-7.5.30, 5.7.41-ndb-7.6.26 Type Boolean Default Value false
Causes info, error, and debug log messages to be prefixed with timestamps.
This option is disabled by default in NDB 7.5 and NDB 7.6. Set it explicitly to
true
to enable. -
Command-Line Format --usage
Display help text and exit; same as
--help
. -
Command-Line Format --verbose=#
Type Numeric Default Value 1
Minimum Value 0
Maximum Value 255
Sets the level for the verbosity of the output. The minimum is 0; the maximum is 255. The default value is 1.
-
Command-Line Format --version
Display version information and exit.
Typical options for this utility are shown here:
ndb_restore [-c connection_string] -n node_id -b backup_id \
[-m] -r --backup-path=/path/to/backup/files
Normally, when restoring from an NDB Cluster backup,
ndb_restore requires at a minimum the
--nodeid
(short form:
-n
),
--backupid
(short form:
-b
), and
--backup-path
options. In
addition, when ndb_restore is used to restore
any tables containing unique indexes, you must include
--disable-indexes
or
--rebuild-indexes
. (Bug
#57782, Bug #11764893)
The -c
option is used to specify a connection
string which tells ndb_restore
where to
locate the cluster management server (see
Section 4.3.3, “NDB Cluster Connection Strings”). If this
option is not used, then ndb_restore attempts
to connect to a management server on
localhost:1186
. This utility acts as a
cluster API node, and so requires a free connection
“slot” to connect to the cluster management server.
This means that there must be at least one
[api]
or [mysqld]
section
that can be used by it in the cluster
config.ini
file. It is a good idea to keep
at least one empty [api]
or
[mysqld]
section in
config.ini
that is not being used for a
MySQL server or other application for this reason (see
Section 4.3.7, “Defining SQL and Other API Nodes in an NDB Cluster”).
You can verify that ndb_restore is connected
to the cluster by using the
SHOW
command in the
ndb_mgm management client. You can also
accomplish this from a system shell, as shown here:
$> ndb_mgm -e "SHOW"
Error reporting.
ndb_restore reports both temporary and
permanent errors. In the case of temporary errors, it may able
to recover from them, and reports Restore successful,
but encountered temporary error, please look at
configuration
in such cases.
After using ndb_restore to initialize an
NDB Cluster for use in circular replication, binary logs on
the SQL node acting as the replica are not automatically
created, and you must cause them to be created manually. To
cause the binary logs to be created, issue a
SHOW TABLES
statement on that
SQL node before running START
SLAVE
. This is a known issue in NDB Cluster.