1.4 Files that Are Backed Up

DBA and development work typically involves logical structures such as tables, rows, columns, the data dictionary, and so on. For backups, you must understand the physical details of how these structures are represented by files.

Table 1.1 Files in a MySQL Enterprise Backup Output Directory

File Name, Pattern, or Extension	Relation to Original Data Files	Notes
ibdata*	The InnoDB system tablespace, containing multiple InnoDB tables and associated indexes.	Because the original files might change while the backup is in progress, the apply-log step applies the same changes to the corresponding backup files.
*.ibd	InnoDB file-per-table tablespaces, each containing a single InnoDB table and associated indexes.	Used for tables created using the innodb_file_per_table option. Because the original files might change while the backup is in progress, the apply-log step applies the same changes to the corresponding backup files.
*.ibz	Compressed form of InnoDB data files from the MySQL data directory.	Produced instead of .ibd files in a compressed backup. The ibdata* files representing the InnoDB system tablespace also receive this extension in a compressed backup. The .ibz files are uncompressed during the apply-log, copy-back, or copy-back-and-apply-log step.
*.frm	Hold metadata about all MySQL tables.	The database is put into a read-only state while these files are copied. These files are copied without changes.
*.MYD	MyISAM table data.	The database is put into a read-only state while these files are copied. These files are copied without changes.
*.MYI	MyISAM index data.	The database is put into a read-only state while these files are copied. These files are copied without changes.
*.CSM	Metadata for CSV tables.	These files are copied without changes. The backup_history and backup_progress tables created by mysqlbackup use the CSV format, so the backup always includes some files with this extension.
*.CSV	Data for CSV tables.	These files are copied without changes. The backup_history and backup_progress tables created by mysqlbackup use the CSV format, so the backup always includes some files with this extension.
*.MRG	MERGE storage engine references to other tables.	The database is put into a read-only state while these files are copied. These files are copied without changes.
*.TRG	Trigger parameters.	The database is put into a read-only state while these files are copied. These files are copied without changes.
*.TRN	Trigger namespace information.	The database is put into a read-only state while these files are copied. These files are copied without changes.
*.opt	Database configuration information.	The database is put into a read-only state while these files are copied. These files are copied without changes.
*.par	Definitions for partitioned tables.	The database is put into a read-only state while these files are copied. These files are copied without changes.
*.ARM	ARCHIVE storage engine table metadata.	The database is put into a read-only state while these files are copied. These files are copied without changes.
*.ARZ	ARCHIVE storage engine table data.	The database is put into a read-only state while these files are copied. These files are copied without changes.
backup-my.cnf	Records the configuration parameters that specify the layout and other important information about the MySQL data files.	The file is created during a backup, and it contains crucial parameters describing the backed-up data like innodb_data_file_path, innodb_log_file_size, innodb_log_files_in_group, and so on. It might also contain other InnoDB parameters like innodb_data_home_dir and innodb_undo_directory if some of the backup repository options were used during the backup. mysqlbackup uses the parameters stored in this file to understand the structure of the backup and to perform various operations. You might need to supply some of these parameters to mysqlbackup during a restore and to mysqld when you start the target server if the target server and the backup are configured differently. See the discussion in Section 4.2.4, “Restoring a Database” for details.
ibbackup_ibd_files	Records names of the .ibd files and their space IDs during an incremental backup.	This file is created during an incremental backup. During a restore, the information in the file is used to delete the tables from the full backup that has been removed between the time of the full backup and the time of the incremental backup.
ibbackup_logfile	A condensed version of the ib_logfile* files from the MySQL data directory.	The InnoDB log files (ib_logfile*) are fixed-size files that are continuously updated during the database's operation. For backup purposes, only the changes that are committed while the backup is in progress are needed. These changes are recorded in ibbackup_logfile, and used to re-create the ib_logfile* files during the apply-log phase.
ibbackup_redo_log_only	Created instead of the ibbackup_logfile for incremental backups taken with the --incremental-with-redo-log-only option.
ib_logfile*	Created in the backup directory by mysqlbackup during the apply-log phase after the initial backup.	These files are not copied from the original data directory, but rather re-created in the backup directory during the apply-log phase after the initial backup, using the changes recorded in the ibbackup_logfile file.
*.bl	Renamed version of each .isl file from the backed-up server.	A .isl file is created when you specify the location of an InnoDB table using the syntax CREATE TABLE ... DATA DIRECTORY = ..., to act like a symbolic link pointing to the tablespace file. (See Creating Tables Externally for details.) The .bl files might or might not be turned back into .isl files during the copy-back operation. If the specified directory does not exist on the server where the backup is restored, mysqlbackup attempts to create it. If the directory cannot be created, the restore operation fails. Thus, if you would like to use the DATA DIRECTORY clause to put tables at different locations or to restore to a server with a different file structure where the corresponding directories cannot be created, edit the .bl files before restoring to point to directories that do exist on the destination server. If the directory on the target server pointed to by a .bl file already contains .ibd files, the --force option is required when you restore the backup.
Timestamped directory, such as 2011-05-26_13-42-02	Created by the --with-timestamp option. All the backup files go inside this subdirectory.	Use the --with-timestamp option to easily keep more than one set of backup data under the same main backup directory.
datadir directory	A subdirectory that stores the data files and database subdirectories from the original MySQL instance.	Created under the backup directory by mysqlbackup.
binary log files	Binary log files from the server, which are included in a backup by default (except when the backup is created with the --use-tts option). They allow a snapshot of the server to be taken, so a server can be cloned to its exact state. Using a full backup as a basis, the binary log files that are included with an incremental backup can be used for a point-in-time recovery (PITR), which restores a database to its state at a certain point in time after the last full backup. See Section 5.3, “Point-in-Time Recovery” for details.	Saved under the datadir directory under the backup directory. Use the --skip-binlog option to exclude binary logs in the backup. For MySQL 5.5 and earlier, as well as all offline backups, use the --log-bin-index option to specify the absolute path of the index file on the MySQL server that lists all the used binary log files, if it is different from the default value of the option, for mysqlbackup to find the binary log files and include them in the backups. The index file itself, with the locations of the binary log files properly updated to point to the files' locations in the backup directory, is included into the backup under the datadir directory. The binary log files are compressed and saved with the .bz extension when being included in a compressed backup. The binary log files and the index file, when included in a backup, are always copied into the restored server's data directory during a restore operation. Use the --skip-binlog option to skip the restoring of the binary log. Notes If any binary log files are missing on the server you are backing up, you should use the --skip-binlog option to avoid mysqlbackup throwing an error for the missing files. No binary log files are copied into the incremental backup if the --use-tts option or the --start-lsn option is used. To include binary log files for the period covered by the incremental backup, do not use the --use-tts option and, instead of --start-lsn, use the --incremental-base option, which provides the necessary information for mysqlbackup to ensure that no gap exists between binary log data included in the previous backup and the current incremental backup.
relay log files	Relay log files from a replica server, which are included in a backup of a replica server by default (except when the backup is created with the --use-tts option). Their inclusion saves the time and resources required for fetching the relay logs from the source when the replica is being restored.	Saved under the datadir directory under the backup directory. Use the --skip-relaylog option to exclude relay logs in the backup. For offline backup, use the --relay-log-index option to specify the absolute path of the index file on the MySQL server that lists all the used relay log files, if it is different from the default value of the option, for mysqlbackup to find the relay log files and include them in the backups. The index file itself, with the locations of the relay log files properly updated to point to the files' locations in the backup directory, is included into the backup under the datadir directory. The relay log files are compressed and saved with the .bz extension when being included in a compressed backup. The relay log files and the index file, when included in a backup, are always copied into the restored server's data directory during a restore operation. Use the --skip-relaylog option to skip the restoring of the relay log.
*.bz	Compressed binary log or relay log files.	The binary log and relay log files are compressed and saved with the .bz extension when being included in a compressed backup. They are decompressed during a restore.
replication metadata repository files	Usually named master.info and relay-log.info, they are included by default in a backup of a replica database in a replication setup. See Replication Metadata Repositories, for details.	Saved under the datadir directory under the backup directory. For an offline backup, use the--master-info-file and --relaylog-info-file options to specify the absolute paths of the information files, if they are different from the default values of the options, for mysqlbackup to find those files and include them in the backups. The copying of these files are skipped during a backup or a restore when the --skip-relay-log option is used.
Backup image file	A single-file backup produced by the backup-to-image command, with a name specified by the --backup-image option.	If your backup data directory consists only of zero-byte files, with a single giant data file in the top-level directory, you have a single-file backup. You can move the image file without losing or damaging the contents inside it, then unpack it with mysqlbackup using the extract option and specifying the same image name with the --backup-image option. Although some extra files such as backup-my.cnf and the meta subdirectory are present in the backup directory, these files are also included in the image file and do not need to be moved along with it.
Any other files in subdirectories under the datadir directory (that is, under backup-dir/datadir/subdir)	Copied from the database subdirectories under the MySQL data directory.	By default, any unrecognized files in subdirectories under the MySQL data directory are copied to the backup. To omit such files, specify the --only-known-file-types option. Note Some limitations apply to this behavior. See the discussion here in Appendix B, Limitations of MySQL Enterprise Backup.
meta directory	A subdirectory that stores files with metadata about the backup.	Created under the backup directory by mysqlbackup. All files listed below go inside the meta subdirectory.
backup_variables.txt	Holds important information about the backup. For use by mysqlbackup only.	mysqlbackup consults and possibly updates this file during operations after the initial backup, such as the apply-log phase or the restore phase.
image_files.xml	Contains the list of all the files (except itself) that are present in the single-file backup produced by the backup-to-image or backup-dir-to-image options. For details about this file, see Section 11.4, “Using the MySQL Enterprise Backup Manifest”.	This file is not modified at any stage once generated.
backup_create.xml	Lists the command line arguments and environment in which the backup was created. For details about this file, see Section 11.4, “Using the MySQL Enterprise Backup Manifest”.	This file is not modified once it is created. You can prevent this file from being generated by specifying the --disable-manifest option.
backup_content.xml	Essential metadata for the files and database definitions of the backup data. It also contains details of all the plugins defined on the backed-up server, by which users should make sure the same plugins are defined in the same manner on the target server for restoration. For details about this file, see Section 11.4, “Using the MySQL Enterprise Backup Manifest”.	This file is not modified once created. You can prevent this file from being generated by specifying the --disable-manifest option.
comments.txt	Produced by the --comments or --comments-file option.	The comments are specified by you to document the purpose or special considerations for this backup job.
backup_gtid_executed.sql	Signifies the backup came from a server with GTIDs enabled.	GTIDs are a replication feature in MySQL 5.6 and higher. See Replication with Global Transaction Identifiers for details. When you back up a server with GTIDs enabled using mysqlbackup, the file named backup_gtid_executed.sql is created in the meta folder under the backup directory. Edit and execute this file after restoring the backup data on a replica server; see Section 6.1, “Setting Up a New Replica” for details.
server-my.cnf	Contains values of the backed-up server's global variables that are set to non-default values. Use this file or server-all.cnf to start the target server for restoration.	During a copy-back or copy-back-and-apply-log operation, the server repository options values (e.g., --datadir, --innodb_data_home_dir, etc.) in the file are modified if the command makes changes to them through the command options. However, during an apply-incremental-backup operation, the values already saved in the file take precedence and they are not modified by the option values supplied through the command. Warning When using the file to restart the target server, change parameters like --tmpdir, --general-log, etc., and any global variable that uses an absolute filepath to avoid the accidental usage of the wrong file locations by the target server.
server-all.cnf	Contains values of all the global variables of the backed-up server. Use this file or server-my.cnf to start the target server for restoration.	During a copy-back or copy-back-and-apply-log operation, the server repository options values (e.g., --datadir, --innodb_data_home_dir, etc.) in the file are modified if the command makes changes to them through the command options. However, during an apply-incremental-backup operation, the values already saved in the file take precedence and they are not modified by the option values supplied through the command. Warning When using the file to restart the target server, change parameters like --tmpdir, --general-log, etc., and any global variable that uses an absolute filepath to avoid the accidental usage of the wrong file locations by the target server.

InnoDB Data

Data managed by the InnoDB storage engine is always backed up. The primary InnoDB-related data files that are backed up include the ibdata* files (which represent the system tablespace and possibly the data for some user tables), any .ibd files (which contains data from user tables created with the file-per-table setting enabled), and the data extracted from the ib_logfile* files (the redo log information representing changes that occur while the backup is running), which is stored in a new backup file ibbackup_logfile.

If you use the compressed backup feature, the .ibd files are renamed in their compressed form to .ibz files.

The files, as they are originally copied, form a raw backup that requires further processing before it is ready to be restored. You then run the apply step, which updates the backup files based on the changes recorded in the ibbackup_logfile file, producing a prepared backup. At this point, the backup data corresponds to a single point in time. The files are now ready to be restored to their original location, or for some other use, such as testing, reporting, or deployment as a replica.

To restore InnoDB tables to their original state, you must also have the corresponding .frm files along with the backup data. Otherwise, the table definitions could be missing or outdated if someone has run ALTER TABLE or DROP TABLE statements since the backup. By default, mysqlbackup automatically copies the .frm files during a backup operation and restores the files during a restore operation.

Data from MyISAM and Other Storage Engines

mysqlbackup also backs up the .MYD files, .MYI files, and the .frm files associated with the MyISAM tables. Files with other extensions that are backed up are shown in this list.

Note

While MySQL Enterprise Backup can back up non-InnoDB data (like MYISAM tables), the MySQL server to be backed up must support InnoDB (i.e., the backup process will fail if the server was started up with the --innodb=OFF or --skip-innodb option), and the server must contain at least one InnoDB table.

MyISAM tables and these other types of files cannot be backed up in the same non-blocking way as InnoDB tables can. They are backed up using the warm backup technique: changes to these tables are prevented while they are being backed up, possibly making the database unresponsive for a time, but no shutdown is required during the backup.

Note

To avoid concurrency issues during backups of busy databases, you can use the --only-innodb or --only-innodb-with-frm option to back up only InnoDB tables and associated data.

Generated Files Included in the Backup

The backup data includes some new files that are produced during the backup process. These files are used to control later tasks such as verifying and restoring the backup data. The files generated during the backup process include:

• meta/backup_create.xml: Lists the command line arguments and environment in which the backup was created.

• meta/backup_content.xml: Essential metadata for the files and database definitions of the backup data.

• backup-my.cnf: Records the crucial configuration parameters that apply to the backup. These configuration parameters are read by mysqlbackup during operations like apply-log to determine how the backup data is structured. These parameters are also checked during a restore operation for their compatibility with your target server's configuration.

• server-my.cnf: Contains values of the backed-up server's global variables that are set to non-default values.

• server-all.cnf: Contains values of all the global variables of the backed-up server.

For details about all the files in the backup directory, see Table 1.1, “Files in a MySQL Enterprise Backup Output Directory”.

Single-File Backups

Depending on your workflow, you might want to perform a single-file backup rather than a directory backup, which produces a separate file for every file on the original server instance. A single-file backup is easier to transfer to a different system, to compress, and to uncompress; it also helps to prevent the situation in which individual files that form parts of a backup are deleted by mistake. It is just as fast as a multi-file backup to do a full restore; restoring individual files can be slower than in a multi-file backup. For instructions, see Section 4.3.5, “Making a Single-File Backup”.