11.1 Optimizing Backup Performance

This section describes the performance considerations for backing up a database with MySQL Enterprise Backup. When optimizing and tuning the backup procedure, measure both the raw performance (how long it takes the backup to complete) and the amount of overhead on the database server. When measuring backup performance, consider:

• The limits imposed by your backup procedures. For example, if you take a backup every 8 hours, the backup must take less than 8 hours to finish.

• The limits imposed by your network and storage infrastructure. For example, if you need to fit many backups on a particular storage device, you might use compressed backups, even if that made the backup process slower.

• The tradeoff between backup time and restore time. You might choose a set of options resulting in a slightly slower backup, if those options enable the restore to be much faster. See Section 11.2, “Optimizing Restore Performance” for performance information for the restore process.

Full or Incremental Backup

After taking a full backup, subsequent backups can be performed more quickly by doing incremental backups, where only the changed data is backed up. For an incremental backup, specify the --incremental or --incremental-with-redo-log-only option to mysqlbackup. See Section 16.7, “Incremental Backup Options” for information about these options. For usage instructions for the backup and apply stages of incremental backups, see Section 4.3.3, “Making a Differential or Incremental Backup”.

Compressed Backup

Compressing the backup data before transmitting it to another server involves additional CPU overhead on the database server where the backup takes place, but less network traffic and less disk I/O on the server that is the final destination for the backup data. Consider the load on your database server, the bandwidth of your network, and the relative capacities of the database and destination servers when deciding whether or not to use compression. See Section 4.3.4, “Making a Compressed Backup” and Section 16.6, “Compression Options” for information about creating compressed backups.

Compression involves a tradeoff between backup performance and restore performance. In an emergency, the time needed to uncompress the backup data before restoring it might be unacceptable. There might also be storage issues if there is not enough free space on the database server to hold both the compressed backup and the uncompressed data. Thus, the more critical the data is, the more likely that you might choose not to use compression: accepting a slower, larger backup to ensure that the restore process is as fast and reliable as possible.

Single-File Backups

The single-file backup by itself is not necessarily faster than the traditional type of backup that produces a directory tree of output files. Its performance advantage comes from combining different steps that you might otherwise have to perform in sequence, such as combining the backup data into a single output file and transferring it to another server. See Section 15.5, “Other Single-File Backup Operations” for the options related to single-file backups, and Section 4.3.1, “Making a Single-File Backup” for usage instructions.

InnoDB Configuration Options Settings

As discussed later, there are a number of reasons why you might prefer to run with the setting innodb_file_per_table=1.

Parallel Backup

mysqlbackup can take advantage of modern multicore CPUs and operating system threads to perform backup operations in parallel. See Section 16.10, “Performance / Scalability / Capacity Options” for the options to control how many threads are used for different aspects of the backup process. If you see that there is unused system capacity during backups, consider increasing the values for these options and testing whether doing so increases backup performance:

• When tuning and testing backup performance using a RAID storage configuration, consider the combination of option settings --read-threads=3 --process-threads=6 --write-threads=3. Compare against the combination --read-threads=1 --process-threads=6 --write-threads=1.

• When tuning and testing backup performance using a non-RAID storage configuration, consider the combination of option settings --read-threads=1 --process-threads=6 --write-threads=1.

• When you increase the values for any of the 3 “threads” options, also increase the value of the --limit-memory option, to give the extra threads enough memory to do their work.

• If the CPU is not too busy (less than 80% CPU utilization), increase the value of the --process-threads option.

• If the storage device that you are backing up from (the source drive) can handle more I/O requests, increase the value of the --read-threads option.

• If the storage device that you are backing up to (the destination drive) can handle more I/O requests, increase the value of the --write-threads option (not applicable to single-file backups, which always use a single write thread).

Depending on your operating system, you can measure resource utilization using commands such as top, iostat, sar, dtrace, or a graphical performance monitor. Do not increase the number of read or write threads once the system iowait value reaches approximately 20%.

MyISAM Considerations

Important

• Although mysqlbackup backs up InnoDB tables without interrupting database use, the final stage that copies non-InnoDB files (such as MyISAM tables and .frm files) temporarily puts the database into a read-only state, using the statement FLUSH TABLES WITH READ LOCK. For best backup performance and minimal impact on database processing:

  1. Do not run long SELECT queries or other SQL statements at the time of the backup run.

  2. Keep your MyISAM tables relatively small and primarily for read-only or read-mostly work.

  Then the locked phase at the end of a mysqlbackup run is short (maybe a few seconds), and does not disturb the normal processing of mysqld much. If the preceding conditions are not met in your database application, use the --only-innodb or --only-innodb-with-frm option to back up only InnoDB tables, or use the --no-locking option to back up non-InnoDB files. Note that MyISAM, .frm, and other files copied under the --no-locking setting cannot be guaranteed to be consistent, if they are updated during this final phase of the backup.

• For a large database, a backup run might take a long time. Always check that mysqlbackup has completed successfully, either by verifying that mysqlbackup returned exit code 0, or by observing that mysqlbackup has printed the text “mysqlbackup completed OK!”.

• Schedule backups during periods when no DDL operations involving tables are running. See Appendix B, Limitations of MySQL Enterprise Backup for restrictions on backups at the same time as DDL operations.

Network Performance

For data processing operations, you might know the conventional advice that Unix sockets are faster than TCP/IP for communicating with the database. Although the mysqlbackup command supports the options --protocol=tcp, --protocol=socket, and --protocol=pipe, these options do not have a significant effect on backup or restore performance. These processes involve file-copy operations rather than client/server network traffic. The database communication controlled by the --protocol option is low-volume. For example, mysqlbackup retrieves information about database parameters through the database connection, but not table or index data.

Data Size

If certain tables or databases contain non-critical information, or are rarely updated, you can leave them out of your most frequent backups and back them up on a less frequent schedule. See Section 16.8, “Partial Backup and Restore Options” for information about the relevant options, and Section 4.3.5, “Making a Partial Backup” for instructions about leaving out data from specific tables, databases, or storage engines. Partial backups are faster because they copy, compress, and transmit a smaller volume of data.

To minimize the overall size of InnoDB data files, consider enabling the MySQL configuration option innodb_file_per_table. This option can minimize data size for InnoDB tables in several ways:

• It prevents the InnoDB system tablespace from ballooning in size, allocating disk space that can afterwards only be used by MySQL. For example, sometimes huge amounts of data are only needed temporarily, or are loaded by mistake or during experimentation. Without the innodb_file_per_table option, the system tablespace expands to hold all this data, and never shrinks afterward.

• It immediately frees the disk space taken up by an InnoDB table and its indexes when the table is dropped or truncated. Each table and its associated indexes are represented by a .ibd file that is deleted or emptied by these DDL operations.

• It allows unused space within a .ibd file to be reclaimed by the OPTIMIZE TABLE statement, when substantial amounts of data are removed or indexes are dropped.

• It enables partial backups where you back up some InnoDB tables and not others, as discussed in Section 4.3.5, “Making a Partial Backup”.

• It allows the use of table compression for InnoDB tables.

In general, using table compression by having ROW_FORMAT=COMPRESSED decreases table sizes and increase backup and restore performance. However, as a trade-off, table compression can potentially increase redo log sizes and thus slow down incremental backups and restores, as well as apply-log operations. See How Compression Works for InnoDB Tables for details.

Avoid creating indexes that are not used by queries. Because indexes take up space in the backup data, unnecessary indexes slow down the backup process. (The copying and scanning mechanisms used by mysqlbackup do not rely on indexes to do their work.) For example, it is typically not helpful to create an index on each column of a table, because only one index is used by any query. Because the primary key columns are included in each InnoDB secondary index, it wastes space to define primary keys composed of numerous or lengthy columns, or multiple secondary indexes with different permutations of the same columns.

Advanced: Apply-Log Phase (for Directory Backups only)

If you store the backup data on a separate machine, and that machine is not as busy the machine hosting the database server, you can offload some postprocessing work (the apply-log phase) to that separate machine. Apply-log Operation

There is always a performance tradeoff between doing the apply-log phase immediately after the initial backup (makes restore faster), or postponing it until right before the restore (makes backup faster). In an emergency, restore performance is the most important consideration. Thus, the more crucial the data is, the more important it is to run the apply-log phase immediately after the backup. Either combine the backup and apply-log phases on the same server by specifying the backup-and-apply-log option, or perform the fast initial backup, transfer the backup data to another server, and then perform the apply-log phase using one of the options from Apply-log Operation.