Documentation Home
MySQL 5.7 Reference Manual
Related Documentation Download this Manual
PDF (US Ltr) - 37.5Mb
PDF (A4) - 37.5Mb
PDF (RPM) - 37.0Mb
EPUB - 10.5Mb
HTML Download (TGZ) - 10.3Mb
HTML Download (Zip) - 10.3Mb
HTML Download (RPM) - 9.0Mb
Eclipse Doc Plugin (TGZ) - 11.2Mb
Eclipse Doc Plugin (Zip) - 13.4Mb
Man Pages (TGZ) - 203.9Kb
Man Pages (Zip) - 309.1Kb
Info (Gzip) - 3.4Mb
Info (Zip) - 3.4Mb
Excerpts from this Manual

MySQL 5.7 Reference Manual  /  ...  /  Overview of Online DDL

15.13.1 Overview of Online DDL

Historically, many DDL operations on InnoDB tables were expensive. Many ALTER TABLE operations worked by creating a new, empty table defined with the requested table options and indexes, then copying the existing rows to the new table one-by-one, updating the indexes as the rows were inserted. After all rows from the original table were copied, the old table was dropped and the copy was renamed with the name of the original table.

MySQL 5.5, and MySQL 5.1 with the InnoDB Plugin, optimized CREATE INDEX and DROP INDEX to avoid the table-copying behavior. That feature was known as Fast Index Creation. MySQL 5.6 enhanced many other types of ALTER TABLE operations to avoid copying the table. Another enhancement allowed SELECT queries and INSERT, UPDATE, and DELETE (DML) statements to proceed while the table is being altered. In MySQL 5.7, ALTER TABLE RENAME INDEX was also enhanced to avoid table copying. This combination of features is now known as online DDL.

This mechanism also means that you can generally speed the overall process of creating and loading a table and associated indexes by creating the table without any secondary indexes, then adding the secondary indexes after the data is loaded.

Although no syntax changes are required in the CREATE INDEX or DROP INDEX commands, some factors affect the performance, space usage, and semantics of this operation (see Section 15.13.9, “Limitations of Online DDL”).

The online DDL enhancements in MySQL 5.6 improved many DDL operations that formerly required a table copy or blocked DML operations on the table, or both. Table 15.10, “Summary of Online Status for DDL Operations” shows the variations of the ALTER TABLE statement and shows how the online DDL feature applies to each.

With the exception of ALTER TABLE partitioning clauses, online DDL operations for partitioned InnoDB tables follow the same rules that apply to regular InnoDB tables. For more information, see Section 15.13.8, “Online DDL for Partitioned InnoDB Tables”.

  • The In-Place? column shows which operations permit the ALGORITHM=INPLACE clause.

  • The Rebuilds Table? column shows which operations rebuild the table. For operations that use the INPLACE algorithm, the table is rebuilt in place. For operations that do not support the INPLACE algorithm, the table copy method is used to rebuild the table.

  • The Permits Concurrent DML? column shows which operations are performed fully online. You can specify LOCK=NONE to assert that concurrent DML is permitted during the DDL operation. MySQL automatically permits concurrent DML when possible.

    Concurrent queries are permitted during all online DDL operations. You can specify LOCK=SHARED to assert that concurrent queries are permitted during a DDL operation. MySQL automatically permits concurrent queries when possible.

  • The Notes column provides additional information and explains exceptions and dependencies related to the Yes/No values of the other columns. An asterisk indicates an exception or dependency.

Table 15.10 Summary of Online Status for DDL Operations

OperationIn-Place?Rebuilds Table?Permits Concurrent DML?Only Modifies Metadata?Notes
CREATE INDEX, ADD INDEXYes*No*YesNoRestrictions apply for FULLTEXT indexes; see next row.
ADD FULLTEXT INDEXYes*No*NoNoAdding the first FULLTEXT index rebuilds the table if there is no user-defined FTS_DOC_ID column. Subsequent FULLTEXT indexes may be added on the same table without rebuilding the table.
ADD SPATIAL INDEXYesNoNoNo 
RENAME INDEXYesNoYesYesOnly modifies table metadata.
DROP INDEXYesNoYesYesOnly modifies table metadata.
OPTIMIZE TABLEYes*YesYesNoIn-place operation is not supported for tables with FULLTEXT indexes.
Set column default valueYesNoYesYesOnly modifies table metadata.
Change auto-increment valueYesNoYesNo*Modifies a value stored in memory, not the data file.
Add foreign key constraintYes*NoYesYesThe INPLACE algorithm is supported when foreign_key_checks is disabled. Otherwise, only the COPY algorithm is supported.
Drop foreign key constraintYesNoYesYesforeign_key_checks can be enabled or disabled.
Rename columnYes*NoYes*YesTo permit concurrent DML, keep the same data type and only change the column name. ALGORITHM=INPLACE is not supported for renaming a generated column.
Add columnYes*Yes*Yes*NoConcurrent DML is not permitted when adding an auto-increment column. Data is reorganized substantially, making it an expensive operation. ALGORITHM=INPLACE is supported for adding a generated virtual column but not for adding a generated stored column. Adding a generated virtual column does not require a table rebuild.
Drop columnYesYes*YesNoData is reorganized substantially, making it an expensive operation. ALGORITHM=INPLACE is supported for dropping a generated column. Dropping a generated virtual column does not require a table rebuild.
Reorder columnsYesYesYesNoData is reorganized substantially, making it an expensive operation.
Change ROW_FORMAT propertyYesYesYesNoData is reorganized substantially, making it an expensive operation.
Change KEY_BLOCK_SIZE propertyYesYesYesNoData is reorganized substantially, making it an expensive operation.
Make column NULLYesYesYesNoData is reorganized substantially, making it an expensive operation.
Make column NOT NULLYes*YesYesNoSTRICT_ALL_TABLES or STRICT_TRANS_TABLES SQL_MODE is required for the operation to succeed. The operation fails if the column contains NULL values. The server prohibits changes to foreign key columns that have the potential to cause loss of referential integrity. See Section 14.1.8, “ALTER TABLE Syntax”. Data is reorganized substantially, making it an expensive operation.
Change column data typeNo*YesNoNoVARCHAR size may be increased using online ALTER TABLE. See InnoDB Online DDL Column Properties for more information.
Add primary keyYes*YesYesNoData is reorganized substantially, making it an expensive operation. ALGORITHM=INPLACE is not permitted under certain conditions if columns have to be converted to NOT NULL. See Example 15.9, “Creating and Dropping the Primary Key”.
Drop primary key and add anotherYesYesYesNoData is reorganized substantially, making it an expensive operation.
Drop primary keyNoYesNoNoOnly ALGORITHM=COPY supports dropping a primary key without adding a new one in the same ALTER TABLE statement.
Convert character setNoYes*NoNoRebuilds the table if the new character encoding is different.
Specify character setNoYes*NoNoRebuilds the table if the new character encoding is different.
Rebuild with FORCE optionYes*YesYesNoUses ALGORITHM=INPLACE. ALGORITHM=INPLACE is not supported for tables with FULLTEXT indexes.
null rebuild using ALTER TABLE ... ENGINE=INNODBYes*YesYesNoUses ALGORITHM=INPLACE. ALGORITHM=INPLACE is not supported for tables with FULLTEXT indexes.
Set STATS_PERSISTENT, STATS_AUTO_RECALC, STATS_SAMPLE_PAGES persistent statistics optionsYesNoYesYesOnly modifies table metadata.
ALTER TABLE … ENCRYPTIONNoYesNoYes 

The following sections shows the basic syntax, and usage notes related to online DDL, for each of the major operations that can be performed with concurrent DML, in-place, or both:

Secondary Indexes

  • Create secondary indexes: CREATE INDEX name ON table (col_list) or ALTER TABLE table ADD INDEX name (col_list). (Creating a FULLTEXT index still requires locking the table.)

  • Drop secondary indexes: DROP INDEX name ON table; or ALTER TABLE table DROP INDEX name

Creating and dropping secondary indexes on InnoDB tables skips the table-copying behavior.

In MySQL 5.6 and higher, the table remains available for read and write operations while the index is being created or dropped. The CREATE INDEX or DROP INDEX statement only finishes after all transactions that are accessing the table are completed, so that the initial state of the index reflects the most recent contents of the table. Previously, modifying the table while an index is being created or dropped typically resulted in a deadlock that cancelled the INSERT, UPDATE, or DELETE statement on the table.

Column Properties

  • Set a default value for a column: ALTER TABLE tbl ALTER COLUMN col SET DEFAULT literal or ALTER TABLE tbl ALTER COLUMN col DROP DEFAULT

    The default values for columns are stored in the .frm file for the table, not the InnoDB data dictionary.

  • Changing the auto-increment value for a column: ALTER TABLE table AUTO_INCREMENT=next_value;

    Especially in a distributed system using replication or sharding, you sometimes reset the auto-increment counter for a table to a specific value. The next row inserted into the table uses the specified value for its auto-increment column. You might also use this technique in a data warehousing environment where you periodically empty all the tables and reload them, and you can restart the auto-increment sequence from 1.

  • Renaming a column: ALTER TABLE tbl CHANGE old_col_name new_col_name datatype

    When you keep the same data type and [NOT] NULL attribute, only changing the column name, this operation can always be performed online.

    You can also rename a column that is part of a foreign key constraint. The foreign key definition is automatically updated to use the new column name. Renaming a column participating in a foreign key only works with the in-place mode of ALTER TABLE. If you use the ALGORITHM=COPY clause, or some other condition causes the command to use ALGORITHM=COPY behind the scenes, the ALTER TABLE statement will fail.

  • Extending VARCHAR size using an in-place ALTER TABLE statement, as in this example:

    ALTER TABLE t1 ALGORITHM=INPLACE, CHANGE COLUMN c1 c1 VARCHAR(255);
    

    The number of length bytes required by a VARCHAR column must remain the same. For VARCHAR values of 0 to 255, one length byte is required to encode the value. For VARCHAR values of 256 bytes or more, two length bytes are required. As a result, in-place ALTER TABLE only supports increasing VARCHAR size from 0 to 255 bytes or increasing VARCHAR size from a value equal to or greater than 256 bytes. In-place ALTER TABLE does not support increasing VARCHAR size from less than 256 bytes to a value equal to or greater than 256 bytes. In this case, the number of required length bytes would change from 1 to 2, which is only supported by a table copy (ALGORITHM=COPY). For example, attempting to change VARCHAR column size from 255 to 256 using in-place ALTER TABLE would return an error:

    ALTER TABLE t1 ALGORITHM=INPLACE, CHANGE COLUMN c1 c1 VARCHAR(256);
    ERROR 0A000: ALGORITHM=INPLACE is not supported. Reason: Cannot change
    column type INPLACE. Try ALGORITHM=COPY.
    

    Decreasing VARCHAR size using in-place ALTER TABLE is not supported. Decreasing VARCHAR size requires a table copy (ALGORITHM=COPY).

Foreign Keys

  • Adding or dropping a foreign key constraint:

    ALTER TABLE tbl1 ADD CONSTRAINT fk_name FOREIGN KEY index (col1) REFERENCES tbl2(col2) referential_actions;
    ALTER TABLE tbl DROP FOREIGN KEY fk_name;
    

    Dropping a foreign key can be performed online with the foreign_key_checks option enabled or disabled. Creating a foreign key online requires foreign_key_checks to be disabled.

    If you do not know the names of the foreign key constraints on a particular table, issue the following statement and find the constraint name in the CONSTRAINT clause for each foreign key:

    show create table table\G
    

    Or, query the information_schema.table_constraints table and use the constraint_name and constraint_type columns to identify the foreign key names.

    You can also drop a foreign key and its associated index in a single statement:

    ALTER TABLE table DROP FOREIGN KEY constraint, DROP INDEX index;
    

If foreign keys are already present in the table being altered (that is, it is a child table containing any FOREIGN KEY ... REFERENCE clauses), additional restrictions apply to online DDL operations, even those not directly involving the foreign key columns:

  • An ALTER TABLE on the child table could wait for another transaction to commit, if a change to the parent table caused associated changes in the child table through an ON UPDATE or ON DELETE clause using the CASCADE or SET NULL parameters.

  • In the same way, if a table is the parent table in a foreign key relationship, even though it does not contain any FOREIGN KEY clauses, it could wait for the ALTER TABLE to complete if an INSERT, UPDATE, or DELETE statement caused an ON UPDATE or ON DELETE action in the child table.

Notes on ALGORITHM=COPY

Any ALTER TABLE operation run with the ALGORITHM=COPY clause prevents concurrent DML operations. Concurrent queries are still allowed. That is, a table-copying operation always includes at least the concurrency restrictions of LOCK=SHARED (allow queries but not DML). You can further restrict concurrency for such operations by specifying LOCK=EXCLUSIVE, which prevents DML and queries.

Concurrent DML but Table Copy Still Required

Some other ALTER TABLE operations allow concurrent DML but still require a table copy. However, the table copy for these operations is faster than it was in MySQL 5.5 and prior.

  • Adding, dropping, or reordering columns.

  • Adding or dropping a primary key.

  • Changing the ROW_FORMAT or KEY_BLOCK_SIZE properties for a table.

  • Changing the nullable status for a column.

  • OPTIMIZE TABLE

  • Rebuilding a table with the FORCE option

  • Rebuilding a table using a null ALTER TABLE ... ENGINE=INNODB statement

Maintaining CREATE TABLE Statements

As your database schema evolves with new columns, data types, constraints, indexes, and so on, keep your CREATE TABLE statements up to date with the latest table definitions. Even with the performance improvements of online DDL, it is more efficient to create stable database structures at the beginning, rather than creating part of the schema and then issuing ALTER TABLE statements afterward.

The main exception to this guideline is for secondary indexes on tables with large numbers of rows. It is typically most efficient to create the table with all details specified except the secondary indexes, load the data, then create the secondary indexes. You can use the same technique with foreign keys (load the data first, then set up the foreign keys) if you know the initial data is clean and do not need consistency checks during the loading process.

Whatever sequence of CREATE TABLE, CREATE INDEX, ALTER TABLE, and similar statements went into putting a table together, you can capture the SQL needed to reconstruct the current form of the table by issuing the statement SHOW CREATE TABLE table\G (uppercase \G required for tidy formatting). This output shows clauses such as numeric precision, NOT NULL, and CHARACTER SET that are sometimes added behind the scenes, which you may want to leave out when cloning the table on a new system or setting up foreign key columns with identical type.


User Comments
  Posted by Rolf Martin-Hoster on July 10, 2013
It should be noted that while compressed innodb tables are updated in place, this an ADD INDEX can still take quite a long time time.
Sign Up Login You must be logged in to post a comment.