WL#6416: InnoDB: Remove the use of *.isl files

In MySQL 5.6, InnoDB introduced *.isl (InnoDB Symbolic Link) files to locate data files for tables created with CREATE TABLE…DATA DIRECTORY. InnoDB refers to them as remote tablespaces, because they are not under the same data directory as ordinary file-per-table tablespaces.

We also added the InnoDB system tables SYS_TABLESPACES and SYS_DATAFILES to store the remote path. That path is not available during the REDO portion of recovery since the system tablespace and system tables are recovered at the same time as all the remote tablespaces.

The purpose of these *.isl files is:

1. Allow file discovery before DD metadata is available.
2. Allow a way to move a tablespace file while the server is offline, and update the DD metadata next time the table is accessed.

We plan to replace the InnoDB SYS_* tables with the Global Data Dictionary tables. It will no longer be possible to update the data dictionary tables from low-level InnoDB code, such as when opening a table. The SYS_DATAFILES table that was introduced in 5.6 was updated from low-level operations.

In the future, InnoDB will be able to find all tablespace files that were mentioned in the REDO log, so that recovery can complete without relying on *.isl files. However, fixing the portability issue requires that the *.isl be consulted during redo log processing if the original file location is not accessible.

A replacement of *.isl files for both use cases will be provided. The replacement mapping will only be consulted during server startup. So, there will not be any surprise updates of the data dictionary when tables are opened during normal server operation.

Note: There never was a way to move files around with SQL, except using a table-rebuilding ALTER TABLE (online operation since MySQL 5.6) or export/import.

1. InnoDB will no longer look for *.isl files.
2. InnoDB will no longer create, delete or rename *.isl files.
3. The capability to move data files while the server is offline is removed.

The purpose of these *.isl files was:

1. Allow file discovery before DD metadata is available.
2. Allow a way to move a tablespace file while the server is offline, and update the DD metadata next time the table is accessed.

The removal of all code related to the *.isl files cleans up SQL semantics. Until now, if a table was not in the InnoDB data dictionary cache, loading a table definition could trigger an update of the data dictionary table SYS_DATAFILES behind the scenes, based on the contents of the file system.

Now, SELECT will be a strictly read-only operation again.

Note: We will provide a replacement for this removed functionality. If any file names need to be rewritten, all of them would be updated at server startup, instead of doing it lazily when opening a table whose meta-data is not yet in the data dictionary cache.

Note: We will remove all InnoDB-private system tables, including SYS_TABLESPACE and SYS_DATAFILES. We will control the creation of data dictionary tables and their metadata on bootstrap, startup, and upgrade.

The following definitions will be removed:

• ib_extention::ISL
• DOT_ISL and the entry in dot_ext[]
• in fil_ibd_open(), the parameter fix_dict and the logic to choose one of 3 Datafile instances
• Datafile::same_filepath_as()
• virtual attributes in class Datafile
• class RemoteDatafile