For a long time, the MySQL development community and many others have wanted a server that worked without FRM files.  The motivation behind removing FRM files, and the design goals around new data dictionary, can be explored in more detail in the blog post by Ståle Deraas “A New Data Dictionary for MySQL”.

And now for the good news! We have a MySQL Labs Release ready with a preview of the new Data Dictionary!

What is in the first MySQL Data Dictionary labs release?

First of all, the FRM files are now gone. The MySQL server no longer creates FRM files, ever. The server stores table meta-data in the data dictionary tables which use the InnoDB storage engine. For more details on the schema definitions of data dictionary tables, see WL#6379. One of the advantages with the Data Dictionary being stored within the InnoDB storage engine is that the MySQL server can now use InnoDB indexes to efficiently search for database objects. In addition to that, it provides crash-safe dictionary operations now and paves the way for transactional operations in the future.

Secondly, we have improved the Information Schema implementation. The Information Schema tables are designed to be simple SQL VIEWs against the data dictionary tables. This eliminates the need for creating temporary tables for every Information Schema table used within a query, as it is done today in MySQL servers without the new Data Dictionary.

The third improvement is the new “Data Dictionary API” framework, which provides object oriented API layers for server, storage engines, and plugins to access the data dictionary.  We have implemented an internal API layer used by the MySQL server to get and update data dictionary information, and an external API layer designed for plugins and storage engines. You can see a diagram of the new API framework below.

You can also see other improvements — for example, InnoDB is now used for the meta-data tables like the ‘time zones’ table, various help tables, a ‘plugins’ table, and a ‘servers’ table in order to provide crash-safe dictionary operations. Dictionary tables are also now created by mysqld itself, removing the need for separate bootstrap operations.

We are working towards the key design goals outlined in Ståle Deraas’ blog post.

Given the fact that this is our very first Data Dictionary labs release, there are a number of issues worthy of special note:

• Some of the dynamic meta data provided in INFORMATION_SCHEMA may not be up-to-date, for instance columns like CARDINALITY, TABLE_ROWS, AUTO_INCREMENT, MAX_DATA_LENGTH, DATA_LENGTH, and DATA_FREE. Running the ANALYZE TABLE command just before the query would ensure that INFORMATION_SCHEMA has the latest values.
• INFORMATION_SCHEMA queries under the LOCK TABLES command are not recommended to be used in the labs release.
• Meta-data for columns within VIEWs are not provided through INFORMATION_SCHEMA tables and SHOW statements.
• Meta-data for temporary tables is not provided through the DESCRIBE command.
• Meta-data for foreign key constraints is not provided through INFORMATION_SCHEMA.
• We are working to make the presentation of Unicode strings better through the Information Schema.
• Collation and character set data is populated only on server install, and may not be changed on a later server restart.
• The preview is not suited for large data sets.
• Upgrading from any previous MySQL database version is not supported.
• The only platform supported for the preview is Linux.

We do not recommend using the labs release in a production environment, but rather to install it on a spare server so that you’re able to preview what’s coming and provide feedback. We look forward to your input! Please let us know if you encounter any bugs or have other general feedback.

For additional details, you can join Alexander Nozdrin at OOW14 where he will talk about “New Data Dictionary: An Internal Server API That Matters“. If you’re unable to make it in person, we will make the presentation available online shortly afterwards.

Thanks for using MySQL!

UPDATE: Alexander’s presentation is now available here!