For a long time, there have been complaints about deficiencies of the data dictionary of MySQL. Many have expressed a lack of love for FRM files, see Morgan’s blog post and Stewart Smith’s post MySQL Architecture.
We are now designing and implementing a new and improved data dictionary for MySQL, and some key design goals are:
- Store dictionary information in transactional storage. We will first focus on InnoDB, but other storage engines might follow
- Consolidate distributed dictionary information for the server into a unified dictionary
- Store all dictionary information in a uniform way, with uniform APIs for all dictionary objects
- Get rid of filesystem-property induced problems
- Make APIs cache-friendly, so caches can eventually be hidden behind the API
- Pave the way for transactional DDL. This is key for improved reliability of replication
- Implement Information Schema as views on dictionary tables, allowing optimization of I_S queries
- Provide an API for serializing dictionary information for ease of integration of other storage engines and for redundancy
- Add versioning of meta-data for assisting upgrades
- Provide an upgrade path from MySQL versions with “old” dictionary that does not require dump/restore
- Provide mechanisms for moving larger data sets around
At http://labs.mysql.com/ you can now preview the new dd in action. Gopal Shankar’s blog post explains what the labs release contains.
You can join Alexander Nozdrin at OOW14 where he will talk about New Data Dictionary: An Internal Server API That Matters.
Alexander will explain the new data dictionary architecture depicted below:
For further descriptions on the schema definitions see WL#6379. The worklog WL#6380 outlines API principles for unified handling of dictionary objects. Given the uniformity of dictionary objects, there will be a lot of common functions, and those are described in WL#7284. WL#6382 describes the API for Table objects.
Please note that all the descriptions are subject to minor changes, and might be updated.