The client library — libmysqlclient — is part of the same source code repository as the server. It comes both as a dynamic link library (.so), also called a shared library, and as a static link library (.a). During the 5.7 development cycle, we’ve made several improvements to the library, which has resulted in a bump from ABI version 18 to version 20. This is the second in a series of blog posts focusing on the client library. If you want to read the whole series, go back to the first post, called The API, the whole API and nothing but the API.

Since I’m primarily a Linux user, the technical details in this post focuses on Linux/Unix like systems, but the general description is common to all operating systems.

API vs. ABI

The MySQL client application programming interface (API) is the data structures and functions in the client library and header files that application developers can use to send queries to servers, read the results, etc. The MySQL manual documents each of the more than 100 data structures and functions.

When discussing shared libraries, the application binary interface (ABI) is also important. While the API is the interface the developer has to adhere to when writing and compiling client applications, the ABI is the interface that must be maintained in order for binary compatibility between versions of a library. There are two classes of changes to library ABIs: compatible changes and incompatible changes.

For instance, if a new function is added to the library, both the API and the ABI are extended, but all existing applications will still compile, and you can even upgrade the library and run existing applications with the new library without recompiling them. This is an example of what’s called a compatible ABI change. The new version of the library is backward compatible with the old one.

However, if a function is deleted from the library, some applications will fail to compile because they expect the function to be there. If you try to replace the old version of the library with the new one, existing applications that depend on that function will not work. This is an incompatible ABI change.

The example above changes both the API and the ABI, but it is also possible to change the ABI in an incompatible way without changing the API. One way of doing that is to change the value of an existing constant that is given as a parameter to a function. In this case, the application will compile just fine, but if you try to swap libraries used by an application that hasn’t been recompiled, the constant will mean different things, so the ABI is incompatible while the API is unchanged.

ABI versioning

When linking statically, the ABI version isn’t important. The library becomes part of the application binary, and it won’t change unless you recompile and relink with another version of the library.

However, dynamic link libraries are loaded from a shared object file at run time. This means that the library can be updated without rebuilding the application. Hence, the application needs a way to make sure it’s loading a library with the correct ABI.

In order to handle incompatible ABI changes, the linker will need to know which ABI the application expects. This information is encoded in the soname (shared object name) of the library. The only thing the linker cares about is that there’s a different soname for each ABI. The common way of achieving this, is to include a version number in the name. E.g., the soname of the libmysqlclient library for MySQL 5.7 is “libmysqlclient.so.20”, where “20” is the ABI version.

The linker will record the soname of the library in the application binary at build time, and when the application is run, the dynamic linker will look for and load the library with the same soname.

The soname is actually all the information the linker needs, but the version number of a library usually contains more components to aid the human user.

Version 20.0.9

At the time of writing, the latest ABI version, or client library version, is 20.0.9. It is shipped with MySQL 5.7.9. The client library ABI is versioned in the same way as most other libraries. The version number has three parts: “major.minor.patch”.

Major

The major number signifies incompatible changes. So the bump in major number from MySQL 5.6 to 5.7 means that we’ve made changes to the ABI that may break applications if you try to replace the old library with the new one without recompiling your applications.

In particular, the change we did when we restricted the export of many undocumented symbols (I discuss this in the first post of this series) caused a major number bump. Even though the API wasn’t change by this (we only removed undocumented functions), the ABI was. This is encoded into the soname of the library, as described above, so old applications that haven’t been recompiled will still look for the old library.

We did make some API changes too, but they aren’t that big, so most applications will happily link with the new library if you recompile them. After recompiling, the applications will depend on “libmysqlclient.so.20”, and the linker is able to distinguish between applications that need the new versions and applications that still need older versions.

Minor

The minor number increases every time we make a compatible change, i.e., a change that only adds symbols. If an existing symbol is removed, or if parameters or behavior changes, it’s an incompatible change that requires a major number bump.

Since we’ve bumped the major number, we’re starting at minor number 0. We may increase this during the 5.7 lifecycle, e.g., if we need to provide a new symbol.

Unlike the major number, this part of the version number is not used by the linker. This means that applications compiled with libmysqlclient version 20.1.x will happily try to run with libmysqlclient version 20.0.9, even though it uses a new function added in 20.1.x. This lack of forward compatibility is not MySQL specific, but a property of the platform’s library versioning policies and common to all libraries.

Patch

The patch number is not used by the linker either. The number signifies that we’ve done something to the library, but we haven’t changed the ABI. This number will typically just increase for each version of MySQL we ship. The ABI version of the library that comes with MySQL 5.7.9 is 20.0.9, and in MySQL 5.7.10 (unless we make compatible ABI changes), the library will be 20.0.10, etc.

Many major versions at the same time

Since MySQL versions have a long support lifetime, we support several versions of the client library at the same time. MySQL 5.5 ships with libmysqlclient 18.0, and MySQL 5.6 with version 18.1. This means that it’s not possible to add symbols in the client library in 5.5 — that would cause a minor number bump, but 18.1 is already taken by the 5.6 library.

With libmysqlclient version 20 in MySQL 5.7, we can add symbols, and as explained in the first blog post in the series, we may have to do so to export more symbols. Another reason is if there’s a severe security bug that we can’t fix in a compatible way. In that case, we would have to add a new function that is secure and deprecate the old one. This would also require a minor number bump.

For us to be able to do those minor number bumps, MySQL 5.8 will have to ship with libmysqlclient version 21, even if it contains no incompatible changes from version 20. One could argue that bumping the number without making an incompatible change is bad, but in practice it’s a moot point since we almost always introduce an incompatible change to the library ABI in a new major MySQL release. 5.8 will probably not be an exception.

With this, we leave the API and the ABI, and in the next post we’ll cover changes in how client applications can and should be built.

Until then, if you have any questions or comments, please use the comment field below.