In MySQL 8.0.14 we introduced binary log encryption at rest. When enabled, this feature makes sure that binary log files  generated by the server are encrypted as soon as they hit persistent storage.

As of MySQL 8.0.17 transient files created by the server for capturing the changes that end up in the binary log stream are also encrypted. These files are only written into if the in-memory buffer of the capture component is exhausted. Typically, in a well tuned MySQL server, with a somewhat predictable workload running through it, this should never happen, or if it does it should seldom happen.

Let’s have a look at what happens under the hood.

Change Capture: Buffering

When the binary log is enabled, there are a couple of capture caches that are created whenever a new session is opened to the MySQL server. These caches live only within the context of the session, i.e., are session local, therefore they are destroyed and their resources freed when the session terminates. The purpose of these caches is to hold the changes produced inside the server for the given session. They hold binary log events.

The changes captured are produced by transactions and a change lives in the cache for as long as a the transaction that produces it lives. When a transaction terminates, i.e., commits or rolls back, the content is discarded and the caches reset. Then these caches are reused and they will hold the changes produced by the next transaction to be executed on the same session.

Now, if the changes produced by a transaction are so large that they exhaust the in-memory caches, then swapping will take place, and some changes will temporarily be written to disk. They are written into a file that is associated with these caches, and therefore follows the same lifecycle and rules of the in-memory caches. This means that once a transaction terminates the contents of the file are also discarded.

For those familiar with the source code, these caches are instances of IO_CACHE. The buffer size of the IO_CACHE used as capture caches is controlled through the the following system variables binlog_cache_size/binlog_stmt_cache_size.

Now, that we have explained how the capture procedure works in terms of storing the changes, it is clear what we have done in MySQL 8.0.17. We changed the capture procedure to also encrypt the changes, when swapping occurs and thus the changes are written to the temporary file. Let’s look at how it is this done.

Change Capture: Encrypting Temporary Files

We have seen that capture caches can spill to disk, in case the changes are too large for the configured cache. In that case, a temporary file will be used to store the changes that do not fit in memory. We have also established the lifecycle of the caches (and their corresponding temporary files). They live for as long as the session that they are associated with lives. Moreover, the caches exist only within the scope of the session and no other component accesses its contents.

This means that the secrets used to encrypt the temporary file do not need to be persisted as sessions do not outlive the session or the server itself and no other entity will require access to these secrets to be able to decode the contents of the cache (and potentially access the contents of the temporary file).

This translates into the fact that each cache will have its own randomly generated password and initialization vector. These live throughout the life time of a transaction and are changed whenever a transaction commits. This reduces the window that the same password and initialization vector are kept in-memory. The encryption algorithm will be the same used for binary log and relay log encryption. These exist only in memory and are never persisted anywhere.

Change Capture: Enabling Transient File Encryption

There is nothing to really highlight here. Encryption for temporary cache files will just follow the settings of the binlog_encryption system variable. If it is enabled, then temporary files encryption will be enabled as well. If it is disabled, then no encryption of temporary files will take place.

Since the temporary files encryption setting is checked and reset on each transaction, then if binlog_encryption changes between transactions, then this will change the setting of this functionality as well.

So, if you are using binary log encryption already, you will not have to do anything. That is simple and straightforward.

Conclusion

As of MySQL 8.0.17, even the temporary files used in certain cases while capturing changes are encrypted. If you are already using binary log encryption, there is nothing that you need to do to use this feature, as it will be enabled once you enable binlog_encryption. If you want to check the development worklog used to track this feature, feel free to have a look at WL#12079.

Enjoy!