Atomics (or GCC intrinsics) were first introduced in InnoDB (5.0) by a patch  from Mark Callaghan’s team at Google for mutexes and rw-locks. InnoDB code then was written in C.  When the code was ported to C++ ,  part of the 5.6 release, there was no C++ standard for atomics. Over time this led to a hodge lodge of code tweaked for different platforms, hardware, operating systems and compilers.

There were many things wrong with what we ended up with. This old code didn’t handle all types, the supported  types were hardcoded for a few platforms and compilers only, with fallback to mutexes for all other access. Memory ordering rules were difficult to reason about because some of the GCC intrinsics  were more strict than required and this made the rules unclear, we usually tried to take the more conservative approach. This would sometimes be at the cost of some performance. What is more important we did not have means to force developers to always access a given variable through atomic operation – the decision to use atomic or non-atomic access was taken independently for each access. Therefore there were some variables on which some operations were performed atomically and some not, causing to be a UndefinedBehaviour at best.

This change to move all atomic operations to the C++11 standard  has increased portability – we use well tested code from C++ standard and don’t need to reinvent the wheel anymore.

There are more advantages of this change, for example we didn’t have atomic operations for bool, and in rw_lock_t we used ulint for field waiters, for which only values 0 and 1 were used. Now it’s defined as std::atomic<bool>, which reduced the size of the rw_lock_t structure. This has the nice side effect off reducing the size of the meta data in the buffer pool so that we can use the buffer pool more efficiently.

As mentioned earlier, there were some challenges with variables accessed with both atomic and non-atomic operations. For example there was atomic usage of a MYSQL_SYSVAR srv_fatal_semaphore_wait_treshold which couldn’t just be changed to std::atomic. Therefore we have split the variable into two: a base regular SYSVAR variable and an atomic variable: srv_fatal_semaphore_wait_extend. So now our base value works as SYSVAR, while the temporatary changes done by the code are done atomically to the additional variable. While on the subject, we analyzed the code around this variable. We did some refactoring there, and it turned out that srv_fatal_semaphore_wait_extend simplified our code.

We also did some changes in monitor counters which you can observe via INFORMATION_SCHEMA.INNODB_METRICS. We had MONITOR_ATOMIC_INC and MONITOR_ATOMIC_DEC which used atomic increment/decrement. Now we changed mon_value in structure monitor_value_t from mon_type_t to std::atomic<mon_type_t>. But we had also MONITOR_INC/MONITOR_DEC and MONITOR_INC_VALUE/MONITOR_DEC_VALUE which also incremented/decremented mon_value non-atomically – we wanted it to be faster, and exact value wasn’t so important. So in this case we had to use .load() and .store() with std::memory_order_relaxed to leave the old behavior. Therefore we have:

This differs from MONITOR_VALUE(monitor).fetch_sub(value, std::memory_order_relaxed), because the latter will be executed atomically.
Using the opportunity while working on this area we also did some refactoring here, replacing some of #defines with inline functions.

This work will help us create more robust software across different platforms and not maintain additional code that is just available in the C++ standard.

Thank you for using MySQL!