Instrumentation interface

MySQL performance schema instrumentation interface.

Introduction

Any code can be instrumented with the performance schema, regardless of the kind of code used:

• code within the server implementation itself
• code within various plugins, including storage engines, compiled and delivered with the server (aka, static plugins)
• code within various plugins, including storage engines, compiled and delivered separately from the server (aka, dynamic plugins)
• code from components, either provided by MySQL or by third parties, compiled and delivered separately (and loaded dynamically).

The instrumentation interface consist of two layers:

• a raw ABI (Application Binary Interface) layer, that exposes the primitive instrumentation functions exported by the performance schema instrumentation
• an API (Application Programming Interface) layer, that provides many helpers for a developer instrumenting some code, to make the instrumentation as easy as possible.

In each case, the API may expand into different code paths, and use different ABI infrastructure, depending on how the code is built. In each case, the API exposed is consistent: instrumented source code is the same, regardless of where it is built.

The following sections details what happens technically for each type of code. The same instrumentation (a mutex) is used, to illustrate.

Non instrumented code

Assume some existing code using a mutex, for example for linux

#include <pthread.h>
 
pthread_mutex_t foo;
 
void do_something()
{
  pthread_mutex_lock(&foo);
  ...
  pthread_mutex_unlock(&foo);
}

MySQL already provides some wrappers, to make the code platform independent, as in

#include "thr_mutex.h"
 
my_mutex_t foo;
 
void do_something()
{
  my_mutex_lock(&foo);
  ...
  my_mutex_unlock(&foo);
}

Code using my_mutex_t is not instrumented for performance measurement.

Instrumentation for server code

To instrument the code, use a mysql_mutex_t instead. All the APIs provided are meant to be used as a simple replacement.

#include "mysql/psi/mysql_mutex.h"
 
mysql_mutex_t foo;
 
void do_something()
{
  mysql_mutex_lock(&foo);
  ...
  mysql_mutex_unlock(&foo);
}

The server code can be built with or without each kind of instrumentation.

If cmake is configured with -DDISABLE_PSI_MUTEX, the mutex instrumentation is not compiled. In this case, HAVE_PSI_MUTEX_INTERFACE is not defined, so that mysql_mutex_lock() expands into

static inline int
inline_mysql_mutex_lock(mysql_mutex_t *that,
                        const char *src_file [[maybe_unused]],
                        uint src_line [[maybe_unused]]
                        )
{
  int result;
 
  ... Non instrumented code ...
  result = my_mutex_lock(&that->m_mutex);
 
  return result;
}

In other words, when not building with instrumentation, mysql_mutex_lock() is simply replaced with the actual call to my_mutex_lock(),

When building with the mutex instrumentation enabled (the default), mysql_mutex_lock() expands into this sequence.

static inline int
inline_mysql_mutex_lock(mysql_mutex_t *that,
                        const char *src_file,
                        uint src_line)
{
  int result;
 
#ifdef HAVE_PSI_MUTEX_INTERFACE
  if (that->m_psi != NULL)
  {
    ... (a) Instrumentation start ...
    PSI_mutex_locker *locker;
    PSI_mutex_locker_state state;
    locker = PSI_MUTEX_CALL(start_mutex_wait)(
      &state, that->m_psi, PSI_MUTEX_LOCK, src_file, src_line);
 
    ... (b) Instrumented code ...
    result = my_mutex_lock(&that->m_mutex);
 
    ... (c) Instrumentation end ...
    if (locker != NULL)
    {
      PSI_MUTEX_CALL(end_mutex_wait)(locker, result);
    }
 
    return result;
  }
#endif
 
  ... Non instrumented code ...
  result = my_mutex_lock(&that->m_mutex);
 
  return result;
}

For a given instrumentation point in the API, the basic coding pattern used is:

• (a) notify the performance schema of the operation about to be performed.
• (b) execute the instrumented code.
• (c) notify the performance schema that the operation is completed.

An opaque "locker" pointer is returned by (a), that is given to (c). This pointer helps the implementation to keep context, for performances.

The following code fragment is annotated to show how in detail this pattern in implemented, when the instrumentation is compiled in statically. In this case, the PSI_MUTEX_CALL macro expands to a function call, which results in

static inline int
inline_mysql_mutex_lock(mysql_mutex_t *that,
                        const char *src_file,
                        uint src_line)
{
  int result;
 
#ifdef HAVE_PSI_MUTEX_INTERFACE
  if (that->m_psi != NULL)
  {
    ... (a) Instrumentation start ...
    PSI_mutex_locker *locker;
    PSI_mutex_locker_state state;
    locker = pfs_start_mutex_wait_v1(
      &state, that->m_psi, PSI_MUTEX_LOCK, src_file, src_line);
 
    ... (b) Instrumented code ...
    result = my_mutex_lock(&that->m_mutex);
 
    ... (c) Instrumentation end ...
    if (locker != NULL)
    {
      pfs_end_mutex_wait_v1(locker, result);
    }
 
    return result;
  }
#endif
 
  ... Non instrumented code ...
  result = my_mutex_lock(&that->m_mutex);
 
  return result;
}

When the performance schema is disabled at runtime, the m_psi pointer is NULL, so the extra overhead is one if statement.

When the performance schema is enabled at runtime, the m_psi pointer points to the actual instrumentation added to the mutex, and two extra calls are present on the code path:

• a call to pfs_start_mutex_wait_v1() before the operation
• a call to pfs_end_mutex_wait_v1() after the operation

Both these calls are static, and point directly to the performance_schema implementation.

Instrumentation for a plugin

Inside a plugin, PSI_MUTEX_CALL expands to a call using a function pointer. This is to decouple the plugin binary from the implementation, to avoid having hard coded references to symbols like pfs_start_mutex_wait_v1() from the plugin library (either shared or static).

The plugin library, however, still depends on a symbol to be provided by the server, in this case psi_mutex_service.

static inline int
inline_mysql_mutex_lock(mysql_mutex_t *that,
                        const char *src_file,
                        uint src_line)
{
  int result;
 
#ifdef HAVE_PSI_MUTEX_INTERFACE
  if (that->m_psi != NULL)
  {
    ... (a) Instrumentation start ...
    PSI_mutex_locker *locker;
    PSI_mutex_locker_state state;
    locker = psi_mutex_service->start_mutex_wait(
      &state, that->m_psi, PSI_MUTEX_LOCK, src_file, src_line);
 
    ... (b) Instrumented code ...
    result = my_mutex_lock(&that->m_mutex);
 
    ... (c) Instrumentation end ...
    if (locker != NULL)
    {
      psi_mutex_service->end_mutex_wait(locker, result);
    }
 
    return result;
  }
#endif
 
  ... Non instrumented code ...
  result = my_mutex_lock(&that->m_mutex);
 
  return result;
}

This solution works overall, but has some limitations.

Instrumentation for component

The goal when compiling a component is to have zero linker dependencies on the server code, and have technical dependencies described as references to services instead.

Another goal is to compile code the same way, independently of the technical choices affecting the implementation (pthread mutexes or native windows critical sections, SAFE_MUTEX debug helpers or not, etc)

For the mutex instrumentation in particular, the pattern used is to have a service that delegates all decisions to the server code. For this reason, all the parameters that might or might not be needed depending on the ultimate implementation (such as FILE and LINE for SAFE_MUTEX) are provided in all cases.

In a component, the code is instrumented using a different header file, everything else is the same.

#include "mysql/component/services/mysql_mutex.h"
 
mysql_mutex_t foo;
 
void do_something()
{
  mysql_mutex_lock(&foo);
  ...
  mysql_mutex_unlock(&foo);
}

The call to mysql_mutex_lock() expands into

REQUIRES_SERVICE_PLACEHOLDER(mysql_mutex_v1);
 
mysql_service_mysql_mutex_v1->lock(M, __FILE__, __LINE__);

In the component metadata, the dependency on the mutex service needs to be declared explicitly, as in:

BEGIN_COMPONENT_REQUIRES(pfs_example)
  REQUIRES_SERVICE(mysql_mutex_v1)
END_COMPONENT_REQUIRES();

When the component is loaded:

• the loader inspects the COMPONENT_REQUIRES metadata, and find the service dependencies
• service dependencies are resolved, which assigns a proper value to the mysql_service_mysql_mutex_v1 pointer.

From this point, code compiled in the component can be executed normally, and calls to mysql_mutex_lock() land into the service implementation, which is a simple instrumented mutex lock compiled in the server.

int
impl_mysql_mutex_lock(
  mysql_mutex_t *that,
  const char *src_file,
  unsigned int src_line)
{
  return mysql_mutex_lock_with_src(that, src_file, src_line);
}

List of available instrumentation

The current list of instrumentation provided by the performance schema is as follows:

List of available instrumentation

Scope	User code		Implementation entry point
	Server or Plugin	Component	Server or Plugin	Component
Mutex (high level)	#include "mysql/psi/mysql_mutex.h" mysql_mutex_lock()	#include "mysql/components/services/mysql_mutex.h" mysql_mutex_lock()	PSI_mutex_bootstrap	REQUIRES_MYSQL_MUTEX_SERVICE
Rwlock (high level)	#include "mysql/psi/mysql_rwlock.h" mysql_rwlock_rdlock()	#include "mysql/components/services/mysql_rwlock.h" mysql_rwlock_rdlock()	PSI_rwlock_bootstrap	REQUIRES_MYSQL_RWLOCK_SERVICE
Conditions (high level)	#include "mysql/psi/mysql_cond.h" mysql_cond_wait()	#include "mysql/components/services/mysql_cond.h" mysql_cond_wait()	PSI_cond_bootstrap	REQUIRES_MYSQL_COND_SERVICE
Mutex (low level)	#include "mysql/psi/psi_mutex.h" PSI_MUTEX_CALL(start_mutex_wait)(...) PSI_MUTEX_CALL(end_mutex_wait)(...)	#include "mysql/components/services/psi_mutex.h" PSI_MUTEX_CALL(start_mutex_wait)(...) PSI_MUTEX_CALL(end_mutex_wait)(...)	PSI_mutex_bootstrap	REQUIRES_PSI_MUTEX_SERVICE
Rwlock (low level)	#include "mysql/psi/psi_rwlock.h" PSI_RWLOCK_CALL(start_rwlock_rdwait)(...) PSI_RWLOCK_CALL(end_rwlock_rdwait)(...)	#include "mysql/components/services/psi_rwlock.h" PSI_RWLOCK_CALL(start_rwlock_rdwait)(...) PSI_RWLOCK_CALL(end_rwlock_rdwait)(...)	PSI_rwlock_bootstrap	REQUIRES_PSI_RWLOCK_SERVICE
Conditions (low level)	#include "mysql/psi/mysql_cond.h" PSI_COND_CALL(start_cond_wait)(...) PSI_COND_CALL(end_cond_wait)(...)	#include "mysql/components/services/psi_cond.h" PSI_COND_CALL(start_cond_wait)(...) PSI_COND_CALL(end_cond_wait)(...)	PSI_cond_bootstrap	REQUIRES_PSI_COND_SERVICE
Errors	#include "mysql/psi/psi_error.h" PSI_ERROR_CALL(log_error)(...)	#include "mysql/components/services/psi_error.h" PSI_ERROR_CALL(log_error)(...)	PSI_error_bootstrap	REQUIRES_PSI_ERROR_SERVICE
File	#include "mysql/psi/psi_file.h" PSI_FILE_CALL(start_file_wait)(...) PSI_FILE_CALL(end_file_wait)(...)	#include "mysql/components/services/psi_file.h" PSI_FILE_CALL(start_file_wait)(...) PSI_FILE_CALL(end_file_wait)(...)	PSI_file_bootstrap	REQUIRES_PSI_FILE_SERVICE
Idle	#include "mysql/psi/psi_idle.h" PSI_IDLE_CALL(start_idle_wait)(...) PSI_IDLE_CALL(end_idle_wait)(...)	#include "mysql/components/services/psi_idle.h" PSI_IDLE_CALL(start_idle_wait)(...) PSI_IDLE_CALL(end_idle_wait)(...)	PSI_idle_bootstrap	REQUIRES_PSI_IDLE_SERVICE
Metadata locks	#include "mysql/psi/psi_mdl.h" PSI_METADATA_CALL(start_metadata_wait)(...) PSI_METADATA_CALL(end_metadata_wait)(...)	#include "mysql/components/services/psi_mdl.h" PSI_METADATA_CALL(start_metadata_wait)(...) PSI_METADATA_CALL(end_metadata_wait)(...)	PSI_mdl_bootstrap	REQUIRES_PSI_MDL_SERVICE
Memory	#include "mysql/psi/psi_memory.h" PSI_MEMORY_CALL(memory_alloc)(...) PSI_MEMORY_CALL(memory_free)(...)	#include "mysql/components/services/psi_memory.h" PSI_MEMORY_CALL(memory_alloc)(...) PSI_MEMORY_CALL(memory_free)(...)	PSI_memory_bootstrap	REQUIRES_PSI_MEMORY_SERVICE
Socket	#include "mysql/psi/psi_socket.h" PSI_SOCKET_CALL(start_socket_wait)(...) PSI_SOCKET_CALL(end_socket_wait)(...)	#include "mysql/components/services/psi_socket.h" PSI_SOCKET_CALL(start_socket_wait)(...) PSI_SOCKET_CALL(end_socket_wait)(...)	PSI_socket_bootstrap	REQUIRES_PSI_SOCKET_SERVICE
Stage	#include "mysql/psi/psi_stage.h" PSI_STAGE_CALL(start_stage)(...)	#include "mysql/components/services/psi_stage.h" PSI_STAGE_CALL(start_stage)(...)	PSI_stage_bootstrap	REQUIRES_PSI_STAGE_SERVICE
Statement	#include "mysql/psi/psi_statement.h" PSI_STATEMENT_CALL(start_statement)(...) PSI_STATEMENT_CALL(end_statement)(...) Extra helpers available: MYSQL_START_STATEMENT() MYSQL_END_STATEMENT()	#include "mysql/components/services/psi_statement.h" PSI_STATEMENT_CALL(start_statement)(...) PSI_STATEMENT_CALL(end_statement)(...)	PSI_statement_bootstrap	REQUIRES_PSI_STATEMENT_SERVICE
Table	#include "mysql/psi/psi_table.h" PSI_TABLE_CALL(start_table_io_wait)(...) PSI_TABLE_CALL(end_table_io_wait)(...)	#include "mysql/components/services/psi_table.h" PSI_TABLE_CALL(start_table_io_wait)(...) PSI_TABLE_CALL(end_table_io_wait)(...)	PSI_table_bootstrap	REQUIRES_PSI_TABLE_SERVICE
System	#include "mysql/psi/psi_system.h" PSI_SYSTEM_CALL(plugin_unload)(...)	#include "mysql/components/services/psi_system.h" PSI_SYSTEM_CALL(plugin_unload)(...)	PSI_system_bootstrap	REQUIRES_PSI_SYSTEM_SERVICE
Thread	#include "mysql/psi/psi_thread.h" PSI_THREAD_CALL(spawn_thread)(...) PSI_THREAD_CALL(delete_thread)(...)	#include "mysql/components/services/psi_thread.h" PSI_THREAD_CALL(spawn_thread)(...) PSI_THREAD_CALL(delete_thread)(...)	PSI_thread_bootstrap	REQUIRES_PSI_THREAD_SERVICE
Transaction	#include "mysql/psi/psi_transaction.h" PSI_TRANSACTION_CALL(start_transaction)(...) PSI_TRANSACTION_CALL(end_transaction)(...)	#include "mysql/components/services/psi_transaction.h" PSI_TRANSACTION_CALL(start_transaction)(...) PSI_TRANSACTION_CALL(end_transaction)(...)	PSI_transaction_bootstrap	REQUIRES_PSI_TRANSACTION_SERVICE
Data locks	#include "mysql/psi/psi_data_lock.h" PSI_DATA_LOCK_CALL(register_data_lock)(...) PSI_DATA_LOCK_CALL(unregister_data_lock)(...)	Not available as a service.	PSI_data_lock_bootstrap	Not available as a service.
TLS Channels	#include "mysql/psi/psi_tls_channel.h" PSI_TLS_CHANNEL_CALL(register_tls_channel)(...) PSI_TLS_CHANNEL_CALL(unregister_tls_channel)(...)	#include "mysql/components/services/psi_tls_channel.h" PSI_TLS_CHANNEL_CALL(register_tls_channel)(...) PSI_TLS_CHANNEL_CALL(unregister_tls_channel)(...)	PSI_tls_channel_bootstrap	REQUIRES_PSI_TLS_CHANNEL_SERVICE