WL#6606: Offload THD initialization and network initialization to worker thread
Affects: Server-5.7
—
Status: Complete
Initialization of THD and it's vio/net initialization happens in the acceptor thread that accepts the connection. THD and network initialization involves acquiring locks, memory allocation of various structures and system calls which are compute-bound as well tasks that may block. The acceptor thread is basically a event loop that waits for new connection events from clients. To maximize the number of connections that can be handled per unit of time, the acceptor thread should spend as much of it's time listening for new connections. This means that the thd initialization should be offloaded from the accept event loop and delegated to worker threads that handle the client connections. This worklog should evolve a generic framework which offloads THD initialization and net/vio initialization to worker for all types of communication channels (shared memory, named pipes and sockets) that client connects with the server. In addition, this worklog will refactor the existing interfaces of the struct scheduler_functions into an object-oriented API and refactor necessary relevant code that concerns it. User Documentation ================== http://dev.mysql.com/doc/relnotes/mysql/5.7/en/news-5-7-2.html
EXISTING DESIGN OF THD INITIALIZATION AND SCHEDULER (CONNECTION HANDLERS) INTERFACES IN CURRENT MYSQL SERVER CODE Clients connect to the mysql server using TCP/IP, shared memory and named pipes. Shared memory and named pipes are available on windows non-embedded servers. TCP/IP, shared memory and named pipes represents different types of communication channel that client uses to connect to the server. In the current server instance, acceptor thread is spawned and this thread listens for connections from the clients. As soon as a connection arrives, THD object is instantiated and intialized. The THD class contains various types of information that pertain to the connection state, thread state and sql state and various other globals including NET structure. THD inherits from classes MDL_context, Statement and Open_tables_state. The THD class contains a member variable NET struct that abstracts out the various client communication mechanisms. The NET struct contains a member variable vio which points to a struct that contains function pointers for various I/O depending on the type of channel (TCP/IP, Shared Memory and Named Pipes). Initialiazing the vio member of NET structure involve allocating a new vio struct using UNIX library call malloc and initializing various member variables that constitutes it. Given the above, THD instantiation and initialization of various component members becomes a task that is time consuming and compute bound and is a bottleneck that limits the connection throughput of the mysqld server. Once the THD instantiation and intialization is completed, add connection callback of appropriate type of connection handler (scheduler) is called. The connection handlers in mysql server to handle connections from clients are of three types: One thread connection handler: - The client connection is handled in the same thread that received the connection. This type of conncection handler is being used for debugging purposes. Thread per connection handler: - Each client connection is being handled in a separate thread. This is the default connection that is used in open source mysql. Thread Pool connection handler: - This handler which is available in the enterprise edition of mysql server uses dynamic pool of threads to manage the client connections. The disadvantage with the one thread per connection scheduler is when large number of simultaneous connections are being made, the system will spend most of it’s time in context switching. This limitation is overcome by the thread pool scheduler. The type of connection handlers to use at runtime can be specified by the use of mysqld option thread_handling and the valid values this option takes are no-threads, one-thread-per-connection and dynamically-loaded (the last one relates to the thread pool scheduler which can be loaded as a dynamic plugin by specifying the path to the .so file using –plugin-load option). The add connection callback of the thread per connection connection handler is responsible for creating a worker thread and passing the THD object to the thread start function which handles the client connection. In case if there are idle worker threads that are cached by the thread cache subsystem, the THD object is pushed on to a FIFO queue of waiting THDs and an idle thread is woken up to pick the THD from the queue and handle the client connection (instead of creating a new worker thread in the callback add connection). Once all of this is done, the acceptor loops back and waits for a new connection to arrive. The amount of time before the acceptor tries to come back and handle another connection should be as minimal as possible for the server to improve its connection throughput (the number of connections per unit of time). All connection handlers implement a set of functions that are used as callbacks from various points in the code and these functions are registered as function pointers with the struct scheduler_functions and the definition of this structure is: struct scheduler_functions { uint max_threads; bool (*init)(void); bool (*init_new_connection_thread)(void); void (*add_connection)(THD* thd); void (*thd_wait_begin)(THD* thd, int wait_type); void (*thd_wait_end)(THD* thd); void (*post_kill_notification)(THD* thd); bool (*end_thread)(THD* thd, bool cache_thread); void (*end)(void); }; The various fields of the above struct are: max_threads:- This indicates the max number of threads that can be spawned. This is used with respect to the thread per connection handler (which indicates max. number of connections) and thread pool connection handler. (As such during refactoring as proposed in the design, it won’t be part of the base connection handler class. This will be used only in the thread pool scheduler and thread per connection scheduler accordingly.) init: - The init handler is currently registered by the thread pool handler and it initializes and sets up the thread pool connection handler. init_new_connection_thread:- This callback is registered by all the connection handlers and is used to initialize thread specific session parameters. Since there is no variation of this method specific to each connection handler and this is something to thread related setup, the plan would be to remove this method as part of the connection handler class that is to be introduced. add_connection:- It is the main callback that is registered by the various connection handlers and it provides the main functionality of what the connection handler does. thd_wait_begin:- This is a callback that is implemented by the thread pool connection handler. When some parts of the mysql server code is going into a wait state (such as I/O that may block), they call this callback to inform the thread pool connection handler. thd_wait_end:- Once the wait (such as from an I/O block, sleep, timeout etc) is over, this callback is called. This callback is also implemented by the thread pool scheduler. post_kill_notification:- Once kill connection has been done, post_kill_notification callback is called. This callback is used to ensure that a thread kill has been notified. [In the new design, since the above three functions are not per say related to connection handling functionalities and rather than specific events [start of a wait state, end of a wait state and post kill that certain connection handlers (like thread pool scheduler are interested), they would be removed from Connection_handler (see Design section) class. This functions need to be called from C file like mysys and have C linkage interface and external programs may use this, current functionality for the above three interfaces would be provided via function pointers.] end_thread:- This is used to release a connection (which involve deallocating THD and release of other structures and cache the thread in thread cache if we use thread per connection handler.) end: - This is used to deinitialize and end a connection handler. This callback is currently being used by the thread pool connection handler. (NOTE: The scheduler interface is exposed as a C API service (in file include/mysql/service_thread_scheduler.h and includes struct scheduler_function) and exposes APIs which are only used by the thread pool plugin scheduler. The plan would be to convert the service provided by this into C++ API and expose it with the refactoring changes and bump up the service version number accordingly.) The sequence flows pertaining to the connection accept and subsequent thd instantiation and initialization and dispatching the client connection to be handled by a worker thread for TCP/IP socket with reference to the existing server code is provided the attachment Design Document.Doc with this worklog. DESIGN OVERVIEW The goal of this worklog is two-fold. The worklog will refactor the initial connection handling code paths and connection handler (scheduler) interfaces into an object-oriented API. The other goal is to avoid thd and vio/net instantiation & initialization in the acceptor thread and offload this phase to worker threads (one thread per connection handler). (Each type of scheduler implementations would be moved into separate files including platform specific implementations as well and this would/may require cmake modification and avoid most of #ifdef definitions in the code as well.) Please refer the Design Document.Doc for the overall class diagram. The overall design will have a Connection_handler base class that will have the virtual functions init, deinit, add_connection and remove_connection. The different types of connections handlers that we have would be abstracted by the classes One_thread_connection_handler, Per_thread_connection_handler and Thread_pool_connection_handler. These classes inherit from the Connection_handler base class and implement the pure virtual function add_connection and remove_connection and override default implementation of the init and deinit methods. A singleton class Connection_handler_mgr is introduced whose provides an API to set the connection handler (which would be done at the time of mysqld startup sequence) and provide an API to get the current connection handler. The Connection_handler_mgr class also provides generic connection related API like checking if number has exceeded the connection limits. Access to connection handler is hidden behind this manager class because it provides for decoupling and extensibility of the code and this provides in ease in handling thread pool connection handler which comes into existence after plugin initialization (changing the current existing connection handler)(In future this provides for an extensibility to implement scenarious like multiple connection handlers serving on different ports etc. with minimal code changes etc.) The data model and design detail of this classes and their use in different scenarios/sequences would be discussed in detail in the design detail section of this document. In terms of offloading the thd initialization/vio&net instantiation, there should be minimal stub information which is required to be passed to a worker thread for instantiating & initializing a thd object. The stub information includes the client connection descriptor that varies across different types of channel (TCP/IP, Shared Memory and Named Pipes). The thd creation and initialization and especially the creation of NET struct in it is specific to each type of channel. Whenever any failure occurs, minimal error information needs to be propagated and the client connection needs to be aborted which means the channel needs to be closed or shutdown. Error propagation can occur very early in the connection phase which will result in a connection being aborted. This calls for defining a Channel_info interface class which has pure virtual functions create_thd and shutdown_channel. The classes Channel_info_socket, Channel_info_shared_mem and Channel_info_named_pipe which implements the Channel_info interface is created. This has state information pertaining to each type of client that communicated with the mysql. Please refer the design doc attached for the class diagram of the Channel_info hierarchy. With the new framework, the acceptor thread instantiates an object that implements the interface Channel_info and passes this to the add connection callback which subsequently gets passed to the start routine of the worker thread. The worker thread needs to use the interface function create_thd of the passed object and in the case of error it can call shutdown_channel to close the type of channel. It is to be noted that Channel_info instance would be created in the acceptor thread and the worker threads will take ownership of the Channel_info instance and is responsible for freeing up this memory once it instantiates the THD object based on the information contained in the Channel_info object.
DESIGN DETAIL Data Model The detailed class hierarchy is attached in the design doc. and the internal details of the class hierarchy are as described below: Connection_handler class The Connection_handler class represents an abstract base class from which different types of connection handlers inherit and implement the pure virtual functions. Its definition is as follows: class Connection_handler { public: virtual ~Connection_handler(); virtual void init(); virtual void add_connection(Channel_info* channel_info) = 0; virtual void release_connection(THD* thd) = 0; virtual void deinit(); }; The methods in it are: init: - The Connection_handler_base class provides a default empty body implemention for this method. The connection handlers One_thread_connection_handler and Per_thread_connection_handler do not override this default implementation and Thread_pool_connection_handler provides a implementation of this method. (In the current, the init function in scheduler_functions struct is set to NULL for one thread connection and per thread connection handler]. The thread pool connection handler will override this virtual function and provide an implementation (thd_pool_init in the current code.) add_connection:- The add_connection method is a pure virtual function and all the connection handlers implement this method. It does the core work that distinguishes the different types of connection handlers. (It provides the functionality provided by the add_connection function pointer member of the struct scheduler_functions.) release_connection:- This method is a pure virtual function and all the connection handlers implement this interface. This method is responsible for performing the functionality related to release of the connection which includes deallocating connection specific structures like THD, decreasing connection count, removing THD from global list etc. deinit: - The Connection_handler base class provides a default empty body implementation of this method. The connection handlers One_thread_connection_handler and Per_thread_connection_handler do not override this default implementation. Thread pool overrides this default implementationOne_thread_connection_handler class 4.1.2 One_thread_connection_handler class One_thread_connection handler class is responsible for implementing the functionality that all connections are being handled in a single worker thread. This is currently there in the code primarily for debugging purposes. Its class definition is as follows: class One_thread_connection_handler : public Connection_handler { private: One_thread_connection_handler(const One_thread_connection_handler&); One_thread_connection_handler& operator=(const One_thread_connection_handler&); public: One_thread_connection_handler(); virtual void add_connection(Channel_info* channel_info); virtual void release_connection(THD* thd); }; The various methods in it are: add_connection:- The add_connection of one_thread_connection_handler class instantiate a new THD based on Channel_info and starts the connection authentication and subsequent query handling loop to handle queries from the client. The setting of thread specific session variables and THD allocation can fail and this method will send ER_OUT_OF_RESORUCES error and the connection would be aborted. release_connection:- The release connection method release the resources associated with the THD object and decrement the connection count and remove the THD from global list and delete thd object associated with it. Also note that the assignment operator and the copy constructor have been declared private to make this class non-copyable. Per_thread_connection_handler Per_thread_connection_handler class encapsulates the functionality of a thread being created for every new connection that mysql server needs to handle. The class definition for Per_thread_connection_handler is as below: class Per_thread_connection_handler : public Connection_handler { private: uint m_max_thread; Per_thread_connection_handler(const Per_thread_connection_handler&); Per_thread_connection_handler& operator= (Per_thread_connection_handler&); public: Per_thread_connection_handler(uint max_threads, bool use_thread_cache); virtual void add_connection(Channel_info* channel_info); virtual void release_connection(THD* thd); }; The various member variables and methods in the Per_thread_connection_handler class are: m_max_threads:- This attribute of the Per_thread_connection_handler indicates the max number of thread that would be spawned to handle the client connections. Since in the Per_thread_connection_handler, there is 1:1 mapping between the client connections and the threads, m_max_threads attribute indicate the maximum number of connections that could be made and this value is got from the max_connections argyument that is specified when mysqld is started. add_connection:- The add_connection implement connection handling pertaining to the Per_thread_connection handler class. If thread cache is available and there are some idle worker threads, channel_info is queued on to a queue that is being maintained by the thread cache and the thread cache will assign a worker thread to service the client connection. The add_connection method can fail because of the instantiation of Channel_info object in which the case the error ER_OUT_OF_RESOURCES would be sent to the client and the connection would be aborted. release_connection:- This is responsible for releasing the connection. The connection count is decremented and the THD object is destroyed. If the thread cache is used (m_thread_cache pointer is not NULL), then the thread is returned to idle pool maintained by the thread cache else the thread specific parameters are reset and released and the thread dies. The copy constructor and the assignment operator have been declared private since this class has to be non copyable. Thread_pool_handler class Thread_pool_handler class provides the functionality of assigning a dynamic pool of threads to manage the client connections. The class definition for Thread_pool_handler class is as follows: class Thread_pool_connection_handler : public Connection_handler { private: uint m_max_threads; Thread_pool_connection_handler(const Per_thread_connection_handler&); Thread_pool_connection_handler& operator=(Per_thread_connection_handler&); Thread_pool_connection_handler(uint max_threads); virtual void init(); virtual void add_connection(Channel_info* channel_info); virtual void release_connection(THD* thd); virtual void deinit(); }; The member variables and various methods in it are: m_max_threads:- It represents the maximum number of threads that the thread pool is allowed to create. It is set to the maximum number of connections plus the maximum of thread group size (which is 128) in the current code. init: - The thread pool connection handler will redefine the empty body init method provided by the Connection handler class. This method will have the same semantics as thd_pool_init method which is initialized to the init callback of struct thread_scheduler in the current implementation.This method initializes the mutexes & condition variables of stall timer thread and of each thread group and their member variables.and creates and starts the background monitoring thread. add_connection:- The add_connection method for the thread pool will have the same semantics as the thd_pool_add_connection in the current code except that it will take a Channel_info object. The thd creation and connection authentication happens in the acceptor thread for this connection handler. The scope of current worklog would be only offloading the THD instantiation initialization on to a worker thread in case of Thread per connection handler and refactoring the initial connection handling code. Based on previous who reviewed the thd initialization too costly bug, it was decided if this will be addressed by a future worklog given the complexities inherent in this approach to be handled by a single worklog. The exact semantics of the current implementation would be maintained. release_connection:- This function will have an empty method definition as thread_pool scheduler does no specialized handling with respect to this callback in struct thread_scheduler. deinit: - This method is responsible for the deinitilization of the initialization that was performed in the the init method. The background thread and the various connection threads are waited for to gracefully terminate by this method. Connection_handler_mgr class This is a singleton class that encapsulates the functionality to set a scheduler and to get a handle to the current scheduler to be used. The class definition for the Connection_handler_mgr class is as follows: class Connection_handler_mgr { private: static Connection_handler_mgr* m_instance; Connection_handler* m_connection_handler; bool check_and_incr_conn_count(); Connection_handler_mgr(); Connection_handler_mgr(const Connection_handler_mgr&); Connection_handler_mgr& operator=(const Connection_handler_mgr&); public: static Connection_handler_mgr* get_instance(); bool set_connection_handler(scheduler_types sched); Connection_handler* get_connection_handler(); void process_new_connection(Channel_info* channel_info); }; The various member variables and methods for the Connection_handler_mgr are: m_instance:- This is a static member attribute that hold the only instance of the connection handler manager object. (Note a simple singleton pattern would be implemented since this will come into existence during the initiliazation/startup sequence of mysqld.) m_connection_handler:- This contains the current scheduler that is being used. check_and_incr_conn_count:- This is a private method which is used by process_new_connection API. This method checks to see if already max_connections are being processed by the client in which case it return false to the caller and the caller abort the incoming connection sending the following error code ER_CON_COUNT_ERROR else it return true incrementing the connection count. This method body is will be protected using the global LOCK_connection_count. get_instance:- This is public interface that provides the access to call the methods of the Connection_handler_mgr class. set_connection_handler:- The set connection handler method is used to set the current scheduler. It takes a scheduler type and creates an instance of the scheduler and stores it in m_connection_handler attribute of the manager class. If connection handler instantiation fails, this method return false else true. get_connection_handler:- The get connection handler method is responsible returning the pointer to the current scheduler instance that is to be used. process_new_connection:- This is a utility API related to connection management functionality. This calls the private method check_and_incr_conn_count to check if this connection can be admitted. If not it calls the Channel_info specific method send_error_and_abort_channel with the error code ER_CON_COUNT_ERROR. It deletes the Channel_info and returns to the caller. In the non error, it calls the current connection handler add_connection to start handling of the new connection. Connection_handler_callback struct The design will decouple thd_wait_begin, thd_wait_end and post_kill_notification function from struct scheduler_functions into a new struct connection_handler_callback. The reason for this decoupling are these function callbacks which when relate to notification of specific events like beginning of wait event in mysqld code, end of wait event in mysqld, post nofitication of kill connection to components like the thread pool connection handler. These functionalities do not relate to the connection specific functionalities provided by the connection handlers and only the Thread pool connection handler components is interested to provide code for this callbacks to act on this events.This C struct interface to this callbacks have been retained because this APIs are required to be used in files like mysys.c which has c linkage and may be used by external programs that compile with this libraries. The definition of this struct is show below: struct Connection_handler_callabck { void (*thd_wait_begin) (THD* thd, int wait_type); void (*thd_wait_end) (THD* thd); void (*post_kill_notification) (THD* thd); }; The semantics of the above function pointers are same as that which exists in the current code and the thread pool connection will register these functionalities and will not be described in detail in the scope of this document. Channel_info interface class The Channel_info is a interface class and hence has pure virtual function create_thd and send_error_and_abort_channel. class Channel_info { public: virtual ~Channel_info(); virtual THD* create_thd() = 0; virtual void send_error_and_close_channel(uint errorcode, bool senderror=true) = 0; }; The various interface functions provided by Channel_info class are: create_thd:- This is a pure virtual function so all derived class which inherits from Channel_info should implement this function. The method responsibility is to instantiate a THD struct and initializes it based on the state information contained for various types of channel like socket, pipe and shared memory. send_error_and_close_channel: - This is a pure virtual function which needs to be defined by the various types of channel info implementations. This method takes as argument an errorcode and a senderror flag whose default value is true (in the default case of true, error would be sent to the client). The purpose of this method is to send error back to the client (if senderror arg. is true) and abort the channel for kind of errors like ER_OUT_OF_RESOURCES, ER_CANT_CREATE_THREAD, ER_CON_COUNT_ERROR that occurs during the initial connection handling path and then abort the channel. A THD struct may not be necessary for error propagation since the THD goes out of scope when the connection gets aborted. This method will tyy and instantiate a vio and init and then create a NET struct (NET object on stack) and initialize it (instead of creating a THD to avoid overhead of error paths) (if this fails error would be logged to server log) and then use the low-level API (vio layer) to send error. Once the error message is sent, depending on the type of channel, the channel would be closed. Channel_info_socket class The Channel_info_socket class represents an abstraction for state information pertaining to the TCP/IP mode of communication that client initiates with the server. It implements the interface class Channel_info and it’s methods create_net and send_error_and_abort_channel. class Channel_info_socket : public Channel_info { public: Channel_info_socket(MYSQL_SOCKET listen_socket, MYSQL_SOCKET connect_socket, bool m_is_tcpip); virtual THD* create_thd(); virtual void send_error_and_close_channel(uint errorcode,bool senderror=true); private: Vio* create_and_init_vio(); MYSQL_SOCKET m_listen_sock; MYSQL_SOCKET m_connect_sock; bool m_is_tcpip; }; The various attributes and methods make up the Channel_info_socket class are: m_listen_sock: - This attribute indicate the listen socket descriptor that is encapsulated in MYSQL_SOCKET m_connect_sock:- This attribute indicate the connect socket descriptor of the client that is wrapped in MYSQL_SOCKET. m_is_tcpip: It is bool flag with value true if the descriptors belong to TCP/IP else false which indicates UNIX local sockets. create_and_init_vio:- This method instantiates a Vio and initializes the vio struct for the socket channel based on the attribute information stored in member variables. Vio instantiation can fail and in which case only an error logging could be done and the connection error count would be incremented and the caller of this method will be responsible to abort the connection. This is a private method which will be used by the public methods create_thd and send_error_and_abort_channel. create_thd: This instantiate a new THD object and initialize the NET field of the object with the vio and other related values. The THD instantiation and vio/net instantiation could possibly fail and the method will return NULL to the caller. send_error_and_close_channel: - This method will take errorcode and create and init a vio struct and then initialize NET struct and then send an error to the client using the NET low-level API for sending an error and the channel would be aborted. In case if the senderror is false, then socket channel shall be closed. 4.1.9 Channel_info_shared_mem The Channel_info_shared_mem class represents an abstraction for the state information pertaining to the shared memory channel of communication (which is used in windows).The related class definitions and the related structures are: class Channel_info_shared_mem : public Channel_info { public: Channel_info_shared_mem(HANDLE handle_client_file_map,char* handle_client_map, HANDLE event_server_wrote, HANDLE event_client_wrote, HANDLE event_client_read, HANDLE event_conn_closed); virtual THD* create_thd(); virtual void send_error_and_close_channel(uint errorcode); private: Vio* create_and_init_vio(); HANDLE m_handle_client_file_map; char* m_handle_client_map; HANDLE m_event_server_wrote; HANDLE m_event_server_read; HANDLE m_event_client_wrote; HANDLE m_event_client_read; HANDLE m_event_conn_closed; }; The various attributes and methods make up the Channel_info_socket class are: m_handle_client_file_map: - This attribute hold the handle to the shared memory file mapping object for the client that connected. m_handle_client_map:- This attribute hold the handle of the view of the file in the process address space. m_event_server_wrote:- It is a handle to the event object to signal the event server has written data. m_event_server_read:- It is a handle to the event object to signal the event server has read data. m_event_client_read:- It is a handle to the event object to signal the event client has data to be received. m_event_client_wrote:- :- It is a handle to the event object to signal the event client has data to be written. m_event_conn_closed:- It is a handle to the event object to signal the event connection has been closed. create_and_init_vio:- This method instantiates a Vio and initializes the vio struct for the shared memory channel based on the attribute information stored in member variables. Vio instantiation can fail and in which case only an error logging could be done and the connection error count would be incremented and the caller of this method will be responsible to abort the connection. This is a private method which will be used by the public methods create_thd and send_error_and_abort_channel. create_thd:- This instantiate a new THD object and initialize the NET field of the object with the vio and other related values. The THD instantiation and vio/net instantiation could possibly fail and the method will return NULL to the caller. send_error_and_close_channel:- This method will take errorcode and create and init a vio struct and then initialize NET struct and then send an error to the client using the NET low-level API for sending an error and the channel would be aborted. In case if the senderror is false, then shared memory channel shall be closed. Channel_info_named_pipe class The Channel_info_named_pipe class represents an abstraction for the state information pertaining to the named pipe mode of communication which is used with windows mysqld server. The related class definitions are: class Channel_info_named_pipe : public Channel_info { public: Channel_info_pipe(HANDLE handle); virtual THD* create_thd(); virtual void send_error_and_close_channel(uint errorcode); private: Vio* create_and_init_vio(); HANDLE m_handle; }; The various attributes and methods make up the Channel_info_named_pipe class are: m_handle: - This attribute holds a handle to the pipe channel. create_and_init_vio:- It creates vio and initializes the vio struct for the pipe channel. create_thd:- This instantiates a THD and initialize the NET struct field of THD. It uses the private method create_and_init_vio to create and initialize a vio. The instantiation of THD struct could fail and the vio struct could also fail in which case we return NULL to the caller of the method. send_error_and_close_channel:- This method will take errorcode and create and init a vio struct and then initialize NET struct and then send an error to the client using the NET low-level API for sending an error and the channel would be aborted. In case if the senderror is false, then pipe channel shall be closed..
Copyright (c) 2000, 2024, Oracle Corporation and/or its affiliates. All rights reserved.