WL#4876: Parse options before initializing mysys
Affects: Server-9.x
—
Status: Complete
In mysqld, mysys is initialized (my_init()) before options are parsed (handle_options()). Thus, options which should influence objects part of mysys, don't. Example: --safe-mutex-deadlock-detector=0 disables mutex deadlock detection in 6.0; but this option is parsed after the pthread_mutex_init() calls of my_init(). So, mutexes initialized by my_init() always do mutex deadlock detection, the option doesn't influence them. Performance Schema is also impacted The performance schema needs to initialize the server in this order: 1) Parse options affecting the performance schema Performance schema specific options. ==================================== --performance-schema-enabled={true|false} --performance-schema-max-mutex-instruments=--performance-schema-max-rwlock-instruments= --performance-schema-max-cond-instruments= --performance-schema-max-thread-instruments= --performance-schema-max-table-instruments= --performance-schema-max-file-instruments= --performance-schema-max-mutex= --performance-schema-max-rwlock= --performance-schema-max-cond= --performance-schema-max-thread= --performance-schema-max-table= --performance-schema-max-file= --performance-schema-events-waits-history-size= --performance-schema-events-waits-history-long-size= 2) Allocate internal buffers based on options parsed in 1), and internally deploy the instrumentation in the server 3) Execute MYSQL_MUTEX_INIT, MYSQL_RWLOCK_INIT, MYSQL_COND_INIT, MYSQL_OPEN, MYSQL_FOPEN and pthread_create in the server code only *after* 2), to collect instrumentation data. Because handle_options() happens too late currently, the performance schema code is duplicating parsing of options from the command line (argc, argv) in main(). This code is a work around that only partially works: - the server honor ./mysqld --performance-schema-xxx - the server ignores --performance-schema-xxx in my.cnf, which is not user friendly and leads to confusion. Also, code is duplicated because of this. The desired result is that handle_options() parses all arguments before initializing any mutex, rwlock, condition, or starting any thread.
It seems impossible to exactly achieve parsing all options before initializing all of mysys because parsing of options (including reading my.cnf): - uses dbug - allocates memory with MEM_ROOT => my_malloc => safemalloc => THR_LOCK_malloc which is created by my_init() (via my_thread_global_init()). With my_strdup() too. - uses my_fopen() => THR_LOCK_open - uses my_stat() and my_dir() (same issues) - uses DYNAMIC_ARRAY => my_malloc() - side effects of certain options: GET_STR_ALLOC => my_malloc(); key cache options => LOCK_global_system_variables - may call my_getopt_error_reporter which can be mysqld.cc:option_error_reporter which calls vprint_msg_to_log() which can take log's mutex. Suggestion is to take an incomplete step instead. Split the initialization of mysys (my_init()) in two pieces: piece for the minimum which option parsing needs (dbug, THR_LOCK_malloc for _mymalloc() and _myfree(), THR_LOCK_open for my_fopen() and my_dir()), and piece for all the rest. Parsing of options and initialization of P_S would move up right after the first piece of my_init(). For example like this: main() { mysys_init_first_piece(); // dbug, the few mutex logger.init_base(); // for logging of errors during option parsing (*) load_defaults(); // moved from init_common_variables() split list of options between P_S and non-P_S options; /* P_S options must be *known* early so that P_S is inited before most objects. Non-P_S options must be *parsed* late (their parsing causes side-effects which rely on their modules having been initialized already; like rpl_filter). */ handle_options(ONLY P_S options) // moved from init_common_variables() my_init_thread(); // if init_P_S() really needs it (?) init_P_S; destroy and init again the few mutexes created early; (**) mysys_init_last_piece(); // rest of code unchanged: init_common_variables(); // which will parse non-P_S options (*) logger is used by mysqld.cc:option_error_reporter() which is invoked when displaying an error in option parsing; so either: - logger is initialized that early - or is initialized later, but then for early parsing of options we use a simpler my_getopt_error_reporter: a default one or one which buffers messages in memory (and so in case of error, logger is initialized, passed the buffered messages for printing). The buffering one is prefeered because the default one would likely be printing to stderr which sometimes is hardly visible (because DBA may by habit only look at log specified in --log-error or at the Windows event log or CSV-based log tables, and not stderr) (**) among those mutexes there are some debug-specific ones like THR_LOCK_malloc, which we may not want to instrument (makes execution path of debug binary closer to release binary). Only a few hours of code inspection suggest that this would be feasible; the implementor should start prototyping this design; if things prove too complicated we will have to choose another design.
Copyright (c) 2000, 2024, Oracle Corporation and/or its affiliates. All rights reserved.