MySQL  8.0.17
Source Code Documentation
log_builtins.cc File Reference
#include "log_builtins_filter_imp.h"
#include "log_builtins_imp.h"
#include "my_dir.h"
#include "mysys_err.h"
#include <mysql/components/services/log_service.h>
#include <mysql/components/services/log_shared.h>
#include "registry.h"
#include "server_component.h"
#include "sql/current_thd.h"
#include "sql/log.h"
#include "sql/mysqld.h"
#include "sql/sql_class.h"
#include "mysql/components/services/log_builtins.h"
#include <syslog.h>

Classes

struct  log_service_cache_entry_free
 
struct  log_line_buffer
 
struct  log_service_cache_entry
 Finding and acquiring a service in the component framework is expensive, and we may use services a log (depending on how many events are logged per second), so we cache the relevant data. More...
 
struct  _log_service_instance
 State of a given instance of a service. More...
 
struct  log_errstream
 An error-stream. More...
 
struct  _log_item_wellknown_key
 Pre-defined "well-known" keys, as opposed to ad hoc ones, for key/value pairs in logging. More...
 

Macros

#define LOG_SUBSYSTEM_TAG   "Server"
 
#define LOG_SERVICES_PREFIX   "log_service"
 Name of the interface that log-services implements. More...
 
#define LOG_BUILTINS_FILTER   "log_filter_internal"
 Name of internal filtering engine (so we may recognize it when the user refers to it by name in log_error_services). More...
 
#define LOG_BUILTINS_SINK   "log_sink_internal"
 Name of internal log writer (so we may recognize it when the user refers to it by name in log_error_services). More...
 
#define LOG_BUILTINS_BUFFER   "log_sink_buffer"
 Name of buffered log writer. More...
 
#define LOG_BUFFERING_TIMEOUT_AFTER   (60)
 If after this many seconds we're still buffering, flush! More...
 
#define LOG_BUFFERING_TIMEOUT_EVERY   (10)
 And thereafter, if after this many more seconds still buffering, flush again! More...
 
#define LOG_BUFFERING_TIME_SCALE   1000000
 Function returns microseconds, we want seconds. More...
 

Typedefs

using cache_entry_with_deleter = unique_ptr< log_service_cache_entry, log_service_cache_entry_free >
 We're caching handles to the services used in error logging as looking them up is costly. More...
 
typedef struct _log_service_instance log_service_instance
 State of a given instance of a service. More...
 
typedef struct _log_item_wellknown_key log_item_wellknown_key
 Pre-defined "well-known" keys, as opposed to ad hoc ones, for key/value pairs in logging. More...
 

Functions

static int log_service_check_if_builtin (const char *name, size_t len)
 Test whether a given log-service name refers to a built-in service (built-in filter or built-in sink at this point). More...
 
static bool log_service_has_characteristics (log_service_cache_entry *sce, int required_flags)
 Test whether given service has all of the given characteristics. More...
 
int log_string_compare (const char *a, const char *b, size_t len, bool case_insensitive)
 Compare two NUL-terminated byte strings. More...
 
bool log_item_generic_type (log_item_type t)
 Predicate used to determine whether a type is generic (generic string, generic float, generic integer) rather than a well-known type. More...
 
bool log_item_string_class (log_item_class c)
 Predicate used to determine whether a class is a string class (C-string or Lex-string). More...
 
bool log_item_numeric_class (log_item_class c)
 Predicate used to determine whether a class is a numeric class (integer or float). More...
 
void log_item_get_int (log_item *li, longlong *i)
 Get an integer value from a log-item of float or integer type. More...
 
void log_item_get_float (log_item *li, double *f)
 Get a float value from a log-item of float or integer type. More...
 
void log_item_get_string (log_item *li, char **str, size_t *len)
 Get a string value from a log-item of C-string or Lex string type. More...
 
int log_item_wellknown_by_name (const char *key, size_t len)
 See whether a string is a wellknown field name. More...
 
int log_item_wellknown_by_type (log_item_type t)
 See whether a type is wellknown. More...
 
const char * log_item_wellknown_get_name (uint idx)
 Accessor: from a record describing a wellknown key, get its name. More...
 
log_item_type log_item_wellknown_get_type (uint idx)
 Accessor: from a record describing a wellknown key, get its type. More...
 
log_item_class log_item_wellknown_get_class (uint idx)
 Accessor: from a record describing a wellknown key, get its class. More...
 
int log_item_inconsistent (log_item *li)
 Sanity check an item. More...
 
void log_item_free (log_item *li)
 Release any of key and value on a log-item that were dynamically allocated. More...
 
log_linelog_line_init ()
 Dynamically allocate and initialize a log_line. More...
 
void log_line_exit (log_line *ll)
 Release a log_line allocated with line_init() More...
 
bool log_line_full (log_line *ll)
 Predicate indicating whether a log line is "willing" to accept any more key/value pairs. More...
 
bool log_line_duplicate (log_line *dst, log_line *src)
 Duplicate a log-event. More...
 
int log_line_item_count (log_line *ll)
 How many items are currently set on the given log_line? More...
 
log_item_type_mask log_line_item_types_seen (log_line *ll, log_item_type_mask m)
 Test whether a given type is presumed present on the log line. More...
 
void log_line_item_free (log_line *ll, size_t elem)
 Release log line item (key/value pair) with the index elem in log line ll. More...
 
void log_line_item_free_all (log_line *ll)
 Release all log line items (key/value pairs) in log line ll. More...
 
void log_line_item_remove (log_line *ll, int elem)
 Release log line item (key/value pair) with the index elem in log line ll. More...
 
int log_line_index_by_name (log_line *ll, const char *key)
 Find the (index of the) last key/value pair of the given name in the log line. More...
 
log_itemlog_line_item_by_name (log_line *ll, const char *key)
 Find the last item matching the given key in the log line. More...
 
int log_line_index_by_type (log_line *ll, log_item_type t)
 Find the (index of the) last key/value pair of the given type in the log line. More...
 
int log_line_index_by_item (log_line *ll, log_item *ref)
 Find the (index of the) last key/value pair of the given type in the log line. More...
 
void log_item_init (log_item *li)
 Initializes a log entry for use. More...
 
log_itemlog_line_item_init (log_line *ll)
 Initializes an entry in a log line for use. More...
 
log_item_datalog_item_set_with_key (log_item *li, log_item_type t, const char *key, uint32 alloc)
 Create new log item with key name "key", and allocation flags of "alloc" (see enum_log_item_free). More...
 
log_item_datalog_line_item_set_with_key (log_line *ll, log_item_type t, const char *key, uint32 alloc)
 Create new log item in log line "ll", with key name "key", and allocation flags of "alloc" (see enum_log_item_free). More...
 
log_item_datalog_item_set (log_item *li, log_item_type t)
 As log_item_set_with_key(), except that the key is automatically derived from the wellknown log_item_type t. More...
 
log_item_datalog_line_item_set (log_line *ll, log_item_type t)
 Create a new log item of well-known type "t" in log line "ll". More...
 
bool log_item_set_int (log_item_data *lid, longlong i)
 Set an integer value on a log_item. More...
 
bool log_item_set_float (log_item_data *lid, double f)
 Set a floating point value on a log_item. More...
 
bool log_item_set_lexstring (log_item_data *lid, const char *s, size_t s_len)
 Set a string value on a log_item. More...
 
bool log_item_set_cstring (log_item_data *lid, const char *s)
 Set a string value on a log_item. More...
 
const char * log_label_from_prio (int prio)
 Convenience function: Derive a log label ("error", "warning", "information") from a severity. More...
 
static int log_sink_trad (void *instance, log_line *ll)
 services: log sinks: basic logging ("classic error-log") Will write timestamp, label, thread-ID, and message to stderr/file. More...
 
static int log_sink_buffer (void *instance, log_line *ll)
 services: log sinks: buffered logging More...
 
void log_sink_buffer_check_timeout (void)
 Convenience function. More...
 
void log_sink_buffer_flush (enum log_sink_buffer_flush_mode mode)
 Process all buffered log-events. More...
 
int log_line_submit (log_line *ll)
 Complete, filter, and write submitted log items. More...
 
int make_iso8601_timestamp (char *buf, ulonglong utime, int mode)
 Make and return an ISO 8601 / RFC 3339 compliant timestamp. More...
 
static size_t log_builtins_stack_get_service_from_var (const char **s, const char **e, char *d)
 Helper: get token from error stack configuration string. More...
 
static my_h_service log_service_get_by_name (const char *name, size_t len)
 Look up a log service by name (in the service registry). More...
 
static log_service_cache_entrylog_service_cache_entry_new (const char *name, size_t name_len, my_h_service srv)
 Create a new entry in the cache of log services. More...
 
static int log_service_get_characteristics (my_h_service service)
 Find out characteristics of a service (e.g. More...
 
log_service_instancelog_service_instance_new (log_service_cache_entry *sce, log_line *ll)
 Allocate and open a new instance of a given service. More...
 
static void log_service_instance_release_all ()
 Close and release all instances of all log services. More...
 
int log_builtins_error_stack_flush ()
 Call flush() on all log_services. More...
 
int log_builtins_error_stack (const char *conf, bool check_only, size_t *pos)
 Set up custom error logging stack. More...
 
int log_builtins_exit ()
 De-initialize the structured logging subsystem. More...
 
int log_builtins_init ()
 Initialize the structured logging subsystem. More...
 

Variables

log_filter_rulesetlog_filter_builtin_rules
 
PSI_memory_key key_memory_log_error_loaded_services
 
PSI_memory_key key_memory_log_error_stack
 
static collation_unordered_map< string, cache_entry_with_deleter > * log_service_cache
 
static mysql_rwlock_t THR_LOCK_log_stack
 Lock for the log "stack" (i.e. More...
 
static mysql_mutex_t THR_LOCK_log_syseventlog
 Make sure only one instance of syslog/Eventlog code runs at a time. More...
 
static mysql_mutex_t THR_LOCK_log_buffered
 Make sure only one instance of the buffered "writer" runs at a time. More...
 
static bool log_builtins_inited = false
 Subsystem initialized and ready to use? More...
 
static log_line_bufferlog_line_buffer_start = nullptr
 Pointer to the first element in the list of buffered log messages. More...
 
static log_line_buffer ** log_line_buffer_tail = &log_line_buffer_start
 Where to write the pointer to the newly-created tail-element of the list. More...
 
static ulonglong log_buffering_timeout = 0
 
static int log_buffering_flushworthy = false
 Only flush on time-out when buffer contains errors. More...
 
static ulonglong log_sink_trad_last = 0
 
static log_service_instancelog_service_instances = nullptr
 anchor More...
 
static const log_item_wellknown_key log_item_wellknown_keys []
 We support a number of predefined keys, such as "error-code" or "message". More...
 
static uint log_item_wellknown_keys_count
 

Macro Definition Documentation

◆ LOG_BUFFERING_TIME_SCALE

#define LOG_BUFFERING_TIME_SCALE   1000000

Function returns microseconds, we want seconds.

◆ LOG_BUFFERING_TIMEOUT_AFTER

#define LOG_BUFFERING_TIMEOUT_AFTER   (60)

If after this many seconds we're still buffering, flush!

◆ LOG_BUFFERING_TIMEOUT_EVERY

#define LOG_BUFFERING_TIMEOUT_EVERY   (10)

And thereafter, if after this many more seconds still buffering, flush again!

◆ LOG_BUILTINS_BUFFER

#define LOG_BUILTINS_BUFFER   "log_sink_buffer"

Name of buffered log writer.

This sink is used internally during start-up until we know where to write and how to filter, and have all the components to do so. While we don't let the DBA add this sink to the logging pipeline once we're out of start-up, we have a name for this to be able to correctly tag its record in the service-cache.

◆ LOG_BUILTINS_FILTER

#define LOG_BUILTINS_FILTER   "log_filter_internal"

Name of internal filtering engine (so we may recognize it when the user refers to it by name in log_error_services).

◆ LOG_BUILTINS_SINK

#define LOG_BUILTINS_SINK   "log_sink_internal"

Name of internal log writer (so we may recognize it when the user refers to it by name in log_error_services).

◆ LOG_SERVICES_PREFIX

#define LOG_SERVICES_PREFIX   "log_service"

Name of the interface that log-services implements.

◆ LOG_SUBSYSTEM_TAG

#define LOG_SUBSYSTEM_TAG   "Server"

Typedef Documentation

◆ cache_entry_with_deleter

We're caching handles to the services used in error logging as looking them up is costly.

◆ log_item_wellknown_key

Pre-defined "well-known" keys, as opposed to ad hoc ones, for key/value pairs in logging.

◆ log_service_instance

State of a given instance of a service.

A service may support being opened several times.

Function Documentation

◆ log_builtins_error_stack()

int log_builtins_error_stack ( const char *  conf,
bool  check_only,
size_t *  pos 
)

Set up custom error logging stack.

Parameters
confThe configuration string
check_onlyIf true, report on whether configuration is valid (i.e. whether all requested services are available), but do not apply the new configuration. if false, set the configuration (acquire the necessary services, update the hash by adding/deleting entries as necessary)
[out]posIf an error occurs and this pointer is non-null, the position in the configuration string where the error occurred will be written to the pointed-to size_t.
Return values
0success
-1expected delimiter not found
-2one or more services not found
-3failed to create service cache entry
-4tried to open multiple instances of a singleton
-5failed to create service instance entry
-6last element in pipeline should be a sink
-7service only available during start-up; may not be set by the user
-101service name may not start with a delimiter
-102delimiters ',' and ';' may not be mixed

◆ log_builtins_error_stack_flush()

int log_builtins_error_stack_flush ( )

Call flush() on all log_services.

flush() function must not try to log anything, as we hold an exclusive lock on the stack.

Return values
0no problems
-1error

◆ log_builtins_exit()

int log_builtins_exit ( )

De-initialize the structured logging subsystem.

Return values
0no errors
-1not stopping, never started

◆ log_builtins_init()

int log_builtins_init ( )

Initialize the structured logging subsystem.

Since we're initializing various locks here, we must call this late enough so this is clean, but early enough so it still happens while we're running single-threaded – this specifically also means we must call it before we start plug-ins / storage engines / external components!

Return values
0no errors
-1couldn't initialize stack lock
-2couldn't initialize built-in default filter
-3couldn't set up service hash
-4couldn't initialize syseventlog lock
-5couldn't set service pipeline
-6couldn't initialize buffered logging lock

◆ log_builtins_stack_get_service_from_var()

static size_t log_builtins_stack_get_service_from_var ( const char **  s,
const char **  e,
char *  d 
)
static

Helper: get token from error stack configuration string.

Parameters
[in,out]sstart of the token (may be positioned on whitespace on call; this will be adjusted to the first non-white character)
[out]eend of the token
[in,out]ddelimiter (in: last used, \0 if none; out: detected here)
Return values
<0an error occur
thelength in bytes of the token

◆ log_item_free()

void log_item_free ( log_item li)

Release any of key and value on a log-item that were dynamically allocated.

Parameters
lilog-item to release the payload of

◆ log_item_generic_type()

bool log_item_generic_type ( log_item_type  t)

Predicate used to determine whether a type is generic (generic string, generic float, generic integer) rather than a well-known type.

Parameters
tlog item type to examine
Return values
trueif generic type
falseif wellknown type

◆ log_item_get_float()

void log_item_get_float ( log_item li,
double *  f 
)

Get a float value from a log-item of float or integer type.

Parameters
lilog item to get the value from
[out]ffloat to store the value in

◆ log_item_get_int()

void log_item_get_int ( log_item li,
longlong i 
)

Get an integer value from a log-item of float or integer type.

Parameters
lilog item to get the value from
[out]ilonglong to store the value in

◆ log_item_get_string()

void log_item_get_string ( log_item li,
char **  str,
size_t *  len 
)

Get a string value from a log-item of C-string or Lex string type.

Parameters
lilog item to get the value from
[out]strchar-pointer to store the pointer to the value in
[out]lensize_t pointer to store the length of the value in

◆ log_item_inconsistent()

int log_item_inconsistent ( log_item li)

Sanity check an item.

Certain log sinks have very low requirements with regard to the data they receive; they write keys as strings, and then data according to the item's class (string, integer, or float), formatted to the sink's standards (e.g. JSON, XML, ...). Code that has higher requirements can use this check to see whether the given item is of a known type (whether generic or wellknown), whether the given type and class agree, and whether in case of a well-known type, the given key is correct for that type. If your code generates items that don't pass this check, you should probably go meditate on it.

Parameters
lithe log_item to check
Return values
LOG_ITEM_OKno problems
LOG_ITEM_TYPE_NOT_FOUNDunknown item type
LOG_ITEM_CLASS_MISMATCHitem_class derived from type isn't what's set on the item
LOG_ITEM_KEY_MISMATCHclass not generic, so key should match wellknown
LOG_ITEM_STRING_NULLclass is string, pointer is nullptr
LOG_ITEM_KEY_NULLno key set (this is legal e.g. on aux items of filter rules, but should not occur in a log_line, i.e., log_sinks are within their rights to discard such items)

◆ log_item_init()

void log_item_init ( log_item li)

Initializes a log entry for use.

This simply puts it in a defined state; if you wish to reset an existing item, see log_item_free().

Parameters
lithe log-item to initialize

◆ log_item_numeric_class()

bool log_item_numeric_class ( log_item_class  c)

Predicate used to determine whether a class is a numeric class (integer or float).

Parameters
clog item class to examine
Return values
trueif of a numeric class
falseif not of a numeric class

◆ log_item_set()

log_item_data* log_item_set ( log_item li,
log_item_type  t 
)

As log_item_set_with_key(), except that the key is automatically derived from the wellknown log_item_type t.

Create new log item with type "t". Will return a pointer to the item's log_item_data struct for convenience. This is mostly interesting for filters and other services that create items that are not part of a log_line; sources etc. that intend to create an item for a log_line (the more common case) should usually use the below line_item_set_with_key() which creates an item (like this function does), but also correctly inserts it into a log_line.

The allocation of this item will be LOG_ITEM_FREE_NONE; specifically, any pre-existing value will be clobbered. It is therefore WRONG a) to use this on a log_item that already has a key; it should only be used on freshly init'd log_items; b) to use this on a log_item that already has a value (specifically, an allocated one); the correct order is to init a log_item, then set up type and key, and finally to set the value. If said value is an allocated string, the log_item's alloc should be bitwise or'd with LOG_ITEM_FREE_VALUE.

Parameters
lithe log_item to work on
tthe item-type
Return values
apointer to the log_item's log_data, for easy chaining: log_item_set_with_key(...)->data_integer= 1;

◆ log_item_set_cstring()

bool log_item_set_cstring ( log_item_data lid,
const char *  s 
)

Set a string value on a log_item.

Fails gracefully if no log_item_data is supplied, so it can safely wrap log_line_item_set[_with_key]().

Parameters
lidlog_item_data struct to set the value on
spointer to NTBS
Return values
truelid was nullptr (possibly: OOM, could not set up log_item)
falseall's well

◆ log_item_set_float()

bool log_item_set_float ( log_item_data lid,
double  f 
)

Set a floating point value on a log_item.

Fails gracefully if no log_item_data is supplied, so it can safely wrap log_line_item_set[_with_key]().

Parameters
lidlog_item_data struct to set the value on
ffloat to set
Return values
truelid was nullptr (possibly: OOM, could not set up log_item)
falseall's well

◆ log_item_set_int()

bool log_item_set_int ( log_item_data lid,
longlong  i 
)

Set an integer value on a log_item.

Fails gracefully if no log_item_data is supplied, so it can safely wrap log_line_item_set[_with_key]().

Parameters
lidlog_item_data struct to set the value on
iinteger to set
Return values
truelid was nullptr (possibly: OOM, could not set up log_item)
falseall's well

◆ log_item_set_lexstring()

bool log_item_set_lexstring ( log_item_data lid,
const char *  s,
size_t  s_len 
)

Set a string value on a log_item.

Fails gracefully if no log_item_data is supplied, so it can safely wrap log_line_item_set[_with_key]().

Parameters
lidlog_item_data struct to set the value on
spointer to string
s_lenlength of string
Return values
truelid was nullptr (possibly: OOM, could not set up log_item)
falseall's well

◆ log_item_set_with_key()

log_item_data* log_item_set_with_key ( log_item li,
log_item_type  t,
const char *  key,
uint32  alloc 
)

Create new log item with key name "key", and allocation flags of "alloc" (see enum_log_item_free).

Will return a pointer to the item's log_item_data struct for convenience. This is mostly interesting for filters and other services that create items that are not part of a log_line; sources etc. that intend to create an item for a log_line (the more common case) should usually use the below line_item_set_with_key() which creates an item (like this function does), but also correctly inserts it into a log_line.

Parameters
lithe log_item to work on
tthe item-type
keythe key to set on the item. ignored for non-generic types (may pass nullptr for those) see alloc
allocLOG_ITEM_FREE_KEY if key was allocated by caller LOG_ITEM_FREE_NONE if key was not allocated Allocated keys will automatically free()d when the log_item is. The log_item's alloc flags will be set to the submitted value; specifically, any pre-existing value will be clobbered. It is therefore WRONG a) to use this on a log_item that already has a key; it should only be used on freshly init'd log_items; b) to use this on a log_item that already has a value (specifically, an allocated one); the correct order is to init a log_item, then set up type and key, and finally to set the value. If said value is an allocated string, the log_item's alloc should be bitwise or'd with LOG_ITEM_FREE_VALUE.
Return values
apointer to the log_item's log_data, for easy chaining: log_item_set_with_key(...)->data_integer= 1;

◆ log_item_string_class()

bool log_item_string_class ( log_item_class  c)

Predicate used to determine whether a class is a string class (C-string or Lex-string).

Parameters
clog item class to examine
Return values
trueif of a string class
falseif not of a string class

◆ log_item_wellknown_by_name()

int log_item_wellknown_by_name ( const char *  key,
size_t  len 
)

See whether a string is a wellknown field name.

Parameters
keypotential key starts here
lenlength of the string to examine
Return values
LOG_ITEM_TYPE_RESERVEDreserved, but not "wellknown" key
LOG_ITEM_TYPE_NOT_FOUNDkey not found
>0index in array of wellknowns

◆ log_item_wellknown_by_type()

int log_item_wellknown_by_type ( log_item_type  t)

See whether a type is wellknown.

Parameters
tlog item type to examine
Return values
LOG_ITEM_TYPE_NOT_FOUNDkey not found
>0index in array of wellknowns

◆ log_item_wellknown_get_class()

log_item_class log_item_wellknown_get_class ( uint  idx)

Accessor: from a record describing a wellknown key, get its class.

Parameters
idxindex in array of wellknowns, see log_item_wellknown_by_...()
Return values
thelog item class for the wellknown key

◆ log_item_wellknown_get_name()

const char* log_item_wellknown_get_name ( uint  idx)

Accessor: from a record describing a wellknown key, get its name.

Parameters
idxindex in array of wellknowns, see log_item_wellknown_by_...()
Return values
name(NTBS)

◆ log_item_wellknown_get_type()

log_item_type log_item_wellknown_get_type ( uint  idx)

Accessor: from a record describing a wellknown key, get its type.

Parameters
idxindex in array of wellknowns, see log_item_wellknown_by_...()
Return values
thelog item type for the wellknown key

◆ log_label_from_prio()

const char* log_label_from_prio ( int  prio)

Convenience function: Derive a log label ("error", "warning", "information") from a severity.

Parameters
priothe severity/prio in question
Returns
a label corresponding to that priority.
Return values
Systemfor prio of SYSTEM_LEVEL
Errorfor prio of ERROR_LEVEL
Warningfor prio of WARNING_LEVEL
Notefor prio of INFORMATION_LEVEL

◆ log_line_duplicate()

bool log_line_duplicate ( log_line dst,
log_line src 
)

Duplicate a log-event.

This is a deep copy where the items (key/value pairs) have their own allocated memory separate from that in the source item.

Parameters
dstlog_line that will hold the copy
srclog_line we copy from
Return values
falseon success
trueif out of memory

◆ log_line_exit()

void log_line_exit ( log_line ll)

Release a log_line allocated with line_init()

Release a log_line allocated with log_line_init.

Parameters
lla log_line previously allocated with line_init()

◆ log_line_full()

bool log_line_full ( log_line ll)

Predicate indicating whether a log line is "willing" to accept any more key/value pairs.

Parameters
llthe log-line to examine
Return values
falseif not full / if able to accept another log_item
trueif full

◆ log_line_index_by_item()

int log_line_index_by_item ( log_line ll,
log_item ref 
)

Find the (index of the) last key/value pair of the given type in the log line.

Find the (index of the) first key/value pair of the given type in the log line.

This variant accepts a reference item and looks for an item that is of the same type (for wellknown types), or one that is of a generic type, and with the same key name (for generic types). For example, a reference item containing a generic string with key "foo" will a generic string, integer, or float with the key "foo".

Parameters
lllog line
refa reference item of the log item type to look for
Return values
<0none found
>=0index of the key/value pair in the log line

◆ log_line_index_by_name()

int log_line_index_by_name ( log_line ll,
const char *  key 
)

Find the (index of the) last key/value pair of the given name in the log line.

Parameters
lllog line
keythe key to look for
Return values
-1none found
-2invalid search-key given
-3no log_line given
>=0index of the key/value pair in the log line

◆ log_line_index_by_type()

int log_line_index_by_type ( log_line ll,
log_item_type  t 
)

Find the (index of the) last key/value pair of the given type in the log line.

Find the (index of the) first key/value pair of the given type in the log line.

Parameters
lllog line
tthe log item type to look for
Return values
<0none found
>=0index of the key/value pair in the log line

◆ log_line_init()

log_line* log_line_init ( )

Dynamically allocate and initialize a log_line.

Initialize a log_line.

Return values
nullptrcould not set up buffer (too small?)
otheraddress of the newly initialized log_line

◆ log_line_item_by_name()

log_item* log_line_item_by_name ( log_line ll,
const char *  key 
)

Find the last item matching the given key in the log line.

Parameters
lllog line
keythe key to look for
Return values
nullptritem not found
otherwisepointer to the item (not a copy thereof!)

◆ log_line_item_count()

int log_line_item_count ( log_line ll)

How many items are currently set on the given log_line?

Parameters
llthe log-line to examine
Return values
thenumber of items set

◆ log_line_item_free()

void log_line_item_free ( log_line ll,
size_t  elem 
)

Release log line item (key/value pair) with the index elem in log line ll.

This frees whichever of key and value were dynamically allocated. This leaves a "gap" in the bag that may immediately be overwritten with an updated element. If the intention is to remove the item without replacing it, use log_line_item_remove() instead!

Parameters
lllog_line
elemindex of the key/value pair to release

◆ log_line_item_free_all()

void log_line_item_free_all ( log_line ll)

Release all log line items (key/value pairs) in log line ll.

This frees whichever keys and values were dynamically allocated.

Parameters
lllog_line

◆ log_line_item_init()

log_item* log_line_item_init ( log_line ll)

Initializes an entry in a log line for use.

This simply puts it in a defined state; if you wish to reset an existing item, see log_item_free(). This resets the element beyond the last. The element count is not adjusted; this is for the caller to do once it sets up a valid element to suit its needs in the cleared slot. Finally, it is up to the caller to make sure that an element can be allocated.

Parameters
llthe log-line to initialize a log_item in
Return values
theaddress of the cleared log_item

◆ log_line_item_remove()

void log_line_item_remove ( log_line ll,
int  elem 
)

Release log line item (key/value pair) with the index elem in log line ll.

This frees whichever of key and value were dynamically allocated. This moves any trailing items to fill the "gap" and decreases the counter of elements in the log line. If the intention is to leave a "gap" in the bag that may immediately be overwritten with an updated element, use log_line_item_free() instead!

Parameters
lllog_line
elemindex of the key/value pair to release

◆ log_line_item_set()

log_item_data* log_line_item_set ( log_line ll,
log_item_type  t 
)

Create a new log item of well-known type "t" in log line "ll".

On success, the number of registered items on the log line is increased, the item's type is added to the log_line's "seen" property, and a pointer to the item's log_item_data struct is returned for convenience.

The allocation of this item will be LOG_ITEM_FREE_NONE; specifically, any pre-existing value will be clobbered. It is therefore WRONG a) to use this on a log_item that already has a key; it should only be used on freshly init'd log_items; b) to use this on a log_item that already has a value (specifically, an allocated one); the correct order is to init a log_item, then set up type and key, and finally to set the value. If said value is an allocated string, the log_item's alloc should be bitwise or'd with LOG_ITEM_FREE_VALUE.

Parameters
llthe log_line to work on
tthe item-type
Return values
!nullptra pointer to the log_item's log_data, for easy chaining: log_line_item_set(...)->data_integer= 1;
nullptrcould not create a log_item in given log_line

◆ log_line_item_set_with_key()

log_item_data* log_line_item_set_with_key ( log_line ll,
log_item_type  t,
const char *  key,
uint32  alloc 
)

Create new log item in log line "ll", with key name "key", and allocation flags of "alloc" (see enum_log_item_free).

On success, the number of registered items on the log line is increased, the item's type is added to the log_line's "seen" property, and a pointer to the item's log_item_data struct is returned for convenience.

Parameters
llthe log_line to work on
tthe item-type
keythe key to set on the item. ignored for non-generic types (may pass nullptr for those) see alloc
allocLOG_ITEM_FREE_KEY if key was allocated by caller LOG_ITEM_FREE_NONE if key was not allocated Allocated keys will automatically free()d when the log_item is. The log_item's alloc flags will be set to the submitted value; specifically, any pre-existing value will be clobbered. It is therefore WRONG a) to use this on a log_item that already has a key; it should only be used on freshly init'd log_items; b) to use this on a log_item that already has a value (specifically, an allocated one); the correct order is to init a log_item, then set up type and key, and finally to set the value. If said value is an allocated string, the log_item's alloc should be bitwise or'd with LOG_ITEM_FREE_VALUE.
Return values
!nullptra pointer to the log_item's log_data, for easy chaining: log_line_item_set_with_key(...)->data_integer= 1;
nullptrcould not create a log_item in given log_line

◆ log_line_item_types_seen()

log_item_type_mask log_line_item_types_seen ( log_line ll,
log_item_type_mask  m 
)

Test whether a given type is presumed present on the log line.

Parameters
llthe log_line to examine
mthe log_type to test for
Return values
0not present
!=0present

◆ log_line_submit()

int log_line_submit ( log_line ll)

Complete, filter, and write submitted log items.

This expects a log_line collection of log-related key/value pairs, e.g. from log_message().

Where missing, timestamp, priority, thread-ID (if any) and so forth are added.

Log item source services, log item filters, and log item sinks are then called.

Parameters
llkey/value pairs describing info to log
Return values
intnumber of fields in created log line

◆ log_service_cache_entry_new()

static log_service_cache_entry* log_service_cache_entry_new ( const char *  name,
size_t  name_len,
my_h_service  srv 
)
static

Create a new entry in the cache of log services.

Parameters
nameName of component that provides the service
name_lenLength of that name
srvThe handle of the log_service
Return values
Anew log_service_cache_entry on success
nullptron failure

◆ log_service_check_if_builtin()

static int log_service_check_if_builtin ( const char *  name,
size_t  len 
)
static

Test whether a given log-service name refers to a built-in service (built-in filter or built-in sink at this point).

Parameters
namethe name – either just the component's, or a fully qualified service.component
lenthe length of the aforementioned name
Return values
ifbuilt-in filter: flags for built-in|singleton|filter
ifbuilt-in sink: flags for built-in|singleton|sink
otherwiseLOG_SERVICE_UNSPECIFIED

◆ log_service_get_by_name()

static my_h_service log_service_get_by_name ( const char *  name,
size_t  len 
)
static

Look up a log service by name (in the service registry).

Parameters
namename of the component
lenlength of that name
Return values
ahandle to that service.

◆ log_service_get_characteristics()

static int log_service_get_characteristics ( my_h_service  service)
static

Find out characteristics of a service (e.g.

whether it is a singleton) by asking it.

(See log_service_chistics for a list of possible characteristics!)

Parameters
servicewhat service to examine
Return values
aset of log_service_chistics flags

◆ log_service_has_characteristics()

static bool log_service_has_characteristics ( log_service_cache_entry sce,
int  required_flags 
)
inlinestatic

Test whether given service has all of the given characteristics.

(See log_service_chistics for a list!)

Parameters
sceservice cache entry for the service in question
required_flagsflags we're interested in (bitwise or'd)
Return values
trueif all given flags are present, false otherwise

◆ log_service_instance_new()

log_service_instance* log_service_instance_new ( log_service_cache_entry sce,
log_line ll 
)

Allocate and open a new instance of a given service.

Parameters
scethe cache-entry for the service
lla log_line containing optional parameters, or nullptr
Returns
a pointer to an instance record or success, nullptr otherwise

◆ log_service_instance_release_all()

static void log_service_instance_release_all ( )
static

Close and release all instances of all log services.

◆ log_sink_buffer()

static int log_sink_buffer ( void *  instance,
log_line ll 
)
static

services: log sinks: buffered logging

During start-up, we buffer log-info until a) we have basic info for the built-in logger (what file to log to, verbosity, and so on), and b) advanced info (any logging components to load, any configuration for them, etc.).

As a failsafe, if start-up takes very, very long, and a time-out is reached before reaching b) and we actually have something worth reporting (e.g. errors, as opposed to info), we try to keep the user informed by using the basic logger configured in a), while going on buffering all info and flushing it to any advanced loggers when b) is reached.

1) This function checks and, if needed, updates the time-out, and calls the flush functions as needed. It is internal to the logger and should not be called from elsewhere.

2) Function will save log-event (if given) for later filtering and output.

3) Function acquires/releases THR_LOCK_log_buffered if initialized.

Parameters
instanceinstance handle Not currently used in this writer; if this changes later, keep in mind that nullptr will be passed if this is called before the structured logger's locks are initialized, so that must remain a valid argument!
llThe log line to write, or nullptr to not add a new logline, but to check whether the time-out has been reached and if so, flush as needed.
Return values
-1can not add event to buffer (OOM?)
>0number of added fields

< log-line buffer

◆ log_sink_buffer_check_timeout()

void log_sink_buffer_check_timeout ( void  )

Convenience function.

Same as log_sink_buffer(), except we only check whether the time-out for buffered logging has been reached during start-up, and act accordingly; no new logging information is added (i.e., we only provide functionality 1 described in the preamble for log_sink_buffer() above).

◆ log_sink_buffer_flush()

void log_sink_buffer_flush ( enum log_sink_buffer_flush_mode  mode)

Process all buffered log-events.

Release all buffered log-events (discard_error_log_messages()), optionally after running them through the error log stack first (flush_error_log_messages()).

LOG_BUFFER_DISCARD_ONLY simply releases all buffered log-events (used when called from discard_error_log_messages()).

LOG_BUFFER_PROCESS_AND_DISCARD sends the events through the configured error log stack first before discarding them. (used when called from flush_error_log_messages()). Must remain safe to call repeatedly (with subsequent calls only outputting any events added after the previous flush).

LOG_BUFFER_REPORT_AND_KEEP is used to show incremental status updates when the start-up takes unusually long. When that happens, we "report" the intermediate state using the built-in sink (as that's the only one we available to us at the time), and "keep" the events around. That way if log sinks other that the built-in one are configured, we can flush all of the buffered events there once initialization completes. (used when log_sink_buffer() detects a time-out, see also: LOG_BUFFERING_TIMEOUT_AFTER, LOG_BUFFERING_TIMEOUT_EVERY)

Parameters
modeLOG_BUFFER_DISCARD_ONLY (to just throw away the buffered events), or LOG_BUFFER_PROCESS_AND_DISCARD to filter/print them first, or LOG_BUFFER_REPORT_AND_KEEP to print an intermediate report on time-out.

To use LOG_BUFFER_REPORT_AND_KEEP in a multi-threaded environment, the caller needs to hold THR_LOCK_log_buffered while calling this; for the other modes, this function will acquire and release the lock as needed; in this second scenario, the lock will not be held while calling log_services (via log_line_submit()).

◆ log_sink_trad()

static int log_sink_trad ( void *  instance,
log_line ll 
)
static

services: log sinks: basic logging ("classic error-log") Will write timestamp, label, thread-ID, and message to stderr/file.

If you should not be able to specify a label, one will be generated for you from the line's priority field.

Parameters
instanceinstance handle
llthe log line to write
Return values
intnumber of added fields, if any

◆ log_string_compare()

int log_string_compare ( const char *  a,
const char *  b,
size_t  len,
bool  case_insensitive 
)

Compare two NUL-terminated byte strings.

Modular logger: log line and key/value manipulation helpers.

Note that when comparing without length limit, the long string is greater if they're equal up to the length of the shorter string, but the shorter string will be considered greater if its "value" up to that point is greater:

compare 'abc','abcd': -100 (longer wins if otherwise same) compare 'abca','abcd': -3 (higher value wins) compare 'abcaaaaa','abcd': -3 (higher value wins)

Parameters
athe first string
bthe second string
lencompare at most this many characters – 0 for no limit
case_insensitiveignore upper/lower case in comparison
Return values
-1a < b
0a == b
1a > b

◆ make_iso8601_timestamp()

int make_iso8601_timestamp ( char *  buf,
ulonglong  utime,
int  mode 
)

Make and return an ISO 8601 / RFC 3339 compliant timestamp.

Heeds log_timestamps.

Parameters
bufA buffer of at least 26 bytes to store the timestamp in (19 + tzinfo tail + \0)
utimeMicroseconds since the epoch
modeif 0, use UTC; if 1, use local time
Return values
lengthof timestamp (excluding \0)

Variable Documentation

◆ key_memory_log_error_loaded_services

PSI_memory_key key_memory_log_error_loaded_services

◆ key_memory_log_error_stack

PSI_memory_key key_memory_log_error_stack

◆ log_buffering_flushworthy

int log_buffering_flushworthy = false
static

Only flush on time-out when buffer contains errors.

◆ log_buffering_timeout

ulonglong log_buffering_timeout = 0
static

◆ log_builtins_inited

bool log_builtins_inited = false
static

Subsystem initialized and ready to use?

◆ log_filter_builtin_rules

log_filter_ruleset* log_filter_builtin_rules

◆ log_item_wellknown_keys

const log_item_wellknown_key log_item_wellknown_keys[]
static

We support a number of predefined keys, such as "error-code" or "message".

These are defined here. We also support user-defined "ad hoc" (or "generic") keys that let users of the error stack add values with arbitrary keys (as long as those keys don't coincide with the wellknown ones, anyway).

The idea here is that we want the flexibility of arbitrary keys, while being able to do certain optimizations for the common case. This also allows us to relatively cheaply add some convenience features, e.g. we know that error symbol ("ER_STARTUP") and error code (1451) are related, and can supply one as the other is submitted. Likewise of course, we can use the error code to fetch the associated registered error message string for that error code. Et cetera!

◆ log_item_wellknown_keys_count

uint log_item_wellknown_keys_count
static
Initial value:
=
static const log_item_wellknown_key log_item_wellknown_keys[]
We support a number of predefined keys, such as "error-code" or "message".
Definition: log_builtins.cc:314
Pre-defined "well-known" keys, as opposed to ad hoc ones, for key/value pairs in logging.
Definition: log_builtins.cc:291

◆ log_line_buffer_start

log_line_buffer* log_line_buffer_start = nullptr
static

Pointer to the first element in the list of buffered log messages.

◆ log_line_buffer_tail

log_line_buffer** log_line_buffer_tail = &log_line_buffer_start
static

Where to write the pointer to the newly-created tail-element of the list.

◆ log_service_cache

collation_unordered_map<string, cache_entry_with_deleter>* log_service_cache
static

◆ log_service_instances

log_service_instance* log_service_instances = nullptr
static

anchor

◆ log_sink_trad_last

ulonglong log_sink_trad_last = 0
static

◆ THR_LOCK_log_buffered

mysql_mutex_t THR_LOCK_log_buffered
static

Make sure only one instance of the buffered "writer" runs at a time.

In normal operation, the log-event will be created dynamically, then it will be fed through the pipeline, and then it will be released. Since the event is allocated in the caller, we can be sure it won't go away wholesale during processing, and since the event is local to the caller, no other thread will tangle with it. It is therefore safe in those cases not to wrap a lock around the event. (The log-pipeline will still grab a shared lock, THR_LOCK_log_stack, to protect the pipeline (not the event) and the log-services cache from being changed while the pipeline is being applied. Likewise, log-services may protect their resources (file-writers will usually take a lock to serialize their writes; the built-in filter will take a lock on its rule-set as that is shared between concurrent threads running the filter, and so on). None of these are intended to protect the event itself though.

In buffered mode on the other hand, we copy each log-event (the original of which, see above, is owned by the caller and local to the thread, and therefore safe without locking) to a global buffer / backlog. As this backlog can be added to by all threads, it must be protected by a lock (once we have fully initialized the subsystem with log_builtins_init() and support multi-threaded mode anyway, as indicated by log_builtins_inited being true, see below). This is that lock.

◆ THR_LOCK_log_stack

mysql_rwlock_t THR_LOCK_log_stack
static

Lock for the log "stack" (i.e.

the list of active log-services). X-locked while stack is changed/configured. S-locked while stack is used.

◆ THR_LOCK_log_syseventlog

mysql_mutex_t THR_LOCK_log_syseventlog
static

Make sure only one instance of syslog/Eventlog code runs at a time.