 |
MySQL 8.3.0
Source Code Documentation
|
|
MySQL 8.3.0
Source Code Documentation
|
#include <mysqld_error.h>#include "log_builtins_filter_imp.h"#include "log_builtins_imp.h"#include "m_string.h"#include "my_atomic.h"#include "my_systime.h"#include "mysys_err.h"#include "sql/derror.h"#include "sql/log.h"#include "sql/mysqld.h"Macros | |
| #define | THROTTLE_DEFAULT_WINDOW_SIZE_IN_SECONDS 60 |
| services: log filter: basic filtering More... | |
| #define | THROTTLE_MICROSECOND_MULTIPLIER 1000000 |
Functions | |
| static bool | log_filter_ruleset_full (log_filter_ruleset *rs) |
| Predicate: can we add any more rules? More... | |
| static log_filter_rule * | log_builtins_filter_rule_init (log_filter_ruleset *ruleset) |
| Initialize a new rule. More... | |
| static int | log_builtins_filter_rule_free (log_filter_rule *ri) |
| Release all resources associated with a filter rule. More... | |
| static void | log_builtins_filter_rule_remove (log_filter_ruleset *ruleset, int elem) |
| Release filter rule (key/value pair) with the index "elem" in "ruleset". More... | |
| static log_filter_ruleset * | log_builtins_filter_ruleset_new (log_filter_tag *tag, size_t count) |
| Create a new set of filter rules. More... | |
| static int | log_builtins_filter_ruleset_lock (log_filter_ruleset *ruleset, log_builtins_filter_lock lt) |
| Lock and get the filter rules. More... | |
| static void | log_builtins_filter_ruleset_drop (log_filter_ruleset *ruleset) |
| Drop an entire filter rule-set. More... | |
| static void | log_builtins_filter_ruleset_unlock (log_filter_ruleset *ruleset) |
| Unlock filter ruleset. More... | |
| static void | log_builtins_filter_ruleset_free (log_filter_ruleset **ruleset) |
| Free filter ruleset. More... | |
| static void | log_builtins_filter_set_defaults (log_filter_ruleset *ruleset) |
| Defaults for when the configuration engine isn't loaded; aim for 5.7 compatibility. More... | |
| int | log_builtins_filter_exit () |
| Deinitialize filtering engine. More... | |
| int | log_builtins_filter_init () |
| Initialize filtering engine. More... | |
| static log_filter_apply | log_filter_try_apply (log_line *ll, int ln, log_filter_rule *r) |
| Apply the action of an individual rule to an individual log line (or a part thereof, i.e. More... | |
| static log_filter_match | log_filter_try_match (log_item *li, log_filter_rule *ri) |
| Try to match an individual log line-field against an individual rule's condition. More... | |
| int | log_builtins_filter_run (log_filter_ruleset *ruleset, log_line *ll) |
| Apply all matching rules from a filter rule set to a given log line. More... | |
| int | log_builtins_filter_update_verbosity (int verbosity) |
| This is part of the 5.7 emulation: If –log_error_verbosity is changed, we generate an artificial filter rule from it here. More... | |
| int | log_builtins_filter_parse_suppression_list (char *list, bool update) |
| @global.log_error_suppression_list accepts a comma-separated list of error-codes that should not be included in the error-log. More... | |
Variables | |
| static bool | filter_inited = false |
| static ulong | filter_rule_uuid = 0 |
| log_filter_ruleset * | log_filter_builtin_rules = nullptr |
| log_filter_tag | rule_tag_builtin = {"log_filter_builtin", nullptr} |
| #define THROTTLE_DEFAULT_WINDOW_SIZE_IN_SECONDS 60 |
services: log filter: basic filtering
This implementation, "dragnet" is currently the default filter and therefore built-in. Basic configuration is built into the server proper (via log_error_verbosity etc.); for advanced configuration, load the service log_filter_dragnet which implements a configuration language for this engine. See there for details about the Configuration Stage. Some of the code-paths are only available via the configuration language or an equivalent service (but not without any such service loaded).
At present, the design is such that multiple threads can call the filter concurrently; the ruleset is global and shared between all users.
FILTERING STAGE
At run time, the filter iterates over its rule-set. For each rule, if the condition contains a well-known item, it looks for an item of that type in the event. If the condition contains an ad hoc-item, it looks for an item of any ad hoc-type with the given key within the event.
If there is a match, the filter will verify whether the storage class of the value in the event and that in the condition are either both strings, or both not. If the classes do not match, it flags an error. Otherwise, it now compares both values using the requested comparator, and reports the result.
If a log event matches a rule, an action ("suppress log line", "delete field", etc.) will be applied to that event.
LOCKING
During the filtering stage, a shared lock on the ruleset is held. An exclusive lock on the ruleset is only taken as response to the user's changing of the filter configuration, which should be rare.
For debugging puroposes, rules feature a counter of how often events matched them; this counter is updated atomically.
Rate-limiting ("throttle") needs some bookkeeping data (when does the current window expire? how many matches have we had so far within the current window? etc.). A write-lock is taken on the individual rule (not the entire ruleset) to update this information; any throttling-related actions taken on the event happen after this lock has been released.
The event itself is not locked.
| #define THROTTLE_MICROSECOND_MULTIPLIER 1000000 |
| int log_builtins_filter_exit | ( | ) |
Deinitialize filtering engine.
| 0 | Success! |
| -1 | De-initialize? Filter wasn't even initialized! |
| int log_builtins_filter_init | ( | ) |
Initialize filtering engine.
We need to do this early, before the component system is up.
| 0 | Success! |
| -1 | Couldn't initialize ruleset |
| -2 | Filter was already initialized? |
| int log_builtins_filter_parse_suppression_list | ( | char * | list, |
| bool | update | ||
| ) |
@global.log_error_suppression_list accepts a comma-separated list of error-codes that should not be included in the error-log.
Events with a severity of System or Error can not be filtered in this way and will always be forwarded to the log-sinks.
This provides simple filtering for cases where the flexibility of the loadable filter-language is not needed. (The same engine is used however, just with a more limited interface.)
For this filtering to be active, @global.log_error_services has to feature "log_filter_internal", as it does by default. When that is the case, one or both of log_error_verbosity and this variable may be used. Only one of "log_filter_internal" and "log_filter_dragnet" should be used at a time.
The semantics follow that of our system variables; that is to say, when called with update==false, the function acts as a check-function that validates the entire list given to it; when called with update==true, it creates filter-rules from the list items. This way, we either create all rules, or no rules, rather than ending up with an incomplete rule-set when we encounter a problem in the input.
The return value encodes the location in the argument where the failure occurred, like so:
| list | list of error-codes that should not appear in the error-log |
| update | false: verify list only true: create filtering rules from suppression list |
| 0 | success |
| !0 | failure (see above) |
|
static |
Release all resources associated with a filter rule.
Leaves a "gap" (an uninitialized rule) for immediate re-filling; if this is undesired, use log_builtins_filter_rule_remove() (see there). Must hold rule-set lock.
| ri | the rule to release |
| the | return value from mysql_rwlock_destroy() |
|
static |
Initialize a new rule.
This clears the first unused rule. It does not update the rules count; this is for the caller to do if it succeeds in setting up the rule to its satisfaction. If the caller fails, it should log_builtins_filter_rule_free() the incomplete rule.
| nullptr | could not initialize rule. Do not call rule_free. |
| !nullptr | the address of the rule. fill in. on success, caller must increase rule count. on failure, it must call rule_free. |
|
static |
Release filter rule (key/value pair) with the index "elem" in "ruleset".
This frees whichever of key and value were dynamically allocated. It then moves any trailing items to fill the "gap" and decreases the counter of elements in the rule-set.
If the intention is to leave a "gap" in the bag that may immediately be overwritten with an updated element, use log_builtins_filter_rule_free() instead.
Caller must hold rule-set lock.
| ruleset | filter rule-set |
| elem | index of the filter rule to release |
|
static |
Drop an entire filter rule-set.
Must hold lock.
|
static |
Free filter ruleset.
|
static |
Lock and get the filter rules.
| ruleset | the ruleset to lock |
| lt | LOG_BUILTINS_LOCK_SHARED lock for reading LOG_BUILTINS_LOCK_EXCLUSIVE lock for writing |
| 0 | lock acquired |
| !0 | failed to acquire lock |
|
static |
Create a new set of filter rules.
| tag | tag for this ruleset |
| count | number of rules to allocate, 0 for default |
| a | pointer to a ruleset structure, or nullptr on failure |
|
static |
Unlock filter ruleset.
| int log_builtins_filter_run | ( | log_filter_ruleset * | ruleset, |
| log_line * | ll | ||
| ) |
Apply all matching rules from a filter rule set to a given log line.
| ruleset | rule-set to apply |
| ll | the current log line |
| int | number of matched rules |
|
static |
Defaults for when the configuration engine isn't loaded; aim for 5.7 compatibility.
| int log_builtins_filter_update_verbosity | ( | int | verbosity | ) |
This is part of the 5.7 emulation: If –log_error_verbosity is changed, we generate an artificial filter rule from it here.
For this filtering to be active, @global.log_error_services has to feature "log_filter_internal", as it does by default. When that is the case, one or both of log_error_verbosity and log_error_suppression_list (see below) may be used. Only one of "log_filter_internal" and "log_filter_dragnet" should be used at a time.
| verbosity | log_error_verbosity style, range(1,3) 1:errors, 2:+=warnings, 3:+=notes |
| 0 | success |
| !0 | failure |
|
static |
Predicate: can we add any more rules?
| rs | the ruleset to check |
| true | full, no more rules can be added |
| false | not full, further rules can be added |
|
static |
Apply the action of an individual rule to an individual log line (or a part thereof, i.e.
a "field"). At this point, we already know that the current log line matches the condition.
| [in,out] | ll | the current log line |
| ln | index of the matching field, -1 for none (when a test for absence matched) | |
| r | the rule to apply. internal state may be changed (i.e. the number of seen matches for a throttle rule) |
| log_filter_apply | 0 on success, an error-code otherwise |
|
static |
Try to match an individual log line-field against an individual rule's condition.
| li | the log item we try to match |
| ri | the rule containing the condition |
| log_filter_match | 0 (LOG_FILTER_MATCH_SUCCESS) on match, 1 (LOG_FILTER_MATCH_UNSATISFIED) or an error-code otherwise |
|
static |
|
static |
| log_filter_ruleset* log_filter_builtin_rules = nullptr |
| log_filter_tag rule_tag_builtin = {"log_filter_builtin", nullptr} |
1.9.2