MySQL 8.0.33
Source Code Documentation
Go to the documentation of this file.
1/* Copyright (c) 2011, 2023, Oracle and/or its affiliates.
3 This program is free software; you can redistribute it and/or modify
4 it under the terms of the GNU General Public License, version 2.0,
5 as published by the Free Software Foundation.
7 This program is also distributed with certain software (including
8 but not limited to OpenSSL) that is licensed under separate terms,
9 as designated in a particular file or component or in included license
10 documentation. The authors of MySQL hereby grant you an additional
11 permission to link the program and your derivative works with the
12 separately licensed software that they have included with MySQL.
14 This program is distributed in the hope that it will be useful,
15 but WITHOUT ANY WARRANTY; without even the implied warranty of
17 GNU General Public License, version 2.0, for more details.
19 You should have received a copy of the GNU General Public License
20 along with this program; if not, write to the Free Software
21 Foundation, Inc., 51 Franklin St, Fifth Floor, Boston, MA 02110-1301 USA */
26#include <limits.h>
27#include <string.h>
28#include <sys/types.h>
30#include "my_compiler.h"
31#include "my_inttypes.h"
32#include "my_sqlcommand.h" // enum_sql_command
33#include "sql/opt_trace_context.h" // Opt_trace_context
35class Cost_estimate;
36class Item;
37class THD;
38class set_var_base;
39class sp_head;
40class sp_printable;
41struct CHARSET_INFO;
42class Table_ref;
43template <class T>
44class List;
47 @file sql/opt_trace.h
48 API for the Optimizer trace (WL#5257)
52 @page PAGE_OPT_TRACE The Optimizer Trace
54 @section INTRODUCTION Introduction
56 This optimizer trace is aimed at producing output, which is readable by
57 humans and by programs, to aid understanding of decisions and actions taken
58 by the MySQL Optimizer.
60 @section OUTPUT_FORMAT Output format
62 The chosen output format is JSON (JavaScript Object Notation).
63 In JSON there are:
64 @li "objects" (unordered set of key-value pairs); equivalent to Python's
65 dictionary or Perl's associative array or hash or STL's hash_map.
66 @li "arrays" (ordered set of values); equivalent to Python's and Perl's list
67 or STL's vector.
68 @li "values": a value can be a string, number, boolean, null,
69 which we all call "scalars", or be an object, array.
71 For example (explanations after "<<" are not part of output):
73 { << start of top object
74 "first_name": "Gustave", << key/value pair (value is string)
75 "last_name": "Eiffel", << key/value pair (value is string)
76 "born": 1832, << key/value pair (value is integer)
77 "contributions_to": [ << key/value pair (value is array)
78 { << 1st item of array is an object (a building)
79 "name": "Eiffel tower",
80 "location": Paris
81 }, << end of 1st item of array
82 {
83 "name": "Liberty statue",
84 "location": "New York"
85 } << end of 2nd item of array
86 ] << end of array
87 } << end of top object
89 For more details, have a look at the syntax at
90 Note that indentation and newlines are superfluous, useful only for
91 human-readability.
92 Note also that there is nothing like a "named object": an object, array or
93 value has no name; but if it is the value of a key/value pair in an
94 enclosing, outer object, then the key can be seen as the inner object's
95 "name".
97 @section USER_ENABLE_TRACING How a user enables/views the trace
100 SET SESSION OPTIMIZER_TRACE="enabled=on"; # enable tracing
101 <statement to trace>; # like SELECT, EXPLAIN SELECT, UPDATE, DELETE...
102 SELECT * FROM information_schema.OPTIMIZER_TRACE;
103 [ repeat last two steps at will ]
104 SET SESSION OPTIMIZER_TRACE="enabled=off"; # disable tracing
107 @c SELECT and @c EXPLAIN SELECT produce the same trace. But there are
108 exceptions regarding subqueries because the two commands treat subqueries
109 differently, for example in
111 SELECT ... WHERE x IN (subq1) AND y IN (subq2)
113 SELECT terminates after executing the first subquery if the related IN
114 predicate is false, so we won't see @c JOIN::optimize() tracing for subq2;
115 whereas EXPLAIN SELECT analyzes all subqueries (see loop at the end of
116 @c select_describe()).
118 @section USER_SELECT_TRACING_STATEMENTS How a user traces only certain
121 When tracing is in force, each SQL statement generates a trace; more
122 exactly, so does any of
126 UPDATE/DELETE and their multi-table variants,
127 SET (unless it manipulates @@@@optimizer_trace),
128 DO,
129 DECLARE/CASE/IF/RETURN (stored routines language elements),
130 CALL.
131 If a command above is prepared and executed in separate steps, preparation
132 and execution are separately traced.
133 By default each new trace overwrites the previous trace. Thus, if a
134 statement contains sub-statements (example: invokes stored procedures,
135 stored functions, triggers), the top statement and sub-statements each
136 generate traces, but at the execution's end only the last sub-statement's
137 trace is visible.
138 If the user wants to see the trace of another sub-statement, she/he can
139 enable/disable tracing around the desired sub-statement, but this requires
140 editing the routine's code, which may not be possible. Another solution is
141 to use
143 SET optimizer_trace_offset=<OFFSET>, optimizer_trace_limit=<LIMIT>
145 where OFFSET is a signed integer, and LIMIT is a positive integer.
146 The effect of this SET is the following:
148 @li all remembered traces are cleared
150 @li a later SELECT on OPTIMIZER_TRACE returns the first LIMIT traces of
151 the OFFSET oldest remembered traces (if OFFSET >= 0), or the first LIMIT
152 traces of the -OFFSET newest remembered traces (if OFFSET < 0).
154 For example,
155 a combination of OFFSET=-1 and LIMIT=1 will make the last trace be shown (as
156 is default), OFFSET=-2 and LIMIT=1 will make the next-to-last be shown,
157 OFFSET=-5 and LIMIT=5 will make the last five traces be shown. Such negative
158 OFFSET can be useful when one knows that the interesting sub-statements are
159 the few last ones of a stored routine, like this:
161 SET optimizer_trace_offset=-5, optimizer_trace_limit=5;
162 CALL stored_routine(); # more than 5 sub-statements in this routine
163 SELECT * FROM information_schema.OPTIMIZER_TRACE; # see only last 5 traces
165 On the opposite, a positive OFFSET can be useful when one knows that the
166 interesting sub-statements are the few first ones of a stored routine.
168 The more those two variables are accurately adjusted, the less memory is
169 used. For example, OFFSET=0 and LIMIT=5 will use memory to remember 5
170 traces, so if only the three first are needed, OFFSET=0 and LIMIT=3 is
171 better (tracing stops after the LIMITth trace, so the 4th and 5th trace are
172 not created and don't take up memory). A stored routine may have a loop
173 which executes many sub-statements and thus generates many traces, which
174 would use a lot of memory; proper OFFSET and LIMIT can restrict tracing to
175 one iteration of the loop for example. This also gains speed, as tracing a
176 sub-statement impacts performance.
178 If OFFSET>=0, only LIMIT traces are kept in memory. If OFFSET<0, that is not
179 true: instead, (-OFFSET) traces are kept in memory; indeed even if LIMIT is
180 smaller than (-OFFSET), so excludes the last statement, the last statement
181 must still be traced because it will be inside LIMIT after executing one
182 more statement (remember than OFFSET<0 is counted from the end: the "window"
183 slides as more statements execute).
185 Such memory and speed gains are the reason why optimizer_trace_offset/limit,
186 which are restrictions at the trace producer level, are offered. They are
187 better than using
191 which is a restriction on the trace consumer level, which saves almost
192 nothing.
194 @section USER_SELECT_TRACING_FEATURES How a user traces only certain
195 optimizer features
198 SET OPTIMIZER_TRACE_FEATURES="feature1=on|off,...";
200 where "feature1" is one optimizer feature. For example "greedy_search": a
201 certain Opt_trace_array at the start of @c
202 Optimize_table_order::choose_table_order() has a flag "GREEDY_SEARCH" passed
203 to its constructor: this means that if the user has turned tracing of greedy
204 search off, this array will not be written to the I_S trace, neither will
205 any children structures. All this disabled "trace chunk" will be replaced by
206 an ellipsis "...".
208 @section DEV_ADDS_TRACING How a developer adds tracing to a function
210 Check @c Opt_trace* usage in @c advance_sj_state():
213 Opt_trace_array trace_choices(trace, "semijoin_strategy_choice");
216 This creates an array for key "semijoin_strategy_choice". We are going to
217 list possible semijoin strategy choices.
220 Opt_trace_object trace_one_strategy(trace);
223 This creates an object without key (normal, it's in an array). This
224 object will describe one single strategy choice.
227 trace_one_strategy.add_alnum("strategy", "FirstMatch");
230 This adds a key/value pair to the just-created object: key is "strategy",
231 value is "FirstMatch". This is the strategy to be described in the
232 just-created object.
235 trace_one_strategy.add("cost", *current_read_time).
236 add("records", *current_record_count);
237 trace_one_strategy.add("chosen", (pos->sj_strategy == SJ_OPT_FIRST_MATCH));
240 This adds 3 key/value pairs: cost of strategy, number of records produced
241 by this strategy, and whether this strategy is chosen.
243 After that, there is similar code for other semijoin strategies.
245 The resulting trace piece (seen in @c information_schema.OPTIMIZER_TRACE) is
247 "semijoin_strategy_choice": [
248 {
249 "strategy": "FirstMatch",
250 "cost": 1,
251 "records": 1,
252 "chosen": true
253 },
254 {
255 "strategy": "DuplicatesWeedout",
256 "cost": 1.1,
257 "records": 1,
258 "duplicate_tables_left": false,
259 "chosen": false
260 }
261 ]
264 For more output examples, check result files of the opt_trace suite in
265 @c mysql-test.
267 Feature can be un-compiled with @code cmake -DOPTIMIZER_TRACE=0 @endcode.
269 @section WITH_DBUG Interaction between trace and DBUG
271 We don't want to have to duplicate code like this:
273 DBUG_PRINT("info",("cost %g",cost));
274 Opt_trace_object(thd->opt_trace).add("cost",cost);
277 Thus, any optimizer trace operation, *even* if tracing is run-time disabled,
278 has an implicit DBUG_PRINT("opt",...) inside. This way, only the
279 second line above is needed, and several DBUG_PRINT() could be removed from
280 the Optimizer code.
281 When tracing is run-time disabled, in a debug binary, traces are still
282 created in order to catch the @c add() calls and write their text to DBUG,
283 but those traces are not visible into INFORMATION_SCHEMA.OPTIMIZER_TRACE: we
284 then say that they "don't support I_S".
285 A debug binary without optimizer trace compiled in, will intentionally not
286 compile.
288 Because opening an object or array, or add()-ing to it, writes to DBUG
289 immediately, a key/value pair and its outer object may be 100 lines
290 apart in the DBUG log.
292 @section ADDING_TRACING Guidelines for adding tracing
294 @li Try to limit the number of distinct "words". For example, when
295 describing an optimizer's decision, the words "chosen" (true/false value,
296 tells whether we are choosing the said optimization), "cause" (free text
297 value, tells why we are making this choice, when it's not obvious)
298 can and should often be used. Having a restricted vocabulary helps
299 consistency. Use "row" instead of "record". Use "tmp" instead of
300 "temporary".
302 @li Use only simple characters for key names: a-ZA-Z_#, and no space. '#'
303 serves to denote a number, like in "select#" .
305 @li Keep in mind than in an object, keys are not ordered; an application may
306 parse the JSON output and output it again with keys order changed; thus
307 when order matters, use an array (which may imply having anonymous objects
308 as items of the array, with keys inside the anonymous objects, see how it's
309 done in @c JOIN::optimize()). Keep in mind that in an object keys should
310 be unique, an application may lose duplicate keys.
312 @section OOM_HANDLING Handling of "out-of-memory" errors
314 All memory allocations (with exceptions: see below) in the Optimizer trace
315 use @c my_error() to report errors, which itself calls @c
316 error_handler_hook. It is the responsibility of the API user to set up a
317 proper @c error_handler_hook which will alert her/him of the OOM
318 problem. When in the server, this is already the case (@c error_handler_hook
319 is @c my_message_sql() which makes the statement fail).
320 Note that the debug binary may crash if OOM (OOM can cause syntax
321 errors...).
323 @section TRACE_SECURITY Description of trace-induced security checks.
325 A trace exposes information. For example if one does SELECT on a view, the
326 trace contains the view's body. So, the user should be allowed to see the
327 trace only if she/he has privilege to see the body, i.e. privilege to do
329 There are similar issues with stored procedures, functions, triggers.
331 We implement this by doing additional checks on SQL objects when tracing is
332 on:
333 @li stored procedures, functions, triggers: checks are done when executing
334 those objects
335 @li base tables and views.
337 Base tables or views are listed in some @c LEX::query_tables.
338 The LEX may be of the executing statement (statement executed by
339 @c mysql_execute_command(), or by
340 @c sp_lex_keeper::reset_lex_and_exec_core()), we check this LEX in the
341 constructor of Opt_trace_start.
342 Or it may be a LEX describing a view, we check this LEX when
343 opening the view (@c open_and_read_view()).
345 Those checks are greatly simplified by disabling traces in case of security
346 context changes. @see opt_trace_disable_if_no_security_context_access().
348 Those checks must be done with the security context of the connected
349 user. Checks with the SUID context would be useless: assume the design is
350 that the basic user does not have DML privileges on tables, but only
351 EXECUTE on SUID-highly-privileged routines (which implement _controlled_
352 _approved_ DMLs): then the SUID context would successfully pass all
353 additional privilege checks, routine would generate tracing, and the
354 connected user would view the trace after the routine's execution, seeing
355 secret information.
357 @section NEXT What a developer should read next
359 The documentation of those classes, in order
361 Opt_trace_context
362 Opt_trace_context_impl
363 Opt_trace_stmt
364 Opt_trace_struct
365 Opt_trace_object
366 Opt_trace_array
368 and then @ref opt_trace.h as a whole.
371class Opt_trace_stmt; // implementation detail local to
374 User-visible information about a trace. @sa Opt_trace_iterator.
377 /**
378 String containing trace.
379 If trace has been end()ed, this is 0-terminated, which is only to aid
380 debugging or unit testing; this property is not relied upon in normal
381 server usage.
382 If trace has not been ended, this is not 0-terminated. That rare case can
383 happen when a substatement reads OPTIMIZER_TRACE (at that stage, the top
384 statement is still executing so its trace is not ended yet, but may still
385 be read by the sub-statement).
386 */
387 const char *trace_ptr;
388 size_t trace_length; ///< length of trace string
389 //// String containing original query. 0-termination: like trace_ptr.
390 const char *query_ptr;
391 size_t query_length; ///< length of query string
392 const CHARSET_INFO *query_charset; ///< charset of query string
393 /**
394 How many bytes this trace is missing (for traces which were truncated
395 because of @@@@optimizer-trace-max-mem-size).
396 */
398 bool missing_priv; ///< whether user lacks privilege to see this trace
402 Iterator over the list of remembered traces.
403 @note due to implementation, the list must not change during an
404 iterator's lifetime, or results may be unexpected (no crash though).
407 public:
408 /**
409 @param ctx context
410 */
413 void next(); ///< Advances iterator to next trace.
415 /**
416 Provides information about the trace on which the iterator is
417 positioned.
418 @param[out] info information returned.
419 The usage pattern is
420 1) instantiate the iterator
421 2) test at_end(), if false: call get_value() and then next()
422 3) repeat (2) until at_end() is true.
423 */
424 void get_value(Opt_trace_info *info) const;
426 /// @returns whether iterator is positioned to the end.
427 bool at_end() const { return cursor == nullptr; }
429 private:
430 /// Pointer to context, from which traces are retrieved
432 const Opt_trace_stmt *cursor; ///< trace which the iterator is positioned on
433 long row_count; ///< how many traces retrieved so far
437 Object and array are both "structured data" and have lots in common, so the
438 Opt_trace_struct is a base class for them.
439 When you want to add a structure to the trace, you create an instance of
440 Opt_trace_object or Opt_trace_array, then you add information to it with
441 add(), then the destructor closes the structure (we use RAII, Resource
442 Acquisition Is Initialization).
446 protected:
447 /**
448 @param ctx_arg Optimizer trace context for this structure
449 @param requires_key_arg whether this structure requires/forbids keys
450 for values put inside it (an object requires them, an
451 array forbids them)
452 @param key key if this structure is the value of a key/value pair,
453 NULL otherwise. This pointer must remain constant and
454 valid until the object is destroyed (to support
455 @ref saved_key).
456 @param feature optimizer feature to which this structure belongs
458 This constructor is never called directly, only from subclasses.
459 */
460 Opt_trace_struct(Opt_trace_context *ctx_arg, bool requires_key_arg,
461 const char *key, Opt_trace_context::feature_value feature)
462 : started(false) {
463 // A first inlined test
464 if (unlikely(ctx_arg->is_started())) {
465 // Tracing enabled: must fully initialize the structure.
466 do_construct(ctx_arg, requires_key_arg, key, feature);
467 }
468 /*
469 Otherwise, just leave "started" to false, it marks that the structure is
470 dummy.
471 */
472 }
475 }
477 public:
478 /**
479 The exception to RAII: this function is an explicit way of ending a
480 structure before it goes out of scope. Don't use it unless RAII mandates
481 a new scope which mandates re-indenting lots of code lines.
482 */
483 void end() {
485 }
487 /**
488 Adds a value (of string type) to the structure. A key is specified, so it
489 adds the key/value pair (the structure must thus be an object).
491 There are two "add_*" variants to add a string value.
492 If the value is 0-terminated and each character
493 - is ASCII 7-bit
494 - has ASCII code >=32 and is neither '"' nor '\\'
495 then add_alnum() should be used. That should be the case for all fixed
496 strings like add_alnum("cause", "cost").
497 Otherwise, add_utf8() should be used; it accepts any UTF8-encoded
498 character in 'value' and will escape characters which JSON requires (and
499 is thus slower than add_alnum()). It should be used for all strings which
500 we get from the server's objects (indeed a table's name, a WHERE
501 condition, may contain "strange" characters).
503 @param key key
504 @param value value
505 @returns a reference to the structure, useful for chaining like this:
506 @verbatim add(x,y).add(z,t).add(u,v) @endverbatim
508 String-related add() variants are named add_[something]():
509 - to avoid confusing the compiler between:
510 add(const char *value, size_t val_length) and
511 add(const char *key, ulonglong value)
512 - and because String::length() returns uint32 and not size_t, so for
513 add(str.ptr(), str.length())
514 compiler may pick
515 add(const char *key, double value) instead of
516 add(const char *value, size_t val_length).
517 */
518 Opt_trace_struct &add_alnum(const char *key, const char *value) {
519 if (likely(!started)) return *this;
520 return do_add(key, value, strlen(value), false);
521 }
523 /**
524 Adds a value (of string type) to the structure. No key is specified, so
525 it adds only the value (the structure must thus be an array).
526 @param value value
527 @returns a reference to the structure
528 */
529 Opt_trace_struct &add_alnum(const char *value) {
530 if (likely(!started)) return *this;
531 return do_add(nullptr, value, strlen(value), false);
532 }
534 /**
535 Like add_alnum() but supports any UTF8 characters in 'value'.
536 Will "escape" 'value' to be JSON-compliant.
537 @param key key
538 @param value value
539 @param val_length length of string 'value'
540 */
541 Opt_trace_struct &add_utf8(const char *key, const char *value,
542 size_t val_length) {
543 if (likely(!started)) return *this;
544 return do_add(key, value, val_length, true);
545 }
547 /// Variant of add_utf8() for adding to an array (no key)
548 Opt_trace_struct &add_utf8(const char *value, size_t val_length) {
549 if (likely(!started)) return *this;
550 return do_add(nullptr, value, val_length, true);
551 }
553 /// Variant of add_utf8() where 'value' is 0-terminated
554 Opt_trace_struct &add_utf8(const char *key, const char *value) {
555 if (likely(!started)) return *this;
556 return do_add(key, value, strlen(value), true);
557 }
559 /// Variant of add_utf8() where 'value' is 0-terminated
560 Opt_trace_struct &add_utf8(const char *value) {
561 if (likely(!started)) return *this;
562 return do_add(nullptr, value, strlen(value), true);
563 }
565 /**
566 Add a value (of Item type) to the structure. The Item should be a
567 condition (like a WHERE clause) which will be pretty-printed into the
568 trace. This is useful for showing condition transformations (equality
569 propagation etc).
570 @param key key
571 @param item the Item
572 @return a reference to the structure
573 */
574 Opt_trace_struct &add(const char *key, Item *item) {
575 if (likely(!started)) return *this;
576 return do_add(key, item);
577 }
579 if (likely(!started)) return *this;
580 return do_add(nullptr, item);
581 }
583 public:
584 Opt_trace_struct &add(const char *key, bool value) {
585 if (likely(!started)) return *this;
586 return do_add(key, value);
587 }
588 Opt_trace_struct &add(bool value) {
589 if (likely(!started)) return *this;
590 return do_add(nullptr, value);
591 }
592 Opt_trace_struct &add(const char *key, int value) {
593 if (likely(!started)) return *this;
594 return do_add(key, static_cast<longlong>(value));
595 }
596 Opt_trace_struct &add(int value) {
597 if (likely(!started)) return *this;
598 return do_add(nullptr, static_cast<longlong>(value));
599 }
600 Opt_trace_struct &add(const char *key, uint value) {
601 if (likely(!started)) return *this;
602 return do_add(key, static_cast<ulonglong>(value));
603 }
605 if (likely(!started)) return *this;
606 return do_add(nullptr, static_cast<ulonglong>(value));
607 }
608 Opt_trace_struct &add(const char *key, ulong value) {
609 if (likely(!started)) return *this;
610 return do_add(key, static_cast<ulonglong>(value));
611 }
612 Opt_trace_struct &add(ulong value) {
613 if (likely(!started)) return *this;
614 return do_add(nullptr, static_cast<ulonglong>(value));
615 }
616 Opt_trace_struct &add(const char *key, longlong value) {
617 if (likely(!started)) return *this;
618 return do_add(key, value);
619 }
621 if (likely(!started)) return *this;
622 return do_add(nullptr, value);
623 }
624 Opt_trace_struct &add(const char *key, ulonglong value) {
625 if (likely(!started)) return *this;
626 return do_add(key, value);
627 }
629 if (likely(!started)) return *this;
630 return do_add(nullptr, value);
631 }
632 Opt_trace_struct &add(const char *key, double value) {
633 if (likely(!started)) return *this;
634 return do_add(key, value);
635 }
636 Opt_trace_struct &add(double value) {
637 if (likely(!started)) return *this;
638 return do_add(nullptr, value);
639 }
640 /// Adds a JSON null object (==Python's "None")
642 if (likely(!started)) return *this;
643 return do_add_null(key);
644 }
645 /**
646 Helper to put the database/table name in an object.
647 @param tab TABLE* pointer
648 */
650 if (likely(!started)) return *this;
651 return do_add_utf8_table(tab);
652 }
653 /**
654 Helper to put the number of query_block in an object.
655 @param select_number number of query_block
656 */
658 return add("select#", select_number);
659 }
660 /**
661 Add a value to the structure.
662 @param key key
663 @param cost the value of Cost_estimate
664 @return a reference to the structure
665 */
666 Opt_trace_struct &add(const char *key, const Cost_estimate &cost) {
667 if (likely(!started)) return *this;
668 return do_add(key, cost);
669 }
671 /**
672 Informs this structure that we are adding data (scalars, structures) to
673 it.
674 This is used only if sending to I_S.
675 @returns whether the structure was empty so far.
676 @note this is reserved for use by Opt_trace_stmt.
677 */
679 const bool old_empty = empty;
680 empty = false;
681 return old_empty;
682 }
683 /**
684 Validates the key about to be added.
685 @note this is reserved for use by Opt_trace_stmt.
687 When adding a value (or array or object) to an array, or a key/value pair
688 to an object, we need to know this outer array or object.
690 It would be possible, when trying to add a key to an array, which is wrong
691 in JSON, or similarly when trying to add a value without any key to an
692 object, to catch it at compilation time, if the adder received, as
693 function parameter, the type of the structure (like @c
694 Opt_trace_array*). Then the @c add(key,val) call would not compile as
695 Opt_trace_array wouldn't feature it.
697 But as explained in comment of class Opt_trace_context we
698 cannot pass down the object, have to maintain a "current object or
699 array" in the Opt_trace_context context (pointer to an instance of
700 Opt_trace_struct), and the adder grabs it from the context.
702 As this current structure is of type "object or array", we cannot do
703 compile-time checks that only suitable functions are used. A call to @c
704 add(key,value) is necessarily legal for the compiler as the structure may
705 be an object, though it will be wrong in case the structure is actually
706 an array at run-time. Thus we have the risk of an untested particular
707 situation where the current structure is not an object (but an array)
708 though the code expected it to be one. This happens in practice, because
709 subqueries are evaluated in many possible places of code, not all of them
710 being known. Same happens, to a lesser extent, with calls to the range
711 optimizer.
712 So at run-time, in check_key(), we detect wrong usage, like adding a value
713 to an object without specifying a key, and then remove the unnecessary
714 key, or add an autogenerated key.
715 */
716 const char *check_key(const char *key);
718 private:
719 /// Not implemented, use add_alnum() instead.
720 Opt_trace_struct &add(const char *key, const char *value);
721 Opt_trace_struct &add(const char *key);
723 /// Full initialization. @sa Opt_trace_struct::Opt_trace_struct
724 void do_construct(Opt_trace_context *ctx, bool requires_key, const char *key,
726 /// Really does destruction
727 void do_destruct();
728 /**
729 Really adds to the object. @sa add().
731 @note add() has an up-front if(), hopefully inlined, so that in the
732 common case - tracing run-time disabled - we have no function call. If
733 tracing is enabled, we call do_add().
734 In a 20-table plan search (as in BUG#50595), the execution time was
735 decreased from 2.6 to 2.0 seconds thanks to this inlined-if trick.
737 @param key key
738 @param value value
739 @param val_length length of string 'value'
740 @param escape do JSON-compliant escaping of 'value'. If 'escape' is
741 false, 'value' should be ASCII. Otherwise, should be UTF8.
742 */
743 Opt_trace_struct &do_add(const char *key, const char *value,
744 size_t val_length, bool escape);
745 Opt_trace_struct &do_add(const char *key, Item *item);
746 Opt_trace_struct &do_add(const char *key, bool value);
747 Opt_trace_struct &do_add(const char *key, longlong value);
748 Opt_trace_struct &do_add(const char *key, ulonglong value);
749 Opt_trace_struct &do_add(const char *key, double value);
750 Opt_trace_struct &do_add_null(const char *key);
752 Opt_trace_struct &do_add(const char *key, const Cost_estimate &value);
754 Opt_trace_struct(const Opt_trace_struct &); ///< not defined
755 Opt_trace_struct &operator=(const Opt_trace_struct &); ///< not defined
757 bool started; ///< Whether the structure does tracing or is dummy
759 /**
760 Whether the structure requires/forbids keys for values inside it.
761 true: this is an object. false: this is an array.
763 @note The canonical way would be to not have such bool per instance, but
764 rather have a pure virtual member function
765 Opt_trace_struct::requires_key(), overloaded by Opt_trace_object
766 (returning true) and by Opt_trace_array (returning false). But
767 Opt_trace_object::requires_key() would not be accessible from
768 Opt_trace_struct::do_construct() (which would complicate coding), whereas
769 the bool is.
770 */
773 /**
774 Whether this structure caused tracing to be disabled in this statement
775 because belonging to a not-traced optimizer feature, in accordance with
776 the value of @@@@optimizer_trace_features.
777 */
779 bool empty; ///< @see set_not_empty()
780 Opt_trace_stmt *stmt; ///< Trace owning the structure
781 /// Key if the structure is the value of a key/value pair, NULL otherwise
782 const char *saved_key;
783#ifndef NDEBUG
784 /**
785 Fixed-length prefix of previous key in this structure, if this structure
786 is an object. Serves to detect when adding two same consecutive keys to
787 an object, which would be wrong.
788 */
789 char previous_key[25];
794 A JSON object (unordered set of key/value pairs).
795 Defines only a constructor, all the rest is inherited from
796 Opt_trace_struct.
799 public:
800 /**
801 Constructs an object. Key is specified, so the object is the value of a
802 key/value pair.
803 @param ctx context for this object
804 @param key key
805 @param feature optimizer feature to which this structure belongs
806 */
808 Opt_trace_context *ctx, const char *key,
810 : Opt_trace_struct(ctx, true, key, feature) {}
811 /**
812 Constructs an object. No key is specified, so the object is just a value
813 (serves for the single root object or for adding the object to an array).
814 @param ctx context for this object
815 @param feature optimizer feature to which this structure belongs
816 */
820 : Opt_trace_struct(ctx, true, nullptr, feature) {}
824 A JSON array (ordered set of values).
825 Defines only a constructor, all the rest in inherited from
826 Opt_trace_struct.
829 public:
830 /**
831 Constructs an array. Key is specified, so the array is the value of a
832 key/value pair.
833 @param ctx context for this array
834 @param key key
835 @param feature optimizer feature to which this structure belongs
836 */
838 Opt_trace_context *ctx, const char *key,
840 : Opt_trace_struct(ctx, false, key, feature) {}
841 /**
842 Constructs an array. No key is specified, so the array is just a value
843 (serves for adding the object to an array).
844 @param ctx context for this array
845 @param feature optimizer feature to which this structure belongs
846 */
850 : Opt_trace_struct(ctx, false, nullptr, feature) {}
854 Instantiate an instance of this class for specific cases where
855 optimizer trace, in a certain section of Optimizer code, should write only
856 to DBUG and not I_S. Example: see
857 Note that this class should rarely be used; the "feature" parameter of
858 Opt_trace_struct is a good alternative.
861 public:
862 /**
863 @param ctx_arg Context.
864 @param disable_arg Whether the instance should really disable
865 anything. If false, the object is dummy. If true,
866 tracing-to-I_S is disabled at construction and
867 re-enabled at destruction.
868 @details A dummy instance is there only for RAII reasons. Imagine we want
869 to do this:
871 {
872 if (x) disable tracing;
873 code;
874 } // tracing should be re-enabled here
876 As we use RAII, we cannot put the instance declaration inside if(x):
878 {
879 if (x) Opt_trace_disable_I_S instance(ctx);
880 code;
881 }
883 because it would be destroyed as soon as the if() block is left, so
884 tracing would be re-enabled before @c code;. It should rather be written
885 as:
887 {
888 Opt_trace_disable_I_S instance(ctx, x); // if !x, does nothing
889 code;
890 } // re-enabling happens here, if x is true
892 */
893 Opt_trace_disable_I_S(Opt_trace_context *ctx_arg, bool disable_arg) {
894 if (disable_arg) {
895 ctx = ctx_arg;
897 } else
898 ctx = nullptr;
899 }
901 /// Destructor. Restores trace's "enabled" property to its previous value.
903 if (ctx != nullptr) ctx->restore_I_S();
904 }
906 private:
907 /** Context. Non-NULL if and only if this instance really does disabling */
911 const Opt_trace_disable_I_S &); // not defined
915 @name Helpers connecting the optimizer trace to THD or Information Schema.
921 public:
922 /**
923 Instantiate this class to start tracing a THD's actions (generally at a
924 statement's start), and to set the "original" query (not transformed, as
925 sent by client) for the new trace. Destructor will end the trace.
927 If in a routine's instruction, there is no "query". To be helpful to the
928 user, we craft a query using the instruction's print(). We don't leave this
929 to the caller, because it would be inefficient if tracing is off.
931 @param thd_arg the THD
932 @param tbl list of tables read/written by the statement.
933 @param sql_command SQL command being prepared or executed
934 @param set_vars what variables are set by this command (only used if
935 sql_command is SQLCOM_SET_OPTION)
936 @param query query
937 @param query_length query's length
938 @param instr routine's instruction, if applicable; if so, 'query'
939 and 'query_length' above are ignored
940 @param query_charset charset which was used to encode this query
942 @note Each sub-statement is responsible for ending the trace which it
943 has started; this class achieves this by keeping some memory inside.
944 */
945 Opt_trace_start(THD *thd_arg, Table_ref *tbl,
946 enum enum_sql_command sql_command,
947 List<set_var_base> *set_vars, const char *query,
948 size_t query_length, sp_printable *instr,
949 const CHARSET_INFO *query_charset);
952 private:
954 bool error; ///< whether trace start() had an error
957class Query_block;
960 Prints SELECT query to optimizer trace. It is not the original query (as in
961 @c Opt_trace_context::set_query()) but a printout of the parse tree
962 (Item-s).
963 @param thd the THD
964 @param query_block query's parse tree
965 @param trace_object Opt_trace_object to which the query will be added
968 Opt_trace_object *trace_object);
971 If the security context is not that of the connected user, inform the trace
972 system that a privilege is missing. With one exception: see below.
974 @param thd the THD
976 This serves to eliminate the following issue.
977 Any information readable by a SELECT may theoretically end up in
978 the trace. And a SELECT may read information from other places than tables:
979 - from views (reading their bodies)
980 - from stored routines (reading their bodies)
981 - from files (reading their content), with LOAD_FILE()
982 - from the list of connections (reading their queries...), with
984 If the connected user has EXECUTE privilege on a routine which does a
985 security context change, the routine can retrieve information internally
986 (if allowed by the SUID context's privileges), and present only a portion
987 of it to the connected user. But with tracing on, all information is
988 possibly in the trace. So the connected user receives more information than
989 the routine's definer intended to provide. Fixing this issue would require
990 adding, near many privilege checks in the server, a new
991 optimizer-trace-specific check done against the connected user's context,
992 to verify that the connected user has the right to see the retrieved
993 information.
995 Instead, our chosen simpler solution is that if we see a security context
996 change where SUID user is not the connected user, we disable tracing. With
997 only one safe exception: if the connected user has all global privileges
998 (because then she/he can find any information anyway). By "all global
999 privileges" we mean everything but WITH GRANT OPTION (that latter one isn't
1000 related to information gathering).
1002 Read access to I_S.OPTIMIZER_TRACE by another user than the connected user
1003 is restricted: @see fill_optimizer_trace_info().
1008 If tracing is on, checks additional privileges for a view, to make sure
1009 that the user has the right to do SHOW CREATE VIEW. For that:
1010 - this function checks SHOW VIEW
1011 - SELECT is tested in opt_trace_disable_if_no_tables_access()
1012 - SELECT + SHOW VIEW is sufficient for SHOW CREATE VIEW.
1013 We also check underlying tables.
1014 If a privilege is missing, notifies the trace system.
1015 This function should be called when the view's underlying tables have not
1016 yet been merged.
1018 @param thd THD context
1019 @param view view to check
1020 @param underlying_tables underlying tables/views of 'view'
1021 */
1023 Table_ref *underlying_tables);
1026 If tracing is on, checks additional privileges on a stored routine, to make
1027 sure that the user has the right to do SHOW CREATE PROCEDURE/FUNCTION. For
1028 that, we use the same checks as in those SHOW commands.
1029 If a privilege is missing, notifies the trace system.
1031 This function is not redundant with
1032 opt_trace_disable_if_no_security_context_access().
1033 Indeed, for a SQL SECURITY INVOKER routine, there is no context change, but
1034 we must still verify that the invoker can do SHOW CREATE.
1036 For triggers, see note in sp_head::execute_trigger().
1038 @param thd The THD
1039 @param sp routine to check
1040 */
1044 Fills information_schema.OPTIMIZER_TRACE with rows (one per trace)
1045 @retval 0 ok
1046 @retval 1 error
1048int fill_optimizer_trace_info(THD *thd, Table_ref *tables, Item *);
1053 Helper for defining query-transformation-related trace objects in one
1054 code line. This produces
1055 {
1056 "transformation": {
1057 "select#": @<select_number@>,
1058 "from": @<from@>,
1059 "to": @<to@>
1060 The objects are left open, so that one can add more to them (often a
1061 "chosen" property after making some computation). Objects get closed when
1062 going out of scope as usual.
1063 @param trace optimizer trace
1064 @param object_level0 name of the outer Opt_trace_object C++ object
1065 @param object_level1 name of the inner Opt_trace_object C++ object
1066 @param select_number number of the being-transformed Query_block
1067 @param from description of the before-transformation state
1068 @param to description of the after-transformation state
1070#define OPT_TRACE_TRANSFORM(trace, object_level0, object_level1, \
1071 select_number, from, to) \
1072 Opt_trace_object object_level0(trace); \
1073 Opt_trace_object object_level1(trace, "transformation"); \
1074 object_level1.add_select_number(select_number); \
1075 object_level1.add_alnum("from", from).add_alnum("to", to);
1077#endif /* OPT_TRACE_INCLUDED */
Used to store optimizer cost estimates.
Definition: handler.h:3657
Base class that is used to represent any kind of expression in a relational query.
Definition: item.h:850
Definition: sql_list.h:433
A JSON array (ordered set of values).
Definition: opt_trace.h:828
Opt_trace_array(Opt_trace_context *ctx, const char *key, Opt_trace_context::feature_value feature=Opt_trace_context::MISC)
Constructs an array.
Definition: opt_trace.h:837
Opt_trace_array(Opt_trace_context *ctx, Opt_trace_context::feature_value feature=Opt_trace_context::MISC)
Constructs an array.
Definition: opt_trace.h:847
A per-session context which is always available at any point of execution, because in practice it's a...
Definition: opt_trace_context.h:89
void disable_I_S_for_this_and_children()
Temporarily disables I_S for this trace and its children.
Definition: opt_trace_context.h:256
Features' numeric values for @@optimizer_trace_features variable.
Definition: opt_trace_context.h:200
Anything unclassified, including the top object (thus, by "inheritance from parent",...
Definition: opt_trace_context.h:215
bool is_started() const
Returns whether there is a current trace.
Definition: opt_trace_context.h:143
void restore_I_S()
Restores I_S support to what it was before the previous call to disable_I_S_for_this_and_children().
Definition: opt_trace_context.h:265
Instantiate an instance of this class for specific cases where optimizer trace, in a certain section ...
Definition: opt_trace.h:860
Opt_trace_disable_I_S(const Opt_trace_disable_I_S &)
Destructor. Restores trace's "enabled" property to its previous value.
Definition: opt_trace.h:902
Opt_trace_disable_I_S(Opt_trace_context *ctx_arg, bool disable_arg)
Definition: opt_trace.h:893
Opt_trace_disable_I_S & operator=(const Opt_trace_disable_I_S &)
Opt_trace_context * ctx
Definition: opt_trace.h:908
Iterator over the list of remembered traces.
Definition: opt_trace.h:406
Opt_trace_iterator(Opt_trace_context *ctx)
bool at_end() const
Definition: opt_trace.h:427
void get_value(Opt_trace_info *info) const
Provides information about the trace on which the iterator is positioned.
const Opt_trace_stmt * cursor
trace which the iterator is positioned on
Definition: opt_trace.h:432
Opt_trace_context * ctx
Pointer to context, from which traces are retrieved.
Definition: opt_trace.h:431
void next()
Advances iterator to next trace.
long row_count
how many traces retrieved so far
Definition: opt_trace.h:433
A JSON object (unordered set of key/value pairs).
Definition: opt_trace.h:798
Opt_trace_object(Opt_trace_context *ctx, Opt_trace_context::feature_value feature=Opt_trace_context::MISC)
Constructs an object.
Definition: opt_trace.h:817
Opt_trace_object(Opt_trace_context *ctx, const char *key, Opt_trace_context::feature_value feature=Opt_trace_context::MISC)
Constructs an object.
Definition: opt_trace.h:807
Definition: opt_trace.h:920
bool error
whether trace start() had an error
Definition: opt_trace.h:954
Opt_trace_context *const ctx
Definition: opt_trace.h:953
Opt_trace_start(THD *thd_arg, Table_ref *tbl, enum enum_sql_command sql_command, List< set_var_base > *set_vars, const char *query, size_t query_length, sp_printable *instr, const CHARSET_INFO *query_charset)
Instantiate this class to start tracing a THD's actions (generally at a statement's start),...
The trace of one statement.
Object and array are both "structured data" and have lots in common, so the Opt_trace_struct is a bas...
Definition: opt_trace.h:445
Opt_trace_struct & add_null(const char *key)
Adds a JSON null object (==Python's "None")
Definition: opt_trace.h:641
Opt_trace_struct & add_select_number(uint select_number)
Helper to put the number of query_block in an object.
Definition: opt_trace.h:657
Opt_trace_struct & do_add_utf8_table(const Table_ref *tab)
char previous_key[25]
Fixed-length prefix of previous key in this structure, if this structure is an object.
Definition: opt_trace.h:789
Opt_trace_struct & add(ulonglong value)
Definition: opt_trace.h:628
const char * saved_key
Key if the structure is the value of a key/value pair, NULL otherwise.
Definition: opt_trace.h:782
Definition: opt_trace.h:473
void do_construct(Opt_trace_context *ctx, bool requires_key, const char *key, Opt_trace_context::feature_value feature)
Full initialization.
Opt_trace_struct & add(Item *item)
Definition: opt_trace.h:578
void end()
The exception to RAII: this function is an explicit way of ending a structure before it goes out of s...
Definition: opt_trace.h:483
Opt_trace_struct & add(ulong value)
Definition: opt_trace.h:612
void do_destruct()
Really does destruction.
Opt_trace_struct & add(int value)
Definition: opt_trace.h:596
bool started
Whether the structure does tracing or is dummy.
Definition: opt_trace.h:757
Opt_trace_struct & add(const char *key, longlong value)
Definition: opt_trace.h:616
Opt_trace_stmt * stmt
Trace owning the structure.
Definition: opt_trace.h:780
bool requires_key
Whether the structure requires/forbids keys for values inside it.
Definition: opt_trace.h:771
Opt_trace_struct & add(longlong value)
Definition: opt_trace.h:620
bool set_not_empty()
Informs this structure that we are adding data (scalars, structures) to it.
Definition: opt_trace.h:678
Opt_trace_struct & do_add(const char *key, const char *value, size_t val_length, bool escape)
Really adds to the object.
Opt_trace_struct & add(double value)
Definition: opt_trace.h:636
Opt_trace_struct & add_alnum(const char *key, const char *value)
Adds a value (of string type) to the structure.
Definition: opt_trace.h:518
Opt_trace_struct & add(const char *key, uint value)
Definition: opt_trace.h:600
Opt_trace_struct & add(const char *key, double value)
Definition: opt_trace.h:632
Opt_trace_struct & operator=(const Opt_trace_struct &)
not defined
const char * check_key(const char *key)
Validates the key about to be added.
Opt_trace_struct & add_utf8(const char *value)
Variant of add_utf8() where 'value' is 0-terminated.
Definition: opt_trace.h:560
Opt_trace_struct & add(const char *key, ulong value)
Definition: opt_trace.h:608
Opt_trace_struct & add_utf8(const char *key, const char *value, size_t val_length)
Like add_alnum() but supports any UTF8 characters in 'value'.
Definition: opt_trace.h:541
Opt_trace_struct(Opt_trace_context *ctx_arg, bool requires_key_arg, const char *key, Opt_trace_context::feature_value feature)
Definition: opt_trace.h:460
bool has_disabled_I_S
Whether this structure caused tracing to be disabled in this statement because belonging to a not-tra...
Definition: opt_trace.h:778
Opt_trace_struct & add(bool value)
Definition: opt_trace.h:588
Opt_trace_struct & add(const char *key, Item *item)
Add a value (of Item type) to the structure.
Definition: opt_trace.h:574
Opt_trace_struct & add_utf8(const char *key, const char *value)
Variant of add_utf8() where 'value' is 0-terminated.
Definition: opt_trace.h:554
Opt_trace_struct & add(uint value)
Definition: opt_trace.h:604
Opt_trace_struct & add_utf8(const char *value, size_t val_length)
Variant of add_utf8() for adding to an array (no key)
Definition: opt_trace.h:548
Opt_trace_struct(const Opt_trace_struct &)
not defined
Opt_trace_struct & add(const char *key, const Cost_estimate &cost)
Add a value to the structure.
Definition: opt_trace.h:666
Opt_trace_struct & do_add_null(const char *key)
Opt_trace_struct & add(const char *key, const char *value)
Not implemented, use add_alnum() instead.
bool empty
Definition: opt_trace.h:779
Opt_trace_struct & add(const char *key, bool value)
Definition: opt_trace.h:584
Opt_trace_struct & add(const char *key)
Opt_trace_struct & add_alnum(const char *value)
Adds a value (of string type) to the structure.
Definition: opt_trace.h:529
Opt_trace_struct & add(const char *key, ulonglong value)
Definition: opt_trace.h:624
Opt_trace_struct & add_utf8_table(const Table_ref *tab)
Helper to put the database/table name in an object.
Definition: opt_trace.h:649
Opt_trace_struct & add(const char *key, int value)
Definition: opt_trace.h:592
This class represents a query block, aka a query specification, which is a query consisting of a SELE...
Definition: sql_lex.h:1155
Query_block * query_block() const override
The query_block which holds the ORDER BY and LIMIT information for this set operation.
Definition: sql_lex.h:1171
For each client connection we create a separate thread with THD serving as a thread/connection descri...
Definition: sql_lexer_thd.h:33
Definition: table.h:2761
A base class for everything that can be set with SET command.
Definition: set_var.h:942
sp_head represents one instance of a stored program.
Definition: sp_head.h:379
sp_printable defines an interface which should be implemented if a class wants report some internal i...
Definition: sp_instr.h:66
Fido Client Authentication nullptr
Header for compiler-dependent features.
constexpr bool likely(bool expr)
Definition: my_compiler.h:54
constexpr bool unlikely(bool expr)
Definition: my_compiler.h:55
Some integer typedefs for easier portability.
unsigned long long int ulonglong
Definition: my_inttypes.h:55
long long int longlong
Definition: my_inttypes.h:54
Definition: my_sqlcommand.h:45
static char * query
Log info(cout, "NOTE")
static std::string escape(const std::string &str)
Escapes (only) apostrophes.
void opt_trace_disable_if_no_stored_proc_func_access(THD *thd, sp_head *sp)
If tracing is on, checks additional privileges on a stored routine, to make sure that the user has th...
void opt_trace_disable_if_no_view_access(THD *thd, Table_ref *view, Table_ref *underlying_tables)
If tracing is on, checks additional privileges for a view, to make sure that the user has the right t...
void opt_trace_print_expanded_query(const THD *thd, Query_block *query_block, Opt_trace_object *trace_object)
Prints SELECT query to optimizer trace.
int fill_optimizer_trace_info(THD *thd, Table_ref *tables, Item *)
Fills information_schema.OPTIMIZER_TRACE with rows (one per trace)
void opt_trace_disable_if_no_security_context_access(THD *thd)
If the security context is not that of the connected user, inform the trace system that a privilege i...
This contains the declaration of class Opt_trace_context, which is needed to declare THD.
required string key
Definition: replication_asynchronous_connection_failover.proto:59
Definition: m_ctype.h:382
User-visible information about a trace.
Definition: opt_trace.h:376
size_t missing_bytes
How many bytes this trace is missing (for traces which were truncated because of @@optimizer-trace-ma...
Definition: opt_trace.h:397
size_t trace_length
length of trace string / String containing original query. 0-termination: like trace_ptr.
Definition: opt_trace.h:388
const CHARSET_INFO * query_charset
charset of query string
Definition: opt_trace.h:392
const char * query_ptr
Definition: opt_trace.h:390
const char * trace_ptr
String containing trace.
Definition: opt_trace.h:387
bool missing_priv
whether user lacks privilege to see this trace
Definition: opt_trace.h:398
size_t query_length
length of query string
Definition: opt_trace.h:391
unsigned int uint