MySQL 8.2.0
Source Code Documentation
Go to the documentation of this file.
1/* Copyright (c) 2011, 2023, Oracle and/or its affiliates.
3 This program is free software; you can redistribute it and/or modify
4 it under the terms of the GNU General Public License, version 2.0,
5 as published by the Free Software Foundation.
7 This program is also distributed with certain software (including
8 but not limited to OpenSSL) that is licensed under separate terms,
9 as designated in a particular file or component or in included license
10 documentation. The authors of MySQL hereby grant you an additional
11 permission to link the program and your derivative works with the
12 separately licensed software that they have included with MySQL.
14 This program is distributed in the hope that it will be useful,
15 but WITHOUT ANY WARRANTY; without even the implied warranty of
17 GNU General Public License, version 2.0, for more details.
19 You should have received a copy of the GNU General Public License
20 along with this program; if not, write to the Free Software
21 Foundation, Inc., 51 Franklin St, Fifth Floor, Boston, MA 02110-1301 USA */
26#include <limits.h>
27#include <string.h>
28#include <sys/types.h>
30#include "my_compiler.h"
31#include "my_inttypes.h"
32#include "my_sqlcommand.h" // enum_sql_command
33#include "sql/opt_trace_context.h" // Opt_trace_context
35class Cost_estimate;
36class Item;
37class THD;
38class set_var_base;
39class sp_head;
40class sp_printable;
41struct CHARSET_INFO;
42class Table_ref;
43template <class T>
44class List;
47 @file sql/opt_trace.h
48 API for the Optimizer trace (WL#5257)
52 @page PAGE_OPT_TRACE The Optimizer Trace
54 @section INTRODUCTION Introduction
56 This optimizer trace is aimed at producing output, which is readable by
57 humans and by programs, to aid understanding of decisions and actions taken
58 by the MySQL Optimizer.
60 @section OUTPUT_FORMAT Output format
62 The chosen output format is JSON (JavaScript Object Notation).
63 In JSON there are:
64 @li "objects" (unordered set of key-value pairs); equivalent to Python's
65 dictionary or Perl's associative array or hash or STL's hash_map.
66 @li "arrays" (ordered set of values); equivalent to Python's and Perl's list
67 or STL's vector.
68 @li "values": a value can be a string, number, boolean, null,
69 which we all call "scalars", or be an object, array.
71 For example (explanations after "<<" are not part of output):
73 { << start of top object
74 "first_name": "Gustave", << key/value pair (value is string)
75 "last_name": "Eiffel", << key/value pair (value is string)
76 "born": 1832, << key/value pair (value is integer)
77 "contributions_to": [ << key/value pair (value is array)
78 { << 1st item of array is an object (a building)
79 "name": "Eiffel tower",
80 "location": Paris
81 }, << end of 1st item of array
82 {
83 "name": "Liberty statue",
84 "location": "New York"
85 } << end of 2nd item of array
86 ] << end of array
87 } << end of top object
89 For more details, have a look at the syntax at
90 Note that indentation and newlines are superfluous, useful only for
91 human-readability.
92 Note also that there is nothing like a "named object": an object, array or
93 value has no name; but if it is the value of a key/value pair in an
94 enclosing, outer object, then the key can be seen as the inner object's
95 "name".
97 @section USER_ENABLE_TRACING How a user enables/views the trace
100 SET SESSION OPTIMIZER_TRACE="enabled=on"; # enable tracing
101 <statement to trace>; # like SELECT, EXPLAIN SELECT, UPDATE, DELETE...
102 SELECT * FROM information_schema.OPTIMIZER_TRACE;
103 [ repeat last two steps at will ]
104 SET SESSION OPTIMIZER_TRACE="enabled=off"; # disable tracing
107 @c SELECT and @c EXPLAIN SELECT produce the same trace. But there are
108 exceptions regarding subqueries because the two commands treat subqueries
109 differently, for example in
111 SELECT ... WHERE x IN (subq1) AND y IN (subq2)
113 SELECT terminates after executing the first subquery if the related IN
114 predicate is false, so we won't see @c JOIN::optimize() tracing for subq2;
115 whereas EXPLAIN SELECT analyzes all subqueries (see loop at the end of
116 @c select_describe()).
118 @section USER_SELECT_TRACING_STATEMENTS How a user traces only certain
121 When tracing is in force, each SQL statement generates a trace; more
122 exactly, so does any of
126 UPDATE/DELETE and their multi-table variants,
127 SET (unless it manipulates @@@@optimizer_trace),
128 DO,
129 DECLARE/CASE/IF/RETURN (stored routines language elements),
130 CALL.
131 If a command above is prepared and executed in separate steps, preparation
132 and execution are separately traced.
133 By default each new trace overwrites the previous trace. Thus, if a
134 statement contains sub-statements (example: invokes stored procedures,
135 stored functions, triggers), the top statement and sub-statements each
136 generate traces, but at the execution's end only the last sub-statement's
137 trace is visible.
138 If the user wants to see the trace of another sub-statement, she/he can
139 enable/disable tracing around the desired sub-statement, but this requires
140 editing the routine's code, which may not be possible. Another solution is
141 to use
143 SET optimizer_trace_offset=<OFFSET>, optimizer_trace_limit=<LIMIT>
145 where OFFSET is a signed integer, and LIMIT is a positive integer.
146 The effect of this SET is the following:
148 @li all remembered traces are cleared
150 @li a later SELECT on OPTIMIZER_TRACE returns the first LIMIT traces of
151 the OFFSET oldest remembered traces (if OFFSET >= 0), or the first LIMIT
152 traces of the -OFFSET newest remembered traces (if OFFSET < 0).
154 For example,
155 a combination of OFFSET=-1 and LIMIT=1 will make the last trace be shown (as
156 is default), OFFSET=-2 and LIMIT=1 will make the next-to-last be shown,
157 OFFSET=-5 and LIMIT=5 will make the last five traces be shown. Such negative
158 OFFSET can be useful when one knows that the interesting sub-statements are
159 the few last ones of a stored routine, like this:
161 SET optimizer_trace_offset=-5, optimizer_trace_limit=5;
162 CALL stored_routine(); # more than 5 sub-statements in this routine
163 SELECT * FROM information_schema.OPTIMIZER_TRACE; # see only last 5 traces
165 On the opposite, a positive OFFSET can be useful when one knows that the
166 interesting sub-statements are the few first ones of a stored routine.
168 The more those two variables are accurately adjusted, the less memory is
169 used. For example, OFFSET=0 and LIMIT=5 will use memory to remember 5
170 traces, so if only the three first are needed, OFFSET=0 and LIMIT=3 is
171 better (tracing stops after the LIMITth trace, so the 4th and 5th trace are
172 not created and don't take up memory). A stored routine may have a loop
173 which executes many sub-statements and thus generates many traces, which
174 would use a lot of memory; proper OFFSET and LIMIT can restrict tracing to
175 one iteration of the loop for example. This also gains speed, as tracing a
176 sub-statement impacts performance.
178 If OFFSET>=0, only LIMIT traces are kept in memory. If OFFSET<0, that is not
179 true: instead, (-OFFSET) traces are kept in memory; indeed even if LIMIT is
180 smaller than (-OFFSET), so excludes the last statement, the last statement
181 must still be traced because it will be inside LIMIT after executing one
182 more statement (remember than OFFSET<0 is counted from the end: the "window"
183 slides as more statements execute).
185 Such memory and speed gains are the reason why optimizer_trace_offset/limit,
186 which are restrictions at the trace producer level, are offered. They are
187 better than using
191 which is a restriction on the trace consumer level, which saves almost
192 nothing.
194 @section USER_SELECT_TRACING_FEATURES How a user traces only certain
195 optimizer features
198 SET OPTIMIZER_TRACE_FEATURES="feature1=on|off,...";
200 where "feature1" is one optimizer feature. For example "greedy_search": a
201 certain Opt_trace_array at the start of @c
202 Optimize_table_order::choose_table_order() has a flag "GREEDY_SEARCH" passed
203 to its constructor: this means that if the user has turned tracing of greedy
204 search off, this array will not be written to the I_S trace, neither will
205 any children structures. All this disabled "trace chunk" will be replaced by
206 an ellipsis "...".
208 @section DEV_ADDS_TRACING How a developer adds tracing to a function
210 Check @c Opt_trace* usage in @c advance_sj_state():
213 Opt_trace_array trace_choices(trace, "semijoin_strategy_choice");
216 This creates an array for key "semijoin_strategy_choice". We are going to
217 list possible semijoin strategy choices.
220 Opt_trace_object trace_one_strategy(trace);
223 This creates an object without key (normal, it's in an array). This
224 object will describe one single strategy choice.
227 trace_one_strategy.add_alnum("strategy", "FirstMatch");
230 This adds a key/value pair to the just-created object: key is "strategy",
231 value is "FirstMatch". This is the strategy to be described in the
232 just-created object.
235 trace_one_strategy.add("cost", *current_read_time).
236 add("records", *current_record_count);
237 trace_one_strategy.add("chosen", (pos->sj_strategy == SJ_OPT_FIRST_MATCH));
240 This adds 3 key/value pairs: cost of strategy, number of records produced
241 by this strategy, and whether this strategy is chosen.
243 After that, there is similar code for other semijoin strategies.
245 The resulting trace piece (seen in @c information_schema.OPTIMIZER_TRACE) is
247 "semijoin_strategy_choice": [
248 {
249 "strategy": "FirstMatch",
250 "cost": 1,
251 "records": 1,
252 "chosen": true
253 },
254 {
255 "strategy": "DuplicatesWeedout",
256 "cost": 1.1,
257 "records": 1,
258 "duplicate_tables_left": false,
259 "chosen": false
260 }
261 ]
264 For more output examples, check result files of the opt_trace suite in
265 @c mysql-test.
267 Feature can be un-compiled with @code cmake -DOPTIMIZER_TRACE=0 @endcode.
269 @section WITH_DBUG Interaction between trace and DBUG
271 We don't want to have to duplicate code like this:
273 DBUG_PRINT("info",("cost %g",cost));
274 Opt_trace_object(thd->opt_trace).add("cost",cost);
277 Thus, any optimizer trace operation, *even* if tracing is run-time disabled,
278 has an implicit DBUG_PRINT("opt",...) inside. This way, only the
279 second line above is needed, and several DBUG_PRINT() could be removed from
280 the Optimizer code.
281 When tracing is run-time disabled, in a debug binary, traces are still
282 created in order to catch the @c add() calls and write their text to DBUG,
283 but those traces are not visible into INFORMATION_SCHEMA.OPTIMIZER_TRACE: we
284 then say that they "don't support I_S".
285 A debug binary without optimizer trace compiled in, will intentionally not
286 compile.
288 Because opening an object or array, or add()-ing to it, writes to DBUG
289 immediately, a key/value pair and its outer object may be 100 lines
290 apart in the DBUG log.
292 @section ADDING_TRACING Guidelines for adding tracing
294 @li Try to limit the number of distinct "words". For example, when
295 describing an optimizer's decision, the words "chosen" (true/false value,
296 tells whether we are choosing the said optimization), "cause" (free text
297 value, tells why we are making this choice, when it's not obvious)
298 can and should often be used. Having a restricted vocabulary helps
299 consistency. Use "row" instead of "record". Use "tmp" instead of
300 "temporary".
302 @li Use only simple characters for key names: a-ZA-Z_#, and no space. '#'
303 serves to denote a number, like in "select#" .
305 @li Keep in mind than in an object, keys are not ordered; an application may
306 parse the JSON output and output it again with keys order changed; thus
307 when order matters, use an array (which may imply having anonymous objects
308 as items of the array, with keys inside the anonymous objects, see how it's
309 done in @c JOIN::optimize()). Keep in mind that in an object keys should
310 be unique, an application may lose duplicate keys.
312 @section OOM_HANDLING Handling of "out-of-memory" errors
314 All memory allocations (with exceptions: see below) in the Optimizer trace
315 use @c my_error() to report errors, which itself calls @c
316 error_handler_hook. It is the responsibility of the API user to set up a
317 proper @c error_handler_hook which will alert her/him of the OOM
318 problem. When in the server, this is already the case (@c error_handler_hook
319 is @c my_message_sql() which makes the statement fail).
320 Note that the debug binary may crash if OOM (OOM can cause syntax
321 errors...).
323 @section TRACE_SECURITY Description of trace-induced security checks.
325 A trace exposes information. For example if one does SELECT on a view, the
326 trace contains the view's body. So, the user should be allowed to see the
327 trace only if she/he has privilege to see the body, i.e. privilege to do
329 There are similar issues with stored procedures, functions, triggers.
331 We implement this by doing additional checks on SQL objects when tracing is
332 on:
333 @li stored procedures, functions, triggers: checks are done when executing
334 those objects
335 @li base tables and views.
337 Base tables or views are listed in some @c LEX::query_tables.
338 The LEX may be of the executing statement (statement executed by
339 @c mysql_execute_command(), or by
340 @c sp_lex_keeper::reset_lex_and_exec_core()), we check this LEX in the
341 constructor of Opt_trace_start.
342 Or it may be a LEX describing a view, we check this LEX when
343 opening the view (@c open_and_read_view()).
345 Those checks are greatly simplified by disabling traces in case of security
346 context changes. @see opt_trace_disable_if_no_security_context_access().
348 Those checks must be done with the security context of the connected
349 user. Checks with the SUID context would be useless: assume the design is
350 that the basic user does not have DML privileges on tables, but only
351 EXECUTE on SUID-highly-privileged routines (which implement _controlled_
352 _approved_ DMLs): then the SUID context would successfully pass all
353 additional privilege checks, routine would generate tracing, and the
354 connected user would view the trace after the routine's execution, seeing
355 secret information.
357 @section NEXT What a developer should read next
359 The documentation of those classes, in order
361 Opt_trace_context
362 Opt_trace_context_impl
363 Opt_trace_stmt
364 Opt_trace_struct
365 Opt_trace_object
366 Opt_trace_array
368 and then @ref opt_trace.h as a whole.
371class Opt_trace_stmt; // implementation detail local to
374 User-visible information about a trace. @sa Opt_trace_iterator.
377 /**
378 String containing trace.
379 If trace has been end()ed, this is 0-terminated, which is only to aid
380 debugging or unit testing; this property is not relied upon in normal
381 server usage.
382 If trace has not been ended, this is not 0-terminated. That rare case can
383 happen when a substatement reads OPTIMIZER_TRACE (at that stage, the top
384 statement is still executing so its trace is not ended yet, but may still
385 be read by the sub-statement).
386 */
387 const char *trace_ptr;
388 size_t trace_length; ///< length of trace string
389 //// String containing original query. 0-termination: like trace_ptr.
390 const char *query_ptr;
391 size_t query_length; ///< length of query string
392 const CHARSET_INFO *query_charset; ///< charset of query string
393 /**
394 How many bytes this trace is missing (for traces which were truncated
395 because of @@@@optimizer-trace-max-mem-size).
396 */
398 bool missing_priv; ///< whether user lacks privilege to see this trace
402 Iterator over the list of remembered traces.
403 @note due to implementation, the list must not change during an
404 iterator's lifetime, or results may be unexpected (no crash though).
407 public:
408 /**
409 @param ctx context
410 */
413 void next(); ///< Advances iterator to next trace.
415 /**
416 Provides information about the trace on which the iterator is
417 positioned.
418 @param[out] info information returned.
419 The usage pattern is
420 1) instantiate the iterator
421 2) test at_end(), if false: call get_value() and then next()
422 3) repeat (2) until at_end() is true.
423 */
424 void get_value(Opt_trace_info *info) const;
426 /// @returns whether iterator is positioned to the end.
427 bool at_end() const { return cursor == nullptr; }
429 private:
430 /// Pointer to context, from which traces are retrieved
432 const Opt_trace_stmt *cursor; ///< trace which the iterator is positioned on
433 long row_count; ///< how many traces retrieved so far
437 Object and array are both "structured data" and have lots in common, so the
438 Opt_trace_struct is a base class for them.
439 When you want to add a structure to the trace, you create an instance of
440 Opt_trace_object or Opt_trace_array, then you add information to it with
441 add(), then the destructor closes the structure (we use RAII, Resource
442 Acquisition Is Initialization).
446 protected:
447 /**
448 @param ctx_arg Optimizer trace context for this structure
449 @param requires_key_arg whether this structure requires/forbids keys
450 for values put inside it (an object requires them, an
451 array forbids them)
452 @param key key if this structure is the value of a key/value pair,
453 NULL otherwise. This pointer must remain constant and
454 valid until the object is destroyed (to support
455 @ref saved_key).
456 @param feature optimizer feature to which this structure belongs
458 This constructor is never called directly, only from subclasses.
459 */
460 Opt_trace_struct(Opt_trace_context *ctx_arg, bool requires_key_arg,
461 const char *key, Opt_trace_context::feature_value feature)
462 : started(false) {
463 // A first inlined test
464 if (unlikely(ctx_arg->is_started())) {
465 // Tracing enabled: must fully initialize the structure.
466 do_construct(ctx_arg, requires_key_arg, key, feature);
467 }
468 /*
469 Otherwise, just leave "started" to false, it marks that the structure is
470 dummy.
471 */
472 }
475 }
477 public:
478 /**
479 The exception to RAII: this function is an explicit way of ending a
480 structure before it goes out of scope. Don't use it unless RAII mandates
481 a new scope which mandates re-indenting lots of code lines.
482 */
483 void end() {
485 }
487 /**
488 Adds a value (of string type) to the structure. A key is specified, so it
489 adds the key/value pair (the structure must thus be an object).
491 There are two "add_*" variants to add a string value.
492 If the value is 0-terminated and each character
493 - is ASCII 7-bit
494 - has ASCII code >=32 and is neither '"' nor '\\'
495 then add_alnum() should be used. That should be the case for all fixed
496 strings like add_alnum("cause", "cost").
497 Otherwise, add_utf8() should be used; it accepts any UTF8-encoded
498 character in 'value' and will escape characters which JSON requires (and
499 is thus slower than add_alnum()). It should be used for all strings which
500 we get from the server's objects (indeed a table's name, a WHERE
501 condition, may contain "strange" characters).
503 @param key key
504 @param value value
505 @returns a reference to the structure, useful for chaining like this:
506 @verbatim add(x,y).add(z,t).add(u,v) @endverbatim
508 String-related add() variants are named add_[something]():
509 - to avoid confusing the compiler between:
510 add(const char *value, size_t val_length) and
511 add(const char *key, ulonglong value)
512 - and because String::length() returns uint32 and not size_t, so for
513 add(str.ptr(), str.length())
514 compiler may pick
515 add(const char *key, double value) instead of
516 add(const char *value, size_t val_length).
517 */
518 Opt_trace_struct &add_alnum(const char *key, const char *value) {
519 if (likely(!started)) return *this;
520 return do_add(key, value, strlen(value), false);
521 }
523 /**
524 Adds a value (of string type) to the structure. No key is specified, so
525 it adds only the value (the structure must thus be an array).
526 @param value value
527 @returns a reference to the structure
528 */
529 Opt_trace_struct &add_alnum(const char *value) {
530 if (likely(!started)) return *this;
531 return do_add(nullptr, value, strlen(value), false);
532 }
534 /**
535 Like add_alnum() but supports any UTF8 characters in 'value'.
536 Will "escape" 'value' to be JSON-compliant.
537 @param key key
538 @param value value
539 @param val_length length of string 'value'
540 */
541 Opt_trace_struct &add_utf8(const char *key, const char *value,
542 size_t val_length) {
543 if (likely(!started)) return *this;
544 return do_add(key, value, val_length, true);
545 }
547 /// Variant of add_utf8() for adding to an array (no key)
548 Opt_trace_struct &add_utf8(const char *value, size_t val_length) {
549 if (likely(!started)) return *this;
550 return do_add(nullptr, value, val_length, true);
551 }
553 /// Variant of add_utf8() where 'value' is 0-terminated
554 Opt_trace_struct &add_utf8(const char *key, const char *value) {
555 if (likely(!started)) return *this;
556 return do_add(key, value, strlen(value), true);
557 }
559 /// Variant of add_utf8() where 'value' is 0-terminated
560 Opt_trace_struct &add_utf8(const char *value) {
561 if (likely(!started)) return *this;
562 return do_add(nullptr, value, strlen(value), true);
563 }
565 /**
566 Add a value (of Item type) to the structure. The Item should be a
567 condition (like a WHERE clause) which will be pretty-printed into the
568 trace. This is useful for showing condition transformations (equality
569 propagation etc).
570 @param key key
571 @param item the Item
572 @return a reference to the structure
573 */
574 Opt_trace_struct &add(const char *key, const Item *item) {
575 if (likely(!started)) return *this;
576 return do_add(key, item);
577 }
578 Opt_trace_struct &add(const Item *item) {
579 if (likely(!started)) return *this;
580 return do_add(nullptr, item);
581 }
582 Opt_trace_struct &add(const char *key, bool value) {
583 if (likely(!started)) return *this;
584 return do_add(key, value);
585 }
586 Opt_trace_struct &add(bool value) {
587 if (likely(!started)) return *this;
588 return do_add(nullptr, value);
589 }
590 Opt_trace_struct &add(const char *key, int value) {
591 if (likely(!started)) return *this;
592 return do_add(key, static_cast<longlong>(value));
593 }
594 Opt_trace_struct &add(int value) {
595 if (likely(!started)) return *this;
596 return do_add(nullptr, static_cast<longlong>(value));
597 }
598 Opt_trace_struct &add(const char *key, uint value) {
599 if (likely(!started)) return *this;
600 return do_add(key, static_cast<ulonglong>(value));
601 }
602 Opt_trace_struct &add(uint value) {
603 if (likely(!started)) return *this;
604 return do_add(nullptr, static_cast<ulonglong>(value));
605 }
606 Opt_trace_struct &add(const char *key, ulong value) {
607 if (likely(!started)) return *this;
608 return do_add(key, static_cast<ulonglong>(value));
609 }
610 Opt_trace_struct &add(ulong value) {
611 if (likely(!started)) return *this;
612 return do_add(nullptr, static_cast<ulonglong>(value));
613 }
614 Opt_trace_struct &add(const char *key, longlong value) {
615 if (likely(!started)) return *this;
616 return do_add(key, value);
617 }
619 if (likely(!started)) return *this;
620 return do_add(nullptr, value);
621 }
622 Opt_trace_struct &add(const char *key, ulonglong value) {
623 if (likely(!started)) return *this;
624 return do_add(key, value);
625 }
627 if (likely(!started)) return *this;
628 return do_add(nullptr, value);
629 }
630 Opt_trace_struct &add(const char *key, double value) {
631 if (likely(!started)) return *this;
632 return do_add(key, value);
633 }
634 Opt_trace_struct &add(double value) {
635 if (likely(!started)) return *this;
636 return do_add(nullptr, value);
637 }
638 /// Adds a JSON null object (==Python's "None")
640 if (likely(!started)) return *this;
641 return do_add_null(key);
642 }
643 /**
644 Helper to put the database/table name in an object.
645 @param tab TABLE* pointer
646 */
648 if (likely(!started)) return *this;
649 return do_add_utf8_table(tab);
650 }
651 /**
652 Helper to put the number of query_block in an object.
653 @param select_number number of query_block
654 */
655 Opt_trace_struct &add_select_number(uint select_number) {
656 return add("select#", select_number);
657 }
658 /**
659 Add a value to the structure.
660 @param key key
661 @param cost the value of Cost_estimate
662 @return a reference to the structure
663 */
664 Opt_trace_struct &add(const char *key, const Cost_estimate &cost) {
665 if (likely(!started)) return *this;
666 return do_add(key, cost);
667 }
669 /**
670 Informs this structure that we are adding data (scalars, structures) to
671 it.
672 This is used only if sending to I_S.
673 @returns whether the structure was empty so far.
674 @note this is reserved for use by Opt_trace_stmt.
675 */
677 const bool old_empty = empty;
678 empty = false;
679 return old_empty;
680 }
681 /**
682 Validates the key about to be added.
683 @note this is reserved for use by Opt_trace_stmt.
685 When adding a value (or array or object) to an array, or a key/value pair
686 to an object, we need to know this outer array or object.
688 It would be possible, when trying to add a key to an array, which is wrong
689 in JSON, or similarly when trying to add a value without any key to an
690 object, to catch it at compilation time, if the adder received, as
691 function parameter, the type of the structure (like @c
692 Opt_trace_array*). Then the @c add(key,val) call would not compile as
693 Opt_trace_array wouldn't feature it.
695 But as explained in comment of class Opt_trace_context we
696 cannot pass down the object, have to maintain a "current object or
697 array" in the Opt_trace_context context (pointer to an instance of
698 Opt_trace_struct), and the adder grabs it from the context.
700 As this current structure is of type "object or array", we cannot do
701 compile-time checks that only suitable functions are used. A call to @c
702 add(key,value) is necessarily legal for the compiler as the structure may
703 be an object, though it will be wrong in case the structure is actually
704 an array at run-time. Thus we have the risk of an untested particular
705 situation where the current structure is not an object (but an array)
706 though the code expected it to be one. This happens in practice, because
707 subqueries are evaluated in many possible places of code, not all of them
708 being known. Same happens, to a lesser extent, with calls to the range
709 optimizer.
710 So at run-time, in check_key(), we detect wrong usage, like adding a value
711 to an object without specifying a key, and then remove the unnecessary
712 key, or add an autogenerated key.
713 */
714 const char *check_key(const char *key);
716 private:
717 /// Not implemented, use add_alnum() instead.
718 Opt_trace_struct &add(const char *key, const char *value);
719 Opt_trace_struct &add(const char *key);
721 /// Full initialization. @sa Opt_trace_struct::Opt_trace_struct
722 void do_construct(Opt_trace_context *ctx, bool requires_key, const char *key,
724 /// Really does destruction
725 void do_destruct();
726 /**
727 Really adds to the object. @sa add().
729 @note add() has an up-front if(), hopefully inlined, so that in the
730 common case - tracing run-time disabled - we have no function call. If
731 tracing is enabled, we call do_add().
732 In a 20-table plan search (as in BUG#50595), the execution time was
733 decreased from 2.6 to 2.0 seconds thanks to this inlined-if trick.
735 @param key key
736 @param value value
737 @param val_length length of string 'value'
738 @param escape do JSON-compliant escaping of 'value'. If 'escape' is
739 false, 'value' should be ASCII. Otherwise, should be UTF8.
740 */
741 Opt_trace_struct &do_add(const char *key, const char *value,
742 size_t val_length, bool escape);
743 Opt_trace_struct &do_add(const char *key, const Item *item);
744 Opt_trace_struct &do_add(const char *key, bool value);
745 Opt_trace_struct &do_add(const char *key, longlong value);
746 Opt_trace_struct &do_add(const char *key, ulonglong value);
747 Opt_trace_struct &do_add(const char *key, double value);
748 Opt_trace_struct &do_add_null(const char *key);
750 Opt_trace_struct &do_add(const char *key, const Cost_estimate &value);
752 Opt_trace_struct(const Opt_trace_struct &); ///< not defined
753 Opt_trace_struct &operator=(const Opt_trace_struct &); ///< not defined
755 bool started; ///< Whether the structure does tracing or is dummy
757 /**
758 Whether the structure requires/forbids keys for values inside it.
759 true: this is an object. false: this is an array.
761 @note The canonical way would be to not have such bool per instance, but
762 rather have a pure virtual member function
763 Opt_trace_struct::requires_key(), overloaded by Opt_trace_object
764 (returning true) and by Opt_trace_array (returning false). But
765 Opt_trace_object::requires_key() would not be accessible from
766 Opt_trace_struct::do_construct() (which would complicate coding), whereas
767 the bool is.
768 */
771 /**
772 Whether this structure caused tracing to be disabled in this statement
773 because belonging to a not-traced optimizer feature, in accordance with
774 the value of @@@@optimizer_trace_features.
775 */
777 bool empty; ///< @see set_not_empty()
778 Opt_trace_stmt *stmt; ///< Trace owning the structure
779 /// Key if the structure is the value of a key/value pair, NULL otherwise
780 const char *saved_key;
781#ifndef NDEBUG
782 /**
783 Fixed-length prefix of previous key in this structure, if this structure
784 is an object. Serves to detect when adding two same consecutive keys to
785 an object, which would be wrong.
786 */
787 char previous_key[25];
792 A JSON object (unordered set of key/value pairs).
793 Defines only a constructor, all the rest is inherited from
794 Opt_trace_struct.
797 public:
798 /**
799 Constructs an object. Key is specified, so the object is the value of a
800 key/value pair.
801 @param ctx context for this object
802 @param key key
803 @param feature optimizer feature to which this structure belongs
804 */
806 Opt_trace_context *ctx, const char *key,
808 : Opt_trace_struct(ctx, true, key, feature) {}
809 /**
810 Constructs an object. No key is specified, so the object is just a value
811 (serves for the single root object or for adding the object to an array).
812 @param ctx context for this object
813 @param feature optimizer feature to which this structure belongs
814 */
818 : Opt_trace_struct(ctx, true, nullptr, feature) {}
822 A JSON array (ordered set of values).
823 Defines only a constructor, all the rest in inherited from
824 Opt_trace_struct.
827 public:
828 /**
829 Constructs an array. Key is specified, so the array is the value of a
830 key/value pair.
831 @param ctx context for this array
832 @param key key
833 @param feature optimizer feature to which this structure belongs
834 */
836 Opt_trace_context *ctx, const char *key,
838 : Opt_trace_struct(ctx, false, key, feature) {}
839 /**
840 Constructs an array. No key is specified, so the array is just a value
841 (serves for adding the object to an array).
842 @param ctx context for this array
843 @param feature optimizer feature to which this structure belongs
844 */
848 : Opt_trace_struct(ctx, false, nullptr, feature) {}
852 Instantiate an instance of this class for specific cases where
853 optimizer trace, in a certain section of Optimizer code, should write only
854 to DBUG and not I_S. Example: see
855 Note that this class should rarely be used; the "feature" parameter of
856 Opt_trace_struct is a good alternative.
859 public:
860 /**
861 @param ctx_arg Context.
862 @param disable_arg Whether the instance should really disable
863 anything. If false, the object is dummy. If true,
864 tracing-to-I_S is disabled at construction and
865 re-enabled at destruction.
866 @details A dummy instance is there only for RAII reasons. Imagine we want
867 to do this:
869 {
870 if (x) disable tracing;
871 code;
872 } // tracing should be re-enabled here
874 As we use RAII, we cannot put the instance declaration inside if(x):
876 {
877 if (x) Opt_trace_disable_I_S instance(ctx);
878 code;
879 }
881 because it would be destroyed as soon as the if() block is left, so
882 tracing would be re-enabled before @c code;. It should rather be written
883 as:
885 {
886 Opt_trace_disable_I_S instance(ctx, x); // if !x, does nothing
887 code;
888 } // re-enabling happens here, if x is true
890 */
891 Opt_trace_disable_I_S(Opt_trace_context *ctx_arg, bool disable_arg) {
892 if (disable_arg) {
893 ctx = ctx_arg;
895 } else
896 ctx = nullptr;
897 }
899 /// Destructor. Restores trace's "enabled" property to its previous value.
901 if (ctx != nullptr) ctx->restore_I_S();
902 }
904 private:
905 /** Context. Non-NULL if and only if this instance really does disabling */
909 const Opt_trace_disable_I_S &); // not defined
913 @name Helpers connecting the optimizer trace to THD or Information Schema.
919 public:
920 /**
921 Instantiate this class to start tracing a THD's actions (generally at a
922 statement's start), and to set the "original" query (not transformed, as
923 sent by client) for the new trace. Destructor will end the trace.
925 If in a routine's instruction, there is no "query". To be helpful to the
926 user, we craft a query using the instruction's print(). We don't leave this
927 to the caller, because it would be inefficient if tracing is off.
929 @param thd_arg the THD
930 @param tbl list of tables read/written by the statement.
931 @param sql_command SQL command being prepared or executed
932 @param set_vars what variables are set by this command (only used if
933 sql_command is SQLCOM_SET_OPTION)
934 @param query query
935 @param query_length query's length
936 @param instr routine's instruction, if applicable; if so, 'query'
937 and 'query_length' above are ignored
938 @param query_charset charset which was used to encode this query
940 @note Each sub-statement is responsible for ending the trace which it
941 has started; this class achieves this by keeping some memory inside.
942 */
943 Opt_trace_start(THD *thd_arg, Table_ref *tbl,
944 enum enum_sql_command sql_command,
945 List<set_var_base> *set_vars, const char *query,
946 size_t query_length, sp_printable *instr,
947 const CHARSET_INFO *query_charset);
950 private:
952 bool error; ///< whether trace start() had an error
955class Query_block;
958 Prints SELECT query to optimizer trace. It is not the original query (as in
959 @c Opt_trace_context::set_query()) but a printout of the parse tree
960 (Item-s).
961 @param thd the THD
962 @param query_block query's parse tree
963 @param trace_object Opt_trace_object to which the query will be added
966 Opt_trace_object *trace_object);
969 If the security context is not that of the connected user, inform the trace
970 system that a privilege is missing. With one exception: see below.
972 @param thd the THD
974 This serves to eliminate the following issue.
975 Any information readable by a SELECT may theoretically end up in
976 the trace. And a SELECT may read information from other places than tables:
977 - from views (reading their bodies)
978 - from stored routines (reading their bodies)
979 - from files (reading their content), with LOAD_FILE()
980 - from the list of connections (reading their queries...), with
982 If the connected user has EXECUTE privilege on a routine which does a
983 security context change, the routine can retrieve information internally
984 (if allowed by the SUID context's privileges), and present only a portion
985 of it to the connected user. But with tracing on, all information is
986 possibly in the trace. So the connected user receives more information than
987 the routine's definer intended to provide. Fixing this issue would require
988 adding, near many privilege checks in the server, a new
989 optimizer-trace-specific check done against the connected user's context,
990 to verify that the connected user has the right to see the retrieved
991 information.
993 Instead, our chosen simpler solution is that if we see a security context
994 change where SUID user is not the connected user, we disable tracing. With
995 only one safe exception: if the connected user has all global privileges
996 (because then she/he can find any information anyway). By "all global
997 privileges" we mean everything but WITH GRANT OPTION (that latter one isn't
998 related to information gathering).
1000 Read access to I_S.OPTIMIZER_TRACE by another user than the connected user
1001 is restricted: @see fill_optimizer_trace_info().
1006 If tracing is on, checks additional privileges for a view, to make sure
1007 that the user has the right to do SHOW CREATE VIEW. For that:
1008 - this function checks SHOW VIEW
1009 - SELECT is tested in opt_trace_disable_if_no_tables_access()
1010 - SELECT + SHOW VIEW is sufficient for SHOW CREATE VIEW.
1011 We also check underlying tables.
1012 If a privilege is missing, notifies the trace system.
1013 This function should be called when the view's underlying tables have not
1014 yet been merged.
1016 @param thd THD context
1017 @param view view to check
1018 @param underlying_tables underlying tables/views of 'view'
1019 */
1021 Table_ref *underlying_tables);
1024 If tracing is on, checks additional privileges on a stored routine, to make
1025 sure that the user has the right to do SHOW CREATE PROCEDURE/FUNCTION. For
1026 that, we use the same checks as in those SHOW commands.
1027 If a privilege is missing, notifies the trace system.
1029 This function is not redundant with
1030 opt_trace_disable_if_no_security_context_access().
1031 Indeed, for a SQL SECURITY INVOKER routine, there is no context change, but
1032 we must still verify that the invoker can do SHOW CREATE.
1034 For triggers, see note in sp_head::execute_trigger().
1036 @param thd The THD
1037 @param sp routine to check
1038 */
1042 Fills information_schema.OPTIMIZER_TRACE with rows (one per trace)
1043 @retval 0 ok
1044 @retval 1 error
1046int fill_optimizer_trace_info(THD *thd, Table_ref *tables, Item *);
1051 Helper for defining query-transformation-related trace objects in one
1052 code line. This produces
1053 {
1054 "transformation": {
1055 "select#": @<select_number@>,
1056 "from": @<from@>,
1057 "to": @<to@>
1058 The objects are left open, so that one can add more to them (often a
1059 "chosen" property after making some computation). Objects get closed when
1060 going out of scope as usual.
1061 @param trace optimizer trace
1062 @param object_level0 name of the outer Opt_trace_object C++ object
1063 @param object_level1 name of the inner Opt_trace_object C++ object
1064 @param select_number number of the being-transformed Query_block
1065 @param from description of the before-transformation state
1066 @param to description of the after-transformation state
1068#define OPT_TRACE_TRANSFORM(trace, object_level0, object_level1, \
1069 select_number, from, to) \
1070 const Opt_trace_object object_level0(trace); \
1071 Opt_trace_object object_level1(trace, "transformation"); \
1072 object_level1.add_select_number(select_number); \
1073 object_level1.add_alnum("from", from).add_alnum("to", to);
1075#endif /* OPT_TRACE_INCLUDED */
Kerberos Client Authentication nullptr
Used to store optimizer cost estimates.
Definition: handler.h:3793
Base class that is used to represent any kind of expression in a relational query.
Definition: item.h:932
Definition: sql_list.h:433
A JSON array (ordered set of values).
Definition: opt_trace.h:826
Opt_trace_array(Opt_trace_context *ctx, const char *key, Opt_trace_context::feature_value feature=Opt_trace_context::MISC)
Constructs an array.
Definition: opt_trace.h:835
Opt_trace_array(Opt_trace_context *ctx, Opt_trace_context::feature_value feature=Opt_trace_context::MISC)
Constructs an array.
Definition: opt_trace.h:845
A per-session context which is always available at any point of execution, because in practice it's a...
Definition: opt_trace_context.h:91
void disable_I_S_for_this_and_children()
Temporarily disables I_S for this trace and its children.
Definition: opt_trace_context.h:258
Features' numeric values for @@optimizer_trace_features variable.
Definition: opt_trace_context.h:202
Anything unclassified, including the top object (thus, by "inheritance from parent",...
Definition: opt_trace_context.h:217
bool is_started() const
Returns whether there is a current trace.
Definition: opt_trace_context.h:145
void restore_I_S()
Restores I_S support to what it was before the previous call to disable_I_S_for_this_and_children().
Definition: opt_trace_context.h:267
Instantiate an instance of this class for specific cases where optimizer trace, in a certain section ...
Definition: opt_trace.h:858
Opt_trace_disable_I_S(const Opt_trace_disable_I_S &)
Destructor. Restores trace's "enabled" property to its previous value.
Definition: opt_trace.h:900
Opt_trace_disable_I_S(Opt_trace_context *ctx_arg, bool disable_arg)
Definition: opt_trace.h:891
Opt_trace_disable_I_S & operator=(const Opt_trace_disable_I_S &)
Opt_trace_context * ctx
Definition: opt_trace.h:906
Iterator over the list of remembered traces.
Definition: opt_trace.h:406
Opt_trace_iterator(Opt_trace_context *ctx)
bool at_end() const
Definition: opt_trace.h:427
void get_value(Opt_trace_info *info) const
Provides information about the trace on which the iterator is positioned.
const Opt_trace_stmt * cursor
trace which the iterator is positioned on
Definition: opt_trace.h:432
Opt_trace_context * ctx
Pointer to context, from which traces are retrieved.
Definition: opt_trace.h:431
void next()
Advances iterator to next trace.
long row_count
how many traces retrieved so far
Definition: opt_trace.h:433
A JSON object (unordered set of key/value pairs).
Definition: opt_trace.h:796
Opt_trace_object(Opt_trace_context *ctx, Opt_trace_context::feature_value feature=Opt_trace_context::MISC)
Constructs an object.
Definition: opt_trace.h:815
Opt_trace_object(Opt_trace_context *ctx, const char *key, Opt_trace_context::feature_value feature=Opt_trace_context::MISC)
Constructs an object.
Definition: opt_trace.h:805
Definition: opt_trace.h:918
bool error
whether trace start() had an error
Definition: opt_trace.h:952
Opt_trace_context *const ctx
Definition: opt_trace.h:951
Opt_trace_start(THD *thd_arg, Table_ref *tbl, enum enum_sql_command sql_command, List< set_var_base > *set_vars, const char *query, size_t query_length, sp_printable *instr, const CHARSET_INFO *query_charset)
Instantiate this class to start tracing a THD's actions (generally at a statement's start),...
The trace of one statement.
Object and array are both "structured data" and have lots in common, so the Opt_trace_struct is a bas...
Definition: opt_trace.h:445
Opt_trace_struct & add_null(const char *key)
Adds a JSON null object (==Python's "None")
Definition: opt_trace.h:639
Opt_trace_struct & add(const Item *item)
Definition: opt_trace.h:578
Opt_trace_struct & add_select_number(uint select_number)
Helper to put the number of query_block in an object.
Definition: opt_trace.h:655
Opt_trace_struct & do_add_utf8_table(const Table_ref *tab)
char previous_key[25]
Fixed-length prefix of previous key in this structure, if this structure is an object.
Definition: opt_trace.h:787
Opt_trace_struct & add(ulonglong value)
Definition: opt_trace.h:626
const char * saved_key
Key if the structure is the value of a key/value pair, NULL otherwise.
Definition: opt_trace.h:780
Definition: opt_trace.h:473
void do_construct(Opt_trace_context *ctx, bool requires_key, const char *key, Opt_trace_context::feature_value feature)
Full initialization.
void end()
The exception to RAII: this function is an explicit way of ending a structure before it goes out of s...
Definition: opt_trace.h:483
Opt_trace_struct & add(ulong value)
Definition: opt_trace.h:610
void do_destruct()
Really does destruction.
Opt_trace_struct & add(int value)
Definition: opt_trace.h:594
bool started
Whether the structure does tracing or is dummy.
Definition: opt_trace.h:755
Opt_trace_struct & add(const char *key, longlong value)
Definition: opt_trace.h:614
Opt_trace_struct & add(const char *key, const Item *item)
Add a value (of Item type) to the structure.
Definition: opt_trace.h:574
Opt_trace_stmt * stmt
Trace owning the structure.
Definition: opt_trace.h:778
bool requires_key
Whether the structure requires/forbids keys for values inside it.
Definition: opt_trace.h:769
Opt_trace_struct & add(longlong value)
Definition: opt_trace.h:618
bool set_not_empty()
Informs this structure that we are adding data (scalars, structures) to it.
Definition: opt_trace.h:676
Opt_trace_struct & do_add(const char *key, const char *value, size_t val_length, bool escape)
Really adds to the object.
Opt_trace_struct & add(double value)
Definition: opt_trace.h:634
Opt_trace_struct & add_alnum(const char *key, const char *value)
Adds a value (of string type) to the structure.
Definition: opt_trace.h:518
Opt_trace_struct & add(const char *key, uint value)
Definition: opt_trace.h:598
Opt_trace_struct & add(const char *key, double value)
Definition: opt_trace.h:630
Opt_trace_struct & operator=(const Opt_trace_struct &)
not defined
const char * check_key(const char *key)
Validates the key about to be added.
Opt_trace_struct & add_utf8(const char *value)
Variant of add_utf8() where 'value' is 0-terminated.
Definition: opt_trace.h:560
Opt_trace_struct & add(const char *key, ulong value)
Definition: opt_trace.h:606
Opt_trace_struct & add_utf8(const char *key, const char *value, size_t val_length)
Like add_alnum() but supports any UTF8 characters in 'value'.
Definition: opt_trace.h:541
Opt_trace_struct(Opt_trace_context *ctx_arg, bool requires_key_arg, const char *key, Opt_trace_context::feature_value feature)
Definition: opt_trace.h:460
bool has_disabled_I_S
Whether this structure caused tracing to be disabled in this statement because belonging to a not-tra...
Definition: opt_trace.h:776
Opt_trace_struct & add(bool value)
Definition: opt_trace.h:586
Opt_trace_struct & add_utf8(const char *key, const char *value)
Variant of add_utf8() where 'value' is 0-terminated.
Definition: opt_trace.h:554
Opt_trace_struct & add(uint value)
Definition: opt_trace.h:602
Opt_trace_struct & add_utf8(const char *value, size_t val_length)
Variant of add_utf8() for adding to an array (no key)
Definition: opt_trace.h:548
Opt_trace_struct(const Opt_trace_struct &)
not defined
Opt_trace_struct & add(const char *key, const Cost_estimate &cost)
Add a value to the structure.
Definition: opt_trace.h:664
Opt_trace_struct & do_add_null(const char *key)
Opt_trace_struct & add(const char *key, const char *value)
Not implemented, use add_alnum() instead.
bool empty
Definition: opt_trace.h:777
Opt_trace_struct & add(const char *key, bool value)
Definition: opt_trace.h:582
Opt_trace_struct & add(const char *key)
Opt_trace_struct & add_alnum(const char *value)
Adds a value (of string type) to the structure.
Definition: opt_trace.h:529
Opt_trace_struct & add(const char *key, ulonglong value)
Definition: opt_trace.h:622
Opt_trace_struct & add_utf8_table(const Table_ref *tab)
Helper to put the database/table name in an object.
Definition: opt_trace.h:647
Opt_trace_struct & add(const char *key, int value)
Definition: opt_trace.h:590
This class represents a query block, aka a query specification, which is a query consisting of a SELE...
Definition: sql_lex.h:1161
Query_block * query_block() const override
The query_block which holds the ORDER BY and LIMIT information for this set operation.
Definition: sql_lex.h:1177
For each client connection we create a separate thread with THD serving as a thread/connection descri...
Definition: sql_lexer_thd.h:35
Definition: table.h:2846
A base class for everything that can be set with SET command.
Definition: set_var.h:949
sp_head represents one instance of a stored program.
Definition: sp_head.h:382
sp_printable defines an interface which should be implemented if a class wants report some internal i...
Definition: sp_instr.h:66
Header for compiler-dependent features.
constexpr bool likely(bool expr)
Definition: my_compiler.h:56
constexpr bool unlikely(bool expr)
Definition: my_compiler.h:57
Some integer typedefs for easier portability.
unsigned long long int ulonglong
Definition: my_inttypes.h:55
long long int longlong
Definition: my_inttypes.h:54
Definition: my_sqlcommand.h:45
static char * query
Log info(cout, "NOTE")
static std::string escape(const std::string &str)
Escapes (only) apostrophes.
void opt_trace_disable_if_no_stored_proc_func_access(THD *thd, sp_head *sp)
If tracing is on, checks additional privileges on a stored routine, to make sure that the user has th...
void opt_trace_disable_if_no_view_access(THD *thd, Table_ref *view, Table_ref *underlying_tables)
If tracing is on, checks additional privileges for a view, to make sure that the user has the right t...
void opt_trace_print_expanded_query(const THD *thd, Query_block *query_block, Opt_trace_object *trace_object)
Prints SELECT query to optimizer trace.
int fill_optimizer_trace_info(THD *thd, Table_ref *tables, Item *)
Fills information_schema.OPTIMIZER_TRACE with rows (one per trace)
void opt_trace_disable_if_no_security_context_access(THD *thd)
If the security context is not that of the connected user, inform the trace system that a privilege i...
This contains the declaration of class Opt_trace_context, which is needed to declare THD.
required string key
Definition: replication_asynchronous_connection_failover.proto:59
Definition: m_ctype.h:422
User-visible information about a trace.
Definition: opt_trace.h:376
size_t missing_bytes
How many bytes this trace is missing (for traces which were truncated because of @@optimizer-trace-ma...
Definition: opt_trace.h:397
size_t trace_length
length of trace string / String containing original query. 0-termination: like trace_ptr.
Definition: opt_trace.h:388
const CHARSET_INFO * query_charset
charset of query string
Definition: opt_trace.h:392
const char * query_ptr
Definition: opt_trace.h:390
const char * trace_ptr
String containing trace.
Definition: opt_trace.h:387
bool missing_priv
whether user lacks privilege to see this trace
Definition: opt_trace.h:398
size_t query_length
length of query string
Definition: opt_trace.h:391