MySQL 8.3.0
Source Code Documentation
Go to the documentation of this file.
1/* Copyright (c) 2011, 2023, Oracle and/or its affiliates.
3 This program is free software; you can redistribute it and/or modify
4 it under the terms of the GNU General Public License, version 2.0,
5 as published by the Free Software Foundation.
7 This program is also distributed with certain software (including
8 but not limited to OpenSSL) that is licensed under separate terms,
9 as designated in a particular file or component or in included license
10 documentation. The authors of MySQL hereby grant you an additional
11 permission to link the program and your derivative works with the
12 separately licensed software that they have included with MySQL.
14 This program is distributed in the hope that it will be useful,
15 but WITHOUT ANY WARRANTY; without even the implied warranty of
17 GNU General Public License, version 2.0, for more details.
19 You should have received a copy of the GNU General Public License
20 along with this program; if not, write to the Free Software
21 Foundation, Inc., 51 Franklin St, Fifth Floor, Boston, MA 02110-1301 USA */
26#include <limits.h>
27#include <string.h>
28#include <sys/types.h>
30#include "my_compiler.h"
31#include "my_inttypes.h"
32#include "my_sqlcommand.h" // enum_sql_command
33#include "sql/opt_trace_context.h" // Opt_trace_context
35class Cost_estimate;
36class Item;
37class THD;
38class set_var_base;
39class sp_head;
40class sp_printable;
41struct CHARSET_INFO;
42class Table_ref;
43template <class T>
44class List;
47 @file sql/opt_trace.h
48 API for the Optimizer trace (WL#5257)
52 @page PAGE_OPT_TRACE The Optimizer Trace
54 @section INTRODUCTION Introduction
56 This optimizer trace is aimed at producing output, which is readable by
57 humans and by programs, to aid understanding of decisions and actions taken
58 by the MySQL Optimizer.
60 @section OUTPUT_FORMAT Output format
62 The chosen output format is JSON (JavaScript Object Notation).
63 In JSON there are:
64 @li "objects" (unordered set of key-value pairs); equivalent to Python's
65 dictionary or Perl's associative array or hash or STL's hash_map.
66 @li "arrays" (ordered set of values); equivalent to Python's and Perl's list
67 or STL's vector.
68 @li "values": a value can be a string, number, boolean, null,
69 which we all call "scalars", or be an object, array.
71 For example (explanations after "<<" are not part of output):
73 { << start of top object
74 "first_name": "Gustave", << key/value pair (value is string)
75 "last_name": "Eiffel", << key/value pair (value is string)
76 "born": 1832, << key/value pair (value is integer)
77 "contributions_to": [ << key/value pair (value is array)
78 { << 1st item of array is an object (a building)
79 "name": "Eiffel tower",
80 "location": Paris
81 }, << end of 1st item of array
82 {
83 "name": "Liberty statue",
84 "location": "New York"
85 } << end of 2nd item of array
86 ] << end of array
87 } << end of top object
89 For more details, have a look at the syntax at
90 Note that indentation and newlines are superfluous, useful only for
91 human-readability.
92 Note also that there is nothing like a "named object": an object, array or
93 value has no name; but if it is the value of a key/value pair in an
94 enclosing, outer object, then the key can be seen as the inner object's
95 "name".
97 @section USER_ENABLE_TRACING How a user enables/views the trace
100 SET SESSION OPTIMIZER_TRACE="enabled=on"; # enable tracing
101 <statement to trace>; # like SELECT, EXPLAIN SELECT, UPDATE, DELETE...
102 SELECT * FROM information_schema.OPTIMIZER_TRACE;
103 [ repeat last two steps at will ]
104 SET SESSION OPTIMIZER_TRACE="enabled=off"; # disable tracing
107 @c SELECT and @c EXPLAIN SELECT produce the same trace. But there are
108 exceptions regarding subqueries because the two commands treat subqueries
109 differently, for example in
111 SELECT ... WHERE x IN (subq1) AND y IN (subq2)
113 SELECT terminates after executing the first subquery if the related IN
114 predicate is false, so we won't see @c JOIN::optimize() tracing for subq2;
115 whereas EXPLAIN SELECT analyzes all subqueries (see loop at the end of
116 @c select_describe()).
118 @section USER_SELECT_TRACING_STATEMENTS How a user traces only certain
121 When tracing is in force, each SQL statement generates a trace; more
122 exactly, so does any of
126 UPDATE/DELETE and their multi-table variants,
127 SET (unless it manipulates @@@@optimizer_trace),
128 DO,
129 DECLARE/CASE/IF/RETURN (stored routines language elements),
130 CALL.
131 If a command above is prepared and executed in separate steps, preparation
132 and execution are separately traced.
133 By default each new trace overwrites the previous trace. Thus, if a
134 statement contains sub-statements (example: invokes stored procedures,
135 stored functions, triggers), the top statement and sub-statements each
136 generate traces, but at the execution's end only the last sub-statement's
137 trace is visible.
138 If the user wants to see the trace of another sub-statement, she/he can
139 enable/disable tracing around the desired sub-statement, but this requires
140 editing the routine's code, which may not be possible. Another solution is
141 to use
143 SET optimizer_trace_offset=<OFFSET>, optimizer_trace_limit=<LIMIT>
145 where OFFSET is a signed integer, and LIMIT is a positive integer.
146 The effect of this SET is the following:
148 @li all remembered traces are cleared
150 @li a later SELECT on OPTIMIZER_TRACE returns the first LIMIT traces of
151 the OFFSET oldest remembered traces (if OFFSET >= 0), or the first LIMIT
152 traces of the -OFFSET newest remembered traces (if OFFSET < 0).
154 For example,
155 a combination of OFFSET=-1 and LIMIT=1 will make the last trace be shown (as
156 is default), OFFSET=-2 and LIMIT=1 will make the next-to-last be shown,
157 OFFSET=-5 and LIMIT=5 will make the last five traces be shown. Such negative
158 OFFSET can be useful when one knows that the interesting sub-statements are
159 the few last ones of a stored routine, like this:
161 SET optimizer_trace_offset=-5, optimizer_trace_limit=5;
162 CALL stored_routine(); # more than 5 sub-statements in this routine
163 SELECT * FROM information_schema.OPTIMIZER_TRACE; # see only last 5 traces
165 On the opposite, a positive OFFSET can be useful when one knows that the
166 interesting sub-statements are the few first ones of a stored routine.
168 The more those two variables are accurately adjusted, the less memory is
169 used. For example, OFFSET=0 and LIMIT=5 will use memory to remember 5
170 traces, so if only the three first are needed, OFFSET=0 and LIMIT=3 is
171 better (tracing stops after the LIMITth trace, so the 4th and 5th trace are
172 not created and don't take up memory). A stored routine may have a loop
173 which executes many sub-statements and thus generates many traces, which
174 would use a lot of memory; proper OFFSET and LIMIT can restrict tracing to
175 one iteration of the loop for example. This also gains speed, as tracing a
176 sub-statement impacts performance.
178 If OFFSET>=0, only LIMIT traces are kept in memory. If OFFSET<0, that is not
179 true: instead, (-OFFSET) traces are kept in memory; indeed even if LIMIT is
180 smaller than (-OFFSET), so excludes the last statement, the last statement
181 must still be traced because it will be inside LIMIT after executing one
182 more statement (remember than OFFSET<0 is counted from the end: the "window"
183 slides as more statements execute).
185 Such memory and speed gains are the reason why optimizer_trace_offset/limit,
186 which are restrictions at the trace producer level, are offered. They are
187 better than using
191 which is a restriction on the trace consumer level, which saves almost
192 nothing.
194 @section USER_SELECT_TRACING_FEATURES How a user traces only certain
195 optimizer features
198 SET OPTIMIZER_TRACE_FEATURES="feature1=on|off,...";
200 where "feature1" is one optimizer feature. For example "greedy_search": a
201 certain Opt_trace_array at the start of @c
202 Optimize_table_order::choose_table_order() has a flag "GREEDY_SEARCH" passed
203 to its constructor: this means that if the user has turned tracing of greedy
204 search off, this array will not be written to the I_S trace, neither will
205 any children structures. All this disabled "trace chunk" will be replaced by
206 an ellipsis "...".
208 @section DEV_ADDS_TRACING How a developer adds tracing to a function
210 Check @c Opt_trace* usage in @c advance_sj_state():
213 Opt_trace_array trace_choices(trace, "semijoin_strategy_choice");
216 This creates an array for key "semijoin_strategy_choice". We are going to
217 list possible semijoin strategy choices.
220 Opt_trace_object trace_one_strategy(trace);
223 This creates an object without key (normal, it's in an array). This
224 object will describe one single strategy choice.
227 trace_one_strategy.add_alnum("strategy", "FirstMatch");
230 This adds a key/value pair to the just-created object: key is "strategy",
231 value is "FirstMatch". This is the strategy to be described in the
232 just-created object.
235 trace_one_strategy.add("cost", *current_read_time).
236 add("records", *current_record_count);
237 trace_one_strategy.add("chosen", (pos->sj_strategy == SJ_OPT_FIRST_MATCH));
240 This adds 3 key/value pairs: cost of strategy, number of records produced
241 by this strategy, and whether this strategy is chosen.
243 After that, there is similar code for other semijoin strategies.
245 The resulting trace piece (seen in @c information_schema.OPTIMIZER_TRACE) is
247 "semijoin_strategy_choice": [
248 {
249 "strategy": "FirstMatch",
250 "cost": 1,
251 "records": 1,
252 "chosen": true
253 },
254 {
255 "strategy": "DuplicatesWeedout",
256 "cost": 1.1,
257 "records": 1,
258 "duplicate_tables_left": false,
259 "chosen": false
260 }
261 ]
264 For more output examples, check result files of the opt_trace suite in
265 @c mysql-test.
267 Feature can be un-compiled with @code cmake -DOPTIMIZER_TRACE=0 @endcode.
269 @section WITH_DBUG Interaction between trace and DBUG
271 We don't want to have to duplicate code like this:
273 DBUG_PRINT("info",("cost %g",cost));
274 Opt_trace_object(thd->opt_trace).add("cost",cost);
277 Thus, any optimizer trace operation, *even* if tracing is run-time disabled,
278 has an implicit DBUG_PRINT("opt",...) inside. This way, only the
279 second line above is needed, and several DBUG_PRINT() could be removed from
280 the Optimizer code.
281 When tracing is run-time disabled, in a debug binary, traces are still
282 created in order to catch the @c add() calls and write their text to DBUG,
283 but those traces are not visible into INFORMATION_SCHEMA.OPTIMIZER_TRACE: we
284 then say that they "don't support I_S".
285 A debug binary without optimizer trace compiled in, will intentionally not
286 compile.
288 Because opening an object or array, or add()-ing to it, writes to DBUG
289 immediately, a key/value pair and its outer object may be 100 lines
290 apart in the DBUG log.
292 @section ADDING_TRACING Guidelines for adding tracing
294 @li Try to limit the number of distinct "words". For example, when
295 describing an optimizer's decision, the words "chosen" (true/false value,
296 tells whether we are choosing the said optimization), "cause" (free text
297 value, tells why we are making this choice, when it's not obvious)
298 can and should often be used. Having a restricted vocabulary helps
299 consistency. Use "row" instead of "record". Use "tmp" instead of
300 "temporary".
302 @li Use only simple characters for key names: a-ZA-Z_#, and no space. '#'
303 serves to denote a number, like in "select#" .
305 @li Keep in mind than in an object, keys are not ordered; an application may
306 parse the JSON output and output it again with keys order changed; thus
307 when order matters, use an array (which may imply having anonymous objects
308 as items of the array, with keys inside the anonymous objects, see how it's
309 done in @c JOIN::optimize()). Keep in mind that in an object keys should
310 be unique, an application may lose duplicate keys.
312 @section OOM_HANDLING Handling of "out-of-memory" errors
314 All memory allocations (with exceptions: see below) in the Optimizer trace
315 use @c my_error() to report errors, which itself calls @c
316 error_handler_hook. It is the responsibility of the API user to set up a
317 proper @c error_handler_hook which will alert her/him of the OOM
318 problem. When in the server, this is already the case (@c error_handler_hook
319 is @c my_message_sql() which makes the statement fail).
320 Note that the debug binary may crash if OOM (OOM can cause syntax
321 errors...).
323 @section TRACE_SECURITY Description of trace-induced security checks.
325 A trace exposes information. For example if one does SELECT on a view, the
326 trace contains the view's body. So, the user should be allowed to see the
327 trace only if she/he has privilege to see the body, i.e. privilege to do
329 There are similar issues with stored procedures, functions, triggers.
331 We implement this by doing additional checks on SQL objects when tracing is
332 on:
333 @li stored procedures, functions, triggers: checks are done when executing
334 those objects
335 @li base tables and views.
337 Base tables or views are listed in some @c LEX::query_tables.
338 The LEX may be of the executing statement (statement executed by
339 @c mysql_execute_command(), or by
340 @c sp_lex_keeper::reset_lex_and_exec_core()), we check this LEX in the
341 constructor of Opt_trace_start.
342 Or it may be a LEX describing a view, we check this LEX when
343 opening the view (@c open_and_read_view()).
345 Those checks are greatly simplified by disabling traces in case of security
346 context changes. @see opt_trace_disable_if_no_security_context_access().
348 Those checks must be done with the security context of the connected
349 user. Checks with the SUID context would be useless: assume the design is
350 that the basic user does not have DML privileges on tables, but only
351 EXECUTE on SUID-highly-privileged routines (which implement _controlled_
352 _approved_ DMLs): then the SUID context would successfully pass all
353 additional privilege checks, routine would generate tracing, and the
354 connected user would view the trace after the routine's execution, seeing
355 secret information.
357 @section NEXT What a developer should read next
359 The documentation of those classes, in order
361 Opt_trace_context
362 Opt_trace_context_impl
363 Opt_trace_stmt
364 Opt_trace_struct
365 Opt_trace_object
366 Opt_trace_array
368 and then @ref opt_trace.h as a whole.
371class Opt_trace_stmt; // implementation detail local to
374 User-visible information about a trace. @sa Opt_trace_iterator.
377 /**
378 String containing trace.
379 If trace has been end()ed, this is 0-terminated, which is only to aid
380 debugging or unit testing; this property is not relied upon in normal
381 server usage.
382 If trace has not been ended, this is not 0-terminated. That rare case can
383 happen when a substatement reads OPTIMIZER_TRACE (at that stage, the top
384 statement is still executing so its trace is not ended yet, but may still
385 be read by the sub-statement).
386 */
387 const char *trace_ptr;
388 size_t trace_length; ///< length of trace string
389 //// String containing original query. 0-termination: like trace_ptr.
390 const char *query_ptr;
391 size_t query_length; ///< length of query string
392 const CHARSET_INFO *query_charset; ///< charset of query string
393 /**
394 How many bytes this trace is missing (for traces which were truncated
395 because of @@@@optimizer-trace-max-mem-size).
396 */
398 bool missing_priv; ///< whether user lacks privilege to see this trace
402 Iterator over the list of remembered traces.
403 @note due to implementation, the list must not change during an
404 iterator's lifetime, or results may be unexpected (no crash though).
407 public:
408 /**
409 @param ctx context
410 */
413 void next(); ///< Advances iterator to next trace.
415 /**
416 Provides information about the trace on which the iterator is
417 positioned.
418 @param[out] info information returned.
419 The usage pattern is
420 1) instantiate the iterator
421 2) test at_end(), if false: call get_value() and then next()
422 3) repeat (2) until at_end() is true.
423 */
424 void get_value(Opt_trace_info *info) const;
426 /// @returns whether iterator is positioned to the end.
427 bool at_end() const { return cursor == nullptr; }
429 private:
430 /// Pointer to context, from which traces are retrieved
432 const Opt_trace_stmt *cursor; ///< trace which the iterator is positioned on
433 long row_count; ///< how many traces retrieved so far
437 Object and array are both "structured data" and have lots in common, so the
438 Opt_trace_struct is a base class for them.
439 When you want to add a structure to the trace, you create an instance of
440 Opt_trace_object or Opt_trace_array, then you add information to it with
441 add(), then the destructor closes the structure (we use RAII, Resource
442 Acquisition Is Initialization).
446 protected:
447 /**
448 @param ctx_arg Optimizer trace context for this structure
449 @param requires_key_arg whether this structure requires/forbids keys
450 for values put inside it (an object requires them, an
451 array forbids them)
452 @param key key if this structure is the value of a key/value pair,
453 NULL otherwise. This pointer must remain constant and
454 valid until the object is destroyed (to support
455 @ref saved_key).
456 @param feature optimizer feature to which this structure belongs
458 This constructor is never called directly, only from subclasses.
459 */
460 Opt_trace_struct(Opt_trace_context *ctx_arg, bool requires_key_arg,
461 const char *key, Opt_trace_context::feature_value feature)
462 : started(false),
463 requires_key(false),
464 has_disabled_I_S(false),
465 empty(false),
466 stmt(nullptr),
468 // A first inlined test
469 if (unlikely(ctx_arg->is_started())) {
470 // Tracing enabled: must fully initialize the structure.
471 do_construct(ctx_arg, requires_key_arg, key, feature);
472 }
473 /*
474 Otherwise, just leave "started" to false, it marks that the structure is
475 dummy.
476 */
477 }
480 }
482 public:
483 /**
484 The exception to RAII: this function is an explicit way of ending a
485 structure before it goes out of scope. Don't use it unless RAII mandates
486 a new scope which mandates re-indenting lots of code lines.
487 */
488 void end() {
490 }
492 /**
493 Adds a value (of string type) to the structure. A key is specified, so it
494 adds the key/value pair (the structure must thus be an object).
496 There are two "add_*" variants to add a string value.
497 If the value is 0-terminated and each character
498 - is ASCII 7-bit
499 - has ASCII code >=32 and is neither '"' nor '\\'
500 then add_alnum() should be used. That should be the case for all fixed
501 strings like add_alnum("cause", "cost").
502 Otherwise, add_utf8() should be used; it accepts any UTF8-encoded
503 character in 'value' and will escape characters which JSON requires (and
504 is thus slower than add_alnum()). It should be used for all strings which
505 we get from the server's objects (indeed a table's name, a WHERE
506 condition, may contain "strange" characters).
508 @param key key
509 @param value value
510 @returns a reference to the structure, useful for chaining like this:
511 @verbatim add(x,y).add(z,t).add(u,v) @endverbatim
513 String-related add() variants are named add_[something]():
514 - to avoid confusing the compiler between:
515 add(const char *value, size_t val_length) and
516 add(const char *key, ulonglong value)
517 - and because String::length() returns uint32 and not size_t, so for
518 add(str.ptr(), str.length())
519 compiler may pick
520 add(const char *key, double value) instead of
521 add(const char *value, size_t val_length).
522 */
523 Opt_trace_struct &add_alnum(const char *key, const char *value) {
524 if (likely(!started)) return *this;
525 return do_add(key, value, strlen(value), false);
526 }
528 /**
529 Adds a value (of string type) to the structure. No key is specified, so
530 it adds only the value (the structure must thus be an array).
531 @param value value
532 @returns a reference to the structure
533 */
534 Opt_trace_struct &add_alnum(const char *value) {
535 if (likely(!started)) return *this;
536 return do_add(nullptr, value, strlen(value), false);
537 }
539 /**
540 Like add_alnum() but supports any UTF8 characters in 'value'.
541 Will "escape" 'value' to be JSON-compliant.
542 @param key key
543 @param value value
544 @param val_length length of string 'value'
545 */
546 Opt_trace_struct &add_utf8(const char *key, const char *value,
547 size_t val_length) {
548 if (likely(!started)) return *this;
549 return do_add(key, value, val_length, true);
550 }
552 /// Variant of add_utf8() for adding to an array (no key)
553 Opt_trace_struct &add_utf8(const char *value, size_t val_length) {
554 if (likely(!started)) return *this;
555 return do_add(nullptr, value, val_length, true);
556 }
558 /// Variant of add_utf8() where 'value' is 0-terminated
559 Opt_trace_struct &add_utf8(const char *key, const char *value) {
560 if (likely(!started)) return *this;
561 return do_add(key, value, strlen(value), true);
562 }
564 /// Variant of add_utf8() where 'value' is 0-terminated
565 Opt_trace_struct &add_utf8(const char *value) {
566 if (likely(!started)) return *this;
567 return do_add(nullptr, value, strlen(value), true);
568 }
570 /**
571 Add a value (of Item type) to the structure. The Item should be a
572 condition (like a WHERE clause) which will be pretty-printed into the
573 trace. This is useful for showing condition transformations (equality
574 propagation etc).
575 @param key key
576 @param item the Item
577 @return a reference to the structure
578 */
579 Opt_trace_struct &add(const char *key, const Item *item) {
580 if (likely(!started)) return *this;
581 return do_add(key, item);
582 }
583 Opt_trace_struct &add(const Item *item) {
584 if (likely(!started)) return *this;
585 return do_add(nullptr, item);
586 }
587 Opt_trace_struct &add(const char *key, bool value) {
588 if (likely(!started)) return *this;
589 return do_add(key, value);
590 }
591 Opt_trace_struct &add(bool value) {
592 if (likely(!started)) return *this;
593 return do_add(nullptr, value);
594 }
595 Opt_trace_struct &add(const char *key, int value) {
596 if (likely(!started)) return *this;
597 return do_add(key, static_cast<longlong>(value));
598 }
599 Opt_trace_struct &add(int value) {
600 if (likely(!started)) return *this;
601 return do_add(nullptr, static_cast<longlong>(value));
602 }
603 Opt_trace_struct &add(const char *key, uint value) {
604 if (likely(!started)) return *this;
605 return do_add(key, static_cast<ulonglong>(value));
606 }
607 Opt_trace_struct &add(uint value) {
608 if (likely(!started)) return *this;
609 return do_add(nullptr, static_cast<ulonglong>(value));
610 }
611 Opt_trace_struct &add(const char *key, ulong value) {
612 if (likely(!started)) return *this;
613 return do_add(key, static_cast<ulonglong>(value));
614 }
615 Opt_trace_struct &add(ulong value) {
616 if (likely(!started)) return *this;
617 return do_add(nullptr, static_cast<ulonglong>(value));
618 }
619 Opt_trace_struct &add(const char *key, longlong value) {
620 if (likely(!started)) return *this;
621 return do_add(key, value);
622 }
624 if (likely(!started)) return *this;
625 return do_add(nullptr, value);
626 }
627 Opt_trace_struct &add(const char *key, ulonglong value) {
628 if (likely(!started)) return *this;
629 return do_add(key, value);
630 }
632 if (likely(!started)) return *this;
633 return do_add(nullptr, value);
634 }
635 Opt_trace_struct &add(const char *key, double value) {
636 if (likely(!started)) return *this;
637 return do_add(key, value);
638 }
639 Opt_trace_struct &add(double value) {
640 if (likely(!started)) return *this;
641 return do_add(nullptr, value);
642 }
643 /// Adds a JSON null object (==Python's "None")
645 if (likely(!started)) return *this;
646 return do_add_null(key);
647 }
648 /**
649 Helper to put the database/table name in an object.
650 @param tab TABLE* pointer
651 */
653 if (likely(!started)) return *this;
654 return do_add_utf8_table(tab);
655 }
656 /**
657 Helper to put the number of query_block in an object.
658 @param select_number number of query_block
659 */
660 Opt_trace_struct &add_select_number(uint select_number) {
661 return add("select#", select_number);
662 }
663 /**
664 Add a value to the structure.
665 @param key key
666 @param cost the value of Cost_estimate
667 @return a reference to the structure
668 */
669 Opt_trace_struct &add(const char *key, const Cost_estimate &cost) {
670 if (likely(!started)) return *this;
671 return do_add(key, cost);
672 }
674 /**
675 Informs this structure that we are adding data (scalars, structures) to
676 it.
677 This is used only if sending to I_S.
678 @returns whether the structure was empty so far.
679 @note this is reserved for use by Opt_trace_stmt.
680 */
682 const bool old_empty = empty;
683 empty = false;
684 return old_empty;
685 }
686 /**
687 Validates the key about to be added.
688 @note this is reserved for use by Opt_trace_stmt.
690 When adding a value (or array or object) to an array, or a key/value pair
691 to an object, we need to know this outer array or object.
693 It would be possible, when trying to add a key to an array, which is wrong
694 in JSON, or similarly when trying to add a value without any key to an
695 object, to catch it at compilation time, if the adder received, as
696 function parameter, the type of the structure (like @c
697 Opt_trace_array*). Then the @c add(key,val) call would not compile as
698 Opt_trace_array wouldn't feature it.
700 But as explained in comment of class Opt_trace_context we
701 cannot pass down the object, have to maintain a "current object or
702 array" in the Opt_trace_context context (pointer to an instance of
703 Opt_trace_struct), and the adder grabs it from the context.
705 As this current structure is of type "object or array", we cannot do
706 compile-time checks that only suitable functions are used. A call to @c
707 add(key,value) is necessarily legal for the compiler as the structure may
708 be an object, though it will be wrong in case the structure is actually
709 an array at run-time. Thus we have the risk of an untested particular
710 situation where the current structure is not an object (but an array)
711 though the code expected it to be one. This happens in practice, because
712 subqueries are evaluated in many possible places of code, not all of them
713 being known. Same happens, to a lesser extent, with calls to the range
714 optimizer.
715 So at run-time, in check_key(), we detect wrong usage, like adding a value
716 to an object without specifying a key, and then remove the unnecessary
717 key, or add an autogenerated key.
718 */
719 const char *check_key(const char *key);
721 private:
722 /// Not implemented, use add_alnum() instead.
723 Opt_trace_struct &add(const char *key, const char *value);
724 Opt_trace_struct &add(const char *key);
726 /// Full initialization. @sa Opt_trace_struct::Opt_trace_struct
727 void do_construct(Opt_trace_context *ctx, bool requires_key, const char *key,
729 /// Really does destruction
730 void do_destruct();
731 /**
732 Really adds to the object. @sa add().
734 @note add() has an up-front if(), hopefully inlined, so that in the
735 common case - tracing run-time disabled - we have no function call. If
736 tracing is enabled, we call do_add().
737 In a 20-table plan search (as in BUG#50595), the execution time was
738 decreased from 2.6 to 2.0 seconds thanks to this inlined-if trick.
740 @param key key
741 @param value value
742 @param val_length length of string 'value'
743 @param escape do JSON-compliant escaping of 'value'. If 'escape' is
744 false, 'value' should be ASCII. Otherwise, should be UTF8.
745 */
746 Opt_trace_struct &do_add(const char *key, const char *value,
747 size_t val_length, bool escape);
748 Opt_trace_struct &do_add(const char *key, const Item *item);
749 Opt_trace_struct &do_add(const char *key, bool value);
750 Opt_trace_struct &do_add(const char *key, longlong value);
751 Opt_trace_struct &do_add(const char *key, ulonglong value);
752 Opt_trace_struct &do_add(const char *key, double value);
753 Opt_trace_struct &do_add_null(const char *key);
755 Opt_trace_struct &do_add(const char *key, const Cost_estimate &value);
757 Opt_trace_struct(const Opt_trace_struct &); ///< not defined
758 Opt_trace_struct &operator=(const Opt_trace_struct &); ///< not defined
760 bool started; ///< Whether the structure does tracing or is dummy
762 /**
763 Whether the structure requires/forbids keys for values inside it.
764 true: this is an object. false: this is an array.
766 @note The canonical way would be to not have such bool per instance, but
767 rather have a pure virtual member function
768 Opt_trace_struct::requires_key(), overloaded by Opt_trace_object
769 (returning true) and by Opt_trace_array (returning false). But
770 Opt_trace_object::requires_key() would not be accessible from
771 Opt_trace_struct::do_construct() (which would complicate coding), whereas
772 the bool is.
773 */
776 /**
777 Whether this structure caused tracing to be disabled in this statement
778 because belonging to a not-traced optimizer feature, in accordance with
779 the value of @@@@optimizer_trace_features.
780 */
782 bool empty; ///< @see set_not_empty()
783 Opt_trace_stmt *stmt; ///< Trace owning the structure
784 /// Key if the structure is the value of a key/value pair, NULL otherwise
785 const char *saved_key;
786#ifndef NDEBUG
787 /**
788 Fixed-length prefix of previous key in this structure, if this structure
789 is an object. Serves to detect when adding two same consecutive keys to
790 an object, which would be wrong.
791 */
792 char previous_key[25];
797 A JSON object (unordered set of key/value pairs).
798 Defines only a constructor, all the rest is inherited from
799 Opt_trace_struct.
802 public:
803 /**
804 Constructs an object. Key is specified, so the object is the value of a
805 key/value pair.
806 @param ctx context for this object
807 @param key key
808 @param feature optimizer feature to which this structure belongs
809 */
811 Opt_trace_context *ctx, const char *key,
813 : Opt_trace_struct(ctx, true, key, feature) {}
814 /**
815 Constructs an object. No key is specified, so the object is just a value
816 (serves for the single root object or for adding the object to an array).
817 @param ctx context for this object
818 @param feature optimizer feature to which this structure belongs
819 */
823 : Opt_trace_struct(ctx, true, nullptr, feature) {}
827 A JSON array (ordered set of values).
828 Defines only a constructor, all the rest in inherited from
829 Opt_trace_struct.
832 public:
833 /**
834 Constructs an array. Key is specified, so the array is the value of a
835 key/value pair.
836 @param ctx context for this array
837 @param key key
838 @param feature optimizer feature to which this structure belongs
839 */
841 Opt_trace_context *ctx, const char *key,
843 : Opt_trace_struct(ctx, false, key, feature) {}
844 /**
845 Constructs an array. No key is specified, so the array is just a value
846 (serves for adding the object to an array).
847 @param ctx context for this array
848 @param feature optimizer feature to which this structure belongs
849 */
853 : Opt_trace_struct(ctx, false, nullptr, feature) {}
857 Instantiate an instance of this class for specific cases where
858 optimizer trace, in a certain section of Optimizer code, should write only
859 to DBUG and not I_S. Example: see
860 Note that this class should rarely be used; the "feature" parameter of
861 Opt_trace_struct is a good alternative.
864 public:
865 /**
866 @param ctx_arg Context.
867 @param disable_arg Whether the instance should really disable
868 anything. If false, the object is dummy. If true,
869 tracing-to-I_S is disabled at construction and
870 re-enabled at destruction.
871 @details A dummy instance is there only for RAII reasons. Imagine we want
872 to do this:
874 {
875 if (x) disable tracing;
876 code;
877 } // tracing should be re-enabled here
879 As we use RAII, we cannot put the instance declaration inside if(x):
881 {
882 if (x) Opt_trace_disable_I_S instance(ctx);
883 code;
884 }
886 because it would be destroyed as soon as the if() block is left, so
887 tracing would be re-enabled before @c code;. It should rather be written
888 as:
890 {
891 Opt_trace_disable_I_S instance(ctx, x); // if !x, does nothing
892 code;
893 } // re-enabling happens here, if x is true
895 */
896 Opt_trace_disable_I_S(Opt_trace_context *ctx_arg, bool disable_arg) {
897 if (disable_arg) {
898 ctx = ctx_arg;
900 } else
901 ctx = nullptr;
902 }
904 /// Destructor. Restores trace's "enabled" property to its previous value.
906 if (ctx != nullptr) ctx->restore_I_S();
907 }
909 private:
910 /** Context. Non-NULL if and only if this instance really does disabling */
914 const Opt_trace_disable_I_S &); // not defined
918 @name Helpers connecting the optimizer trace to THD or Information Schema.
924 public:
925 /**
926 Instantiate this class to start tracing a THD's actions (generally at a
927 statement's start), and to set the "original" query (not transformed, as
928 sent by client) for the new trace. Destructor will end the trace.
930 If in a routine's instruction, there is no "query". To be helpful to the
931 user, we craft a query using the instruction's print(). We don't leave this
932 to the caller, because it would be inefficient if tracing is off.
934 @param thd_arg the THD
935 @param tbl list of tables read/written by the statement.
936 @param sql_command SQL command being prepared or executed
937 @param set_vars what variables are set by this command (only used if
938 sql_command is SQLCOM_SET_OPTION)
939 @param query query
940 @param query_length query's length
941 @param instr routine's instruction, if applicable; if so, 'query'
942 and 'query_length' above are ignored
943 @param query_charset charset which was used to encode this query
945 @note Each sub-statement is responsible for ending the trace which it
946 has started; this class achieves this by keeping some memory inside.
947 */
948 Opt_trace_start(THD *thd_arg, Table_ref *tbl,
949 enum enum_sql_command sql_command,
950 List<set_var_base> *set_vars, const char *query,
951 size_t query_length, sp_printable *instr,
952 const CHARSET_INFO *query_charset);
955 private:
957 bool error; ///< whether trace start() had an error
960class Query_block;
963 Prints SELECT query to optimizer trace. It is not the original query (as in
964 @c Opt_trace_context::set_query()) but a printout of the parse tree
965 (Item-s).
966 @param thd the THD
967 @param query_block query's parse tree
968 @param trace_object Opt_trace_object to which the query will be added
971 Opt_trace_object *trace_object);
974 If the security context is not that of the connected user, inform the trace
975 system that a privilege is missing. With one exception: see below.
977 @param thd the THD
979 This serves to eliminate the following issue.
980 Any information readable by a SELECT may theoretically end up in
981 the trace. And a SELECT may read information from other places than tables:
982 - from views (reading their bodies)
983 - from stored routines (reading their bodies)
984 - from files (reading their content), with LOAD_FILE()
985 - from the list of connections (reading their queries...), with
987 If the connected user has EXECUTE privilege on a routine which does a
988 security context change, the routine can retrieve information internally
989 (if allowed by the SUID context's privileges), and present only a portion
990 of it to the connected user. But with tracing on, all information is
991 possibly in the trace. So the connected user receives more information than
992 the routine's definer intended to provide. Fixing this issue would require
993 adding, near many privilege checks in the server, a new
994 optimizer-trace-specific check done against the connected user's context,
995 to verify that the connected user has the right to see the retrieved
996 information.
998 Instead, our chosen simpler solution is that if we see a security context
999 change where SUID user is not the connected user, we disable tracing. With
1000 only one safe exception: if the connected user has all global privileges
1001 (because then she/he can find any information anyway). By "all global
1002 privileges" we mean everything but WITH GRANT OPTION (that latter one isn't
1003 related to information gathering).
1005 Read access to I_S.OPTIMIZER_TRACE by another user than the connected user
1006 is restricted: @see fill_optimizer_trace_info().
1011 If tracing is on, checks additional privileges for a view, to make sure
1012 that the user has the right to do SHOW CREATE VIEW. For that:
1013 - this function checks SHOW VIEW
1014 - SELECT is tested in opt_trace_disable_if_no_tables_access()
1015 - SELECT + SHOW VIEW is sufficient for SHOW CREATE VIEW.
1016 We also check underlying tables.
1017 If a privilege is missing, notifies the trace system.
1018 This function should be called when the view's underlying tables have not
1019 yet been merged.
1021 @param thd THD context
1022 @param view view to check
1023 @param underlying_tables underlying tables/views of 'view'
1024 */
1026 Table_ref *underlying_tables);
1029 If tracing is on, checks additional privileges on a stored routine, to make
1030 sure that the user has the right to do SHOW CREATE PROCEDURE/FUNCTION. For
1031 that, we use the same checks as in those SHOW commands.
1032 If a privilege is missing, notifies the trace system.
1034 This function is not redundant with
1035 opt_trace_disable_if_no_security_context_access().
1036 Indeed, for a SQL SECURITY INVOKER routine, there is no context change, but
1037 we must still verify that the invoker can do SHOW CREATE.
1039 For triggers, see note in sp_head::execute_trigger().
1041 @param thd The THD
1042 @param sp routine to check
1043 */
1047 Fills information_schema.OPTIMIZER_TRACE with rows (one per trace)
1048 @retval 0 ok
1049 @retval 1 error
1051int fill_optimizer_trace_info(THD *thd, Table_ref *tables, Item *);
1056 Helper for defining query-transformation-related trace objects in one
1057 code line. This produces
1058 {
1059 "transformation": {
1060 "select#": @<select_number@>,
1061 "from": @<from@>,
1062 "to": @<to@>
1063 The objects are left open, so that one can add more to them (often a
1064 "chosen" property after making some computation). Objects get closed when
1065 going out of scope as usual.
1066 @param trace optimizer trace
1067 @param object_level0 name of the outer Opt_trace_object C++ object
1068 @param object_level1 name of the inner Opt_trace_object C++ object
1069 @param select_number number of the being-transformed Query_block
1070 @param from description of the before-transformation state
1071 @param to description of the after-transformation state
1073#define OPT_TRACE_TRANSFORM(trace, object_level0, object_level1, \
1074 select_number, from, to) \
1075 const Opt_trace_object object_level0(trace); \
1076 Opt_trace_object object_level1(trace, "transformation"); \
1077 object_level1.add_select_number(select_number); \
1078 object_level1.add_alnum("from", from).add_alnum("to", to);
1080#endif /* OPT_TRACE_INCLUDED */
Kerberos Client Authentication nullptr
Used to store optimizer cost estimates.
Definition: handler.h:3840
Base class that is used to represent any kind of expression in a relational query.
Definition: item.h:933
Definition: sql_list.h:434
A JSON array (ordered set of values).
Definition: opt_trace.h:831
Opt_trace_array(Opt_trace_context *ctx, const char *key, Opt_trace_context::feature_value feature=Opt_trace_context::MISC)
Constructs an array.
Definition: opt_trace.h:840
Opt_trace_array(Opt_trace_context *ctx, Opt_trace_context::feature_value feature=Opt_trace_context::MISC)
Constructs an array.
Definition: opt_trace.h:850
A per-session context which is always available at any point of execution, because in practice it's a...
Definition: opt_trace_context.h:91
void disable_I_S_for_this_and_children()
Temporarily disables I_S for this trace and its children.
Definition: opt_trace_context.h:258
Features' numeric values for @@optimizer_trace_features variable.
Definition: opt_trace_context.h:202
Anything unclassified, including the top object (thus, by "inheritance from parent",...
Definition: opt_trace_context.h:217
bool is_started() const
Returns whether there is a current trace.
Definition: opt_trace_context.h:145
void restore_I_S()
Restores I_S support to what it was before the previous call to disable_I_S_for_this_and_children().
Definition: opt_trace_context.h:267
Instantiate an instance of this class for specific cases where optimizer trace, in a certain section ...
Definition: opt_trace.h:863
Opt_trace_disable_I_S(const Opt_trace_disable_I_S &)
Destructor. Restores trace's "enabled" property to its previous value.
Definition: opt_trace.h:905
Opt_trace_disable_I_S(Opt_trace_context *ctx_arg, bool disable_arg)
Definition: opt_trace.h:896
Opt_trace_disable_I_S & operator=(const Opt_trace_disable_I_S &)
Opt_trace_context * ctx
Definition: opt_trace.h:911
Iterator over the list of remembered traces.
Definition: opt_trace.h:406
Opt_trace_iterator(Opt_trace_context *ctx)
bool at_end() const
Definition: opt_trace.h:427
void get_value(Opt_trace_info *info) const
Provides information about the trace on which the iterator is positioned.
const Opt_trace_stmt * cursor
trace which the iterator is positioned on
Definition: opt_trace.h:432
Opt_trace_context * ctx
Pointer to context, from which traces are retrieved.
Definition: opt_trace.h:431
void next()
Advances iterator to next trace.
long row_count
how many traces retrieved so far
Definition: opt_trace.h:433
A JSON object (unordered set of key/value pairs).
Definition: opt_trace.h:801
Opt_trace_object(Opt_trace_context *ctx, Opt_trace_context::feature_value feature=Opt_trace_context::MISC)
Constructs an object.
Definition: opt_trace.h:820
Opt_trace_object(Opt_trace_context *ctx, const char *key, Opt_trace_context::feature_value feature=Opt_trace_context::MISC)
Constructs an object.
Definition: opt_trace.h:810
Definition: opt_trace.h:923
bool error
whether trace start() had an error
Definition: opt_trace.h:957
Opt_trace_context *const ctx
Definition: opt_trace.h:956
Opt_trace_start(THD *thd_arg, Table_ref *tbl, enum enum_sql_command sql_command, List< set_var_base > *set_vars, const char *query, size_t query_length, sp_printable *instr, const CHARSET_INFO *query_charset)
Instantiate this class to start tracing a THD's actions (generally at a statement's start),...
The trace of one statement.
Object and array are both "structured data" and have lots in common, so the Opt_trace_struct is a bas...
Definition: opt_trace.h:445
Opt_trace_struct & add_null(const char *key)
Adds a JSON null object (==Python's "None")
Definition: opt_trace.h:644
Opt_trace_struct & add(const Item *item)
Definition: opt_trace.h:583
Opt_trace_struct & add_select_number(uint select_number)
Helper to put the number of query_block in an object.
Definition: opt_trace.h:660
Opt_trace_struct & do_add_utf8_table(const Table_ref *tab)
char previous_key[25]
Fixed-length prefix of previous key in this structure, if this structure is an object.
Definition: opt_trace.h:792
Opt_trace_struct & add(ulonglong value)
Definition: opt_trace.h:631
const char * saved_key
Key if the structure is the value of a key/value pair, NULL otherwise.
Definition: opt_trace.h:785
Definition: opt_trace.h:478
void do_construct(Opt_trace_context *ctx, bool requires_key, const char *key, Opt_trace_context::feature_value feature)
Full initialization.
void end()
The exception to RAII: this function is an explicit way of ending a structure before it goes out of s...
Definition: opt_trace.h:488
Opt_trace_struct & add(ulong value)
Definition: opt_trace.h:615
void do_destruct()
Really does destruction.
Opt_trace_struct & add(int value)
Definition: opt_trace.h:599
bool started
Whether the structure does tracing or is dummy.
Definition: opt_trace.h:760
Opt_trace_struct & add(const char *key, longlong value)
Definition: opt_trace.h:619
Opt_trace_struct & add(const char *key, const Item *item)
Add a value (of Item type) to the structure.
Definition: opt_trace.h:579
Opt_trace_stmt * stmt
Trace owning the structure.
Definition: opt_trace.h:783
bool requires_key
Whether the structure requires/forbids keys for values inside it.
Definition: opt_trace.h:774
Opt_trace_struct & add(longlong value)
Definition: opt_trace.h:623
bool set_not_empty()
Informs this structure that we are adding data (scalars, structures) to it.
Definition: opt_trace.h:681
Opt_trace_struct & do_add(const char *key, const char *value, size_t val_length, bool escape)
Really adds to the object.
Opt_trace_struct & add(double value)
Definition: opt_trace.h:639
Opt_trace_struct & add_alnum(const char *key, const char *value)
Adds a value (of string type) to the structure.
Definition: opt_trace.h:523
Opt_trace_struct & add(const char *key, uint value)
Definition: opt_trace.h:603
Opt_trace_struct & add(const char *key, double value)
Definition: opt_trace.h:635
Opt_trace_struct & operator=(const Opt_trace_struct &)
not defined
const char * check_key(const char *key)
Validates the key about to be added.
Opt_trace_struct & add_utf8(const char *value)
Variant of add_utf8() where 'value' is 0-terminated.
Definition: opt_trace.h:565
Opt_trace_struct & add(const char *key, ulong value)
Definition: opt_trace.h:611
Opt_trace_struct & add_utf8(const char *key, const char *value, size_t val_length)
Like add_alnum() but supports any UTF8 characters in 'value'.
Definition: opt_trace.h:546
Opt_trace_struct(Opt_trace_context *ctx_arg, bool requires_key_arg, const char *key, Opt_trace_context::feature_value feature)
Definition: opt_trace.h:460
bool has_disabled_I_S
Whether this structure caused tracing to be disabled in this statement because belonging to a not-tra...
Definition: opt_trace.h:781
Opt_trace_struct & add(bool value)
Definition: opt_trace.h:591
Opt_trace_struct & add_utf8(const char *key, const char *value)
Variant of add_utf8() where 'value' is 0-terminated.
Definition: opt_trace.h:559
Opt_trace_struct & add(uint value)
Definition: opt_trace.h:607
Opt_trace_struct & add_utf8(const char *value, size_t val_length)
Variant of add_utf8() for adding to an array (no key)
Definition: opt_trace.h:553
Opt_trace_struct(const Opt_trace_struct &)
not defined
Opt_trace_struct & add(const char *key, const Cost_estimate &cost)
Add a value to the structure.
Definition: opt_trace.h:669
Opt_trace_struct & do_add_null(const char *key)
Opt_trace_struct & add(const char *key, const char *value)
Not implemented, use add_alnum() instead.
bool empty
Definition: opt_trace.h:782
Opt_trace_struct & add(const char *key, bool value)
Definition: opt_trace.h:587
Opt_trace_struct & add(const char *key)
Opt_trace_struct & add_alnum(const char *value)
Adds a value (of string type) to the structure.
Definition: opt_trace.h:534
Opt_trace_struct & add(const char *key, ulonglong value)
Definition: opt_trace.h:627
Opt_trace_struct & add_utf8_table(const Table_ref *tab)
Helper to put the database/table name in an object.
Definition: opt_trace.h:652
Opt_trace_struct & add(const char *key, int value)
Definition: opt_trace.h:595
This class represents a query block, aka a query specification, which is a query consisting of a SELE...
Definition: sql_lex.h:1161
Query_block * query_block() const override
The query_block which holds the ORDER BY and LIMIT information for this set operation.
Definition: sql_lex.h:1177
For each client connection we create a separate thread with THD serving as a thread/connection descri...
Definition: sql_lexer_thd.h:35
Definition: table.h:2853
A base class for everything that can be set with SET command.
Definition: set_var.h:949
sp_head represents one instance of a stored program.
Definition: sp_head.h:382
sp_printable defines an interface which should be implemented if a class wants report some internal i...
Definition: sp_instr.h:66
Header for compiler-dependent features.
constexpr bool likely(bool expr)
Definition: my_compiler.h:56
constexpr bool unlikely(bool expr)
Definition: my_compiler.h:57
Some integer typedefs for easier portability.
unsigned long long int ulonglong
Definition: my_inttypes.h:55
long long int longlong
Definition: my_inttypes.h:54
Definition: my_sqlcommand.h:45
static char * query
static std::string escape(const std::string &str)
Escapes (only) apostrophes.
void opt_trace_disable_if_no_stored_proc_func_access(THD *thd, sp_head *sp)
If tracing is on, checks additional privileges on a stored routine, to make sure that the user has th...
void opt_trace_disable_if_no_view_access(THD *thd, Table_ref *view, Table_ref *underlying_tables)
If tracing is on, checks additional privileges for a view, to make sure that the user has the right t...
void opt_trace_print_expanded_query(const THD *thd, Query_block *query_block, Opt_trace_object *trace_object)
Prints SELECT query to optimizer trace.
int fill_optimizer_trace_info(THD *thd, Table_ref *tables, Item *)
Fills information_schema.OPTIMIZER_TRACE with rows (one per trace)
void opt_trace_disable_if_no_security_context_access(THD *thd)
If the security context is not that of the connected user, inform the trace system that a privilege i...
This contains the declaration of class Opt_trace_context, which is needed to declare THD.
required string key
Definition: replication_asynchronous_connection_failover.proto:59
Definition: m_ctype.h:422
User-visible information about a trace.
Definition: opt_trace.h:376
size_t missing_bytes
How many bytes this trace is missing (for traces which were truncated because of @@optimizer-trace-ma...
Definition: opt_trace.h:397
size_t trace_length
length of trace string / String containing original query. 0-termination: like trace_ptr.
Definition: opt_trace.h:388
const CHARSET_INFO * query_charset
charset of query string
Definition: opt_trace.h:392
const char * query_ptr
Definition: opt_trace.h:390
const char * trace_ptr
String containing trace.
Definition: opt_trace.h:387
bool missing_priv
whether user lacks privilege to see this trace
Definition: opt_trace.h:398
size_t query_length
length of query string
Definition: opt_trace.h:391