MySQL  8.0.27
Source Code Documentation
Go to the documentation of this file.
1 /* Copyright (c) 2011, 2021, Oracle and/or its affiliates.
3  This program is free software; you can redistribute it and/or modify
4  it under the terms of the GNU General Public License, version 2.0,
5  as published by the Free Software Foundation.
7  This program is also distributed with certain software (including
8  but not limited to OpenSSL) that is licensed under separate terms,
9  as designated in a particular file or component or in included license
10  documentation. The authors of MySQL hereby grant you an additional
11  permission to link the program and your derivative works with the
12  separately licensed software that they have included with MySQL.
14  This program is distributed in the hope that it will be useful,
15  but WITHOUT ANY WARRANTY; without even the implied warranty of
17  GNU General Public License, version 2.0, for more details.
19  You should have received a copy of the GNU General Public License
20  along with this program; if not, write to the Free Software
21  Foundation, Inc., 51 Franklin St, Fifth Floor, Boston, MA 02110-1301 USA */
26 #include <limits.h>
27 #include <string.h>
28 #include <sys/types.h>
30 #include "my_compiler.h"
31 #include "my_inttypes.h"
32 #include "my_sqlcommand.h" // enum_sql_command
33 #include "sql/opt_trace_context.h" // Opt_trace_context
35 class Cost_estimate;
36 class Item;
37 class THD;
38 class set_var_base;
39 class sp_head;
40 class sp_printable;
41 struct CHARSET_INFO;
42 struct TABLE_LIST;
43 template <class T>
44 class List;
46 /**
47  @file sql/opt_trace.h
48  API for the Optimizer trace (WL#5257)
49 */
51 /**
52  @page PAGE_OPT_TRACE The Optimizer Trace
54  @section INTRODUCTION Introduction
56  This optimizer trace is aimed at producing output, which is readable by
57  humans and by programs, to aid understanding of decisions and actions taken
58  by the MySQL Optimizer.
60  @section OUTPUT_FORMAT Output format
62  The chosen output format is JSON (JavaScript Object Notation).
63  In JSON there are:
64  @li "objects" (unordered set of key-value pairs); equivalent to Python's
65  dictionary or Perl's associative array or hash or STL's hash_map.
66  @li "arrays" (ordered set of values); equivalent to Python's and Perl's list
67  or STL's vector.
68  @li "values": a value can be a string, number, boolean, null,
69  which we all call "scalars", or be an object, array.
71  For example (explanations after "<<" are not part of output):
72 @verbatim
73  { << start of top object
74  "first_name": "Gustave", << key/value pair (value is string)
75  "last_name": "Eiffel", << key/value pair (value is string)
76  "born": 1832, << key/value pair (value is integer)
77  "contributions_to": [ << key/value pair (value is array)
78  { << 1st item of array is an object (a building)
79  "name": "Eiffel tower",
80  "location": Paris
81  }, << end of 1st item of array
82  {
83  "name": "Liberty statue",
84  "location": "New York"
85  } << end of 2nd item of array
86  ] << end of array
87  } << end of top object
88 @endverbatim
89  For more details, have a look at the syntax at
90  Note that indentation and newlines are superfluous, useful only for
91  human-readability.
92  Note also that there is nothing like a "named object": an object, array or
93  value has no name; but if it is the value of a key/value pair in an
94  enclosing, outer object, then the key can be seen as the inner object's
95  "name".
97  @section USER_ENABLE_TRACING How a user enables/views the trace
99 @verbatim
100  SET SESSION OPTIMIZER_TRACE="enabled=on"; # enable tracing
101  <statement to trace>; # like SELECT, EXPLAIN SELECT, UPDATE, DELETE...
102  SELECT * FROM information_schema.OPTIMIZER_TRACE;
103  [ repeat last two steps at will ]
104  SET SESSION OPTIMIZER_TRACE="enabled=off"; # disable tracing
105 @endverbatim
107  @c SELECT and @c EXPLAIN SELECT produce the same trace. But there are
108  exceptions regarding subqueries because the two commands treat subqueries
109  differently, for example in
110 @verbatim
111  SELECT ... WHERE x IN (subq1) AND y IN (subq2)
112 @endverbatim
113  SELECT terminates after executing the first subquery if the related IN
114  predicate is false, so we won't see @c JOIN::optimize() tracing for subq2;
115  whereas EXPLAIN SELECT analyzes all subqueries (see loop at the end of
116  @c select_describe()).
118  @section USER_SELECT_TRACING_STATEMENTS How a user traces only certain
119 statements
121  When tracing is in force, each SQL statement generates a trace; more
122  exactly, so does any of
123  SELECT,
126  UPDATE/DELETE and their multi-table variants,
127  SET (unless it manipulates @@@@optimizer_trace),
128  DO,
129  DECLARE/CASE/IF/RETURN (stored routines language elements),
130  CALL.
131  If a command above is prepared and executed in separate steps, preparation
132  and execution are separately traced.
133  By default each new trace overwrites the previous trace. Thus, if a
134  statement contains sub-statements (example: invokes stored procedures,
135  stored functions, triggers), the top statement and sub-statements each
136  generate traces, but at the execution's end only the last sub-statement's
137  trace is visible.
138  If the user wants to see the trace of another sub-statement, she/he can
139  enable/disable tracing around the desired sub-statement, but this requires
140  editing the routine's code, which may not be possible. Another solution is
141  to use
142 @verbatim
143  SET optimizer_trace_offset=<OFFSET>, optimizer_trace_limit=<LIMIT>
144 @endverbatim
145  where OFFSET is a signed integer, and LIMIT is a positive integer.
146  The effect of this SET is the following:
148  @li all remembered traces are cleared
150  @li a later SELECT on OPTIMIZER_TRACE returns the first LIMIT traces of
151  the OFFSET oldest remembered traces (if OFFSET >= 0), or the first LIMIT
152  traces of the -OFFSET newest remembered traces (if OFFSET < 0).
154  For example,
155  a combination of OFFSET=-1 and LIMIT=1 will make the last trace be shown (as
156  is default), OFFSET=-2 and LIMIT=1 will make the next-to-last be shown,
157  OFFSET=-5 and LIMIT=5 will make the last five traces be shown. Such negative
158  OFFSET can be useful when one knows that the interesting sub-statements are
159  the few last ones of a stored routine, like this:
160 @verbatim
161  SET optimizer_trace_offset=-5, optimizer_trace_limit=5;
162  CALL stored_routine(); # more than 5 sub-statements in this routine
163  SELECT * FROM information_schema.OPTIMIZER_TRACE; # see only last 5 traces
164 @endverbatim
165  On the opposite, a positive OFFSET can be useful when one knows that the
166  interesting sub-statements are the few first ones of a stored routine.
168  The more those two variables are accurately adjusted, the less memory is
169  used. For example, OFFSET=0 and LIMIT=5 will use memory to remember 5
170  traces, so if only the three first are needed, OFFSET=0 and LIMIT=3 is
171  better (tracing stops after the LIMITth trace, so the 4th and 5th trace are
172  not created and don't take up memory). A stored routine may have a loop
173  which executes many sub-statements and thus generates many traces, which
174  would use a lot of memory; proper OFFSET and LIMIT can restrict tracing to
175  one iteration of the loop for example. This also gains speed, as tracing a
176  sub-statement impacts performance.
178  If OFFSET>=0, only LIMIT traces are kept in memory. If OFFSET<0, that is not
179  true: instead, (-OFFSET) traces are kept in memory; indeed even if LIMIT is
180  smaller than (-OFFSET), so excludes the last statement, the last statement
181  must still be traced because it will be inside LIMIT after executing one
182  more statement (remember than OFFSET<0 is counted from the end: the "window"
183  slides as more statements execute).
185  Such memory and speed gains are the reason why optimizer_trace_offset/limit,
186  which are restrictions at the trace producer level, are offered. They are
187  better than using
188 @verbatim
190 @endverbatim
191  which is a restriction on the trace consumer level, which saves almost
192  nothing.
194  @section USER_SELECT_TRACING_FEATURES How a user traces only certain
195  optimizer features
197 @verbatim
198  SET OPTIMIZER_TRACE_FEATURES="feature1=on|off,...";
199 @endverbatim
200  where "feature1" is one optimizer feature. For example "greedy_search": a
201  certain Opt_trace_array at the start of @c
202  Optimize_table_order::choose_table_order() has a flag "GREEDY_SEARCH" passed
203  to its constructor: this means that if the user has turned tracing of greedy
204  search off, this array will not be written to the I_S trace, neither will
205  any children structures. All this disabled "trace chunk" will be replaced by
206  an ellipsis "...".
208  @section DEV_ADDS_TRACING How a developer adds tracing to a function
210  Check @c Opt_trace* usage in @c advance_sj_state():
212 @verbatim
213  Opt_trace_array trace_choices(trace, "semijoin_strategy_choice");
214 @endverbatim
216  This creates an array for key "semijoin_strategy_choice". We are going to
217  list possible semijoin strategy choices.
219 @verbatim
220  Opt_trace_object trace_one_strategy(trace);
221 @endverbatim
223  This creates an object without key (normal, it's in an array). This
224  object will describe one single strategy choice.
226 @verbatim
227  trace_one_strategy.add_alnum("strategy", "FirstMatch");
228 @endverbatim
230  This adds a key/value pair to the just-created object: key is "strategy",
231  value is "FirstMatch". This is the strategy to be described in the
232  just-created object.
234 @verbatim
235  trace_one_strategy.add("cost", *current_read_time).
236  add("records", *current_record_count);
237  trace_one_strategy.add("chosen", (pos->sj_strategy == SJ_OPT_FIRST_MATCH));
238 @endverbatim
240  This adds 3 key/value pairs: cost of strategy, number of records produced
241  by this strategy, and whether this strategy is chosen.
243  After that, there is similar code for other semijoin strategies.
245  The resulting trace piece (seen in @c information_schema.OPTIMIZER_TRACE) is
246 @verbatim
247  "semijoin_strategy_choice": [
248  {
249  "strategy": "FirstMatch",
250  "cost": 1,
251  "records": 1,
252  "chosen": true
253  },
254  {
255  "strategy": "DuplicatesWeedout",
256  "cost": 1.1,
257  "records": 1,
258  "duplicate_tables_left": false,
259  "chosen": false
260  }
261  ]
262 @endverbatim
264  For more output examples, check result files of the opt_trace suite in
265  @c mysql-test.
267  Feature can be un-compiled with @code cmake -DOPTIMIZER_TRACE=0 @endcode.
269  @section WITH_DBUG Interaction between trace and DBUG
271  We don't want to have to duplicate code like this:
272 @verbatim
273  DBUG_PRINT("info",("cost %g",cost));
274  Opt_trace_object(thd->opt_trace).add("cost",cost);
275 @endverbatim
277  Thus, any optimizer trace operation, *even* if tracing is run-time disabled,
278  has an implicit DBUG_PRINT("opt",...) inside. This way, only the
279  second line above is needed, and several DBUG_PRINT() could be removed from
280  the Optimizer code.
281  When tracing is run-time disabled, in a debug binary, traces are still
282  created in order to catch the @c add() calls and write their text to DBUG,
283  but those traces are not visible into INFORMATION_SCHEMA.OPTIMIZER_TRACE: we
284  then say that they "don't support I_S".
285  A debug binary without optimizer trace compiled in, will intentionally not
286  compile.
288  Because opening an object or array, or add()-ing to it, writes to DBUG
289  immediately, a key/value pair and its outer object may be 100 lines
290  apart in the DBUG log.
292  @section ADDING_TRACING Guidelines for adding tracing
294  @li Try to limit the number of distinct "words". For example, when
295  describing an optimizer's decision, the words "chosen" (true/false value,
296  tells whether we are choosing the said optimization), "cause" (free text
297  value, tells why we are making this choice, when it's not obvious)
298  can and should often be used. Having a restricted vocabulary helps
299  consistency. Use "row" instead of "record". Use "tmp" instead of
300  "temporary".
302  @li Use only simple characters for key names: a-ZA-Z_#, and no space. '#'
303  serves to denote a number, like in "select#" .
305  @li Keep in mind than in an object, keys are not ordered; an application may
306  parse the JSON output and output it again with keys order changed; thus
307  when order matters, use an array (which may imply having anonymous objects
308  as items of the array, with keys inside the anonymous objects, see how it's
309  done in @c JOIN::optimize()). Keep in mind that in an object keys should
310  be unique, an application may lose duplicate keys.
312  @section OOM_HANDLING Handling of "out-of-memory" errors
314  All memory allocations (with exceptions: see below) in the Optimizer trace
315  use @c my_error() to report errors, which itself calls @c
316  error_handler_hook. It is the responsibility of the API user to set up a
317  proper @c error_handler_hook which will alert her/him of the OOM
318  problem. When in the server, this is already the case (@c error_handler_hook
319  is @c my_message_sql() which makes the statement fail).
320  Note that the debug binary may crash if OOM (OOM can cause syntax
321  errors...).
323  @section TRACE_SECURITY Description of trace-induced security checks.
325  A trace exposes information. For example if one does SELECT on a view, the
326  trace contains the view's body. So, the user should be allowed to see the
327  trace only if she/he has privilege to see the body, i.e. privilege to do
329  There are similar issues with stored procedures, functions, triggers.
331  We implement this by doing additional checks on SQL objects when tracing is
332  on:
333  @li stored procedures, functions, triggers: checks are done when executing
334  those objects
335  @li base tables and views.
337  Base tables or views are listed in some @c LEX::query_tables.
338  The LEX may be of the executing statement (statement executed by
339  @c mysql_execute_command(), or by
340  @c sp_lex_keeper::reset_lex_and_exec_core()), we check this LEX in the
341  constructor of Opt_trace_start.
342  Or it may be a LEX describing a view, we check this LEX when
343  opening the view (@c open_and_read_view()).
345  Those checks are greatly simplified by disabling traces in case of security
346  context changes. @see opt_trace_disable_if_no_security_context_access().
348  Those checks must be done with the security context of the connected
349  user. Checks with the SUID context would be useless: assume the design is
350  that the basic user does not have DML privileges on tables, but only
351  EXECUTE on SUID-highly-privileged routines (which implement _controlled_
352  _approved_ DMLs): then the SUID context would successfully pass all
353  additional privilege checks, routine would generate tracing, and the
354  connected user would view the trace after the routine's execution, seeing
355  secret information.
357  @section NEXT What a developer should read next
359  The documentation of those classes, in order
360 @code
361  Opt_trace_context
362  Opt_trace_context_impl
363  Opt_trace_stmt
364  Opt_trace_struct
365  Opt_trace_object
366  Opt_trace_array
367 @endcode
368  and then @ref opt_trace.h as a whole.
369 */
371 class Opt_trace_stmt; // implementation detail local to
373 /**
374  User-visible information about a trace. @sa Opt_trace_iterator.
375 */
377  /**
378  String containing trace.
379  If trace has been end()ed, this is 0-terminated, which is only to aid
380  debugging or unit testing; this property is not relied upon in normal
381  server usage.
382  If trace has not been ended, this is not 0-terminated. That rare case can
383  happen when a substatement reads OPTIMIZER_TRACE (at that stage, the top
384  statement is still executing so its trace is not ended yet, but may still
385  be read by the sub-statement).
386  */
387  const char *trace_ptr;
388  size_t trace_length; ///< length of trace string
389  //// String containing original query. 0-termination: like trace_ptr.
390  const char *query_ptr;
391  size_t query_length; ///< length of query string
392  const CHARSET_INFO *query_charset; ///< charset of query string
393  /**
394  How many bytes this trace is missing (for traces which were truncated
395  because of @@@@optimizer-trace-max-mem-size).
396  */
398  bool missing_priv; ///< whether user lacks privilege to see this trace
399 };
401 /**
402  Iterator over the list of remembered traces.
403  @note due to implementation, the list must not change during an
404  iterator's lifetime, or results may be unexpected (no crash though).
405 */
407  public:
408  /**
409  @param ctx context
410  */
413  void next(); ///< Advances iterator to next trace.
415  /**
416  Provides information about the trace on which the iterator is
417  positioned.
418  @param[out] info information returned.
419  The usage pattern is
420  1) instantiate the iterator
421  2) test at_end(), if false: call get_value() and then next()
422  3) repeat (2) until at_end() is true.
423  */
424  void get_value(Opt_trace_info *info) const;
426  /// @returns whether iterator is positioned to the end.
427  bool at_end() const { return cursor == nullptr; }
429  private:
430  /// Pointer to context, from which traces are retrieved
432  const Opt_trace_stmt *cursor; ///< trace which the iterator is positioned on
433  long row_count; ///< how many traces retrieved so far
434 };
436 /**
437  Object and array are both "structured data" and have lots in common, so the
438  Opt_trace_struct is a base class for them.
439  When you want to add a structure to the trace, you create an instance of
440  Opt_trace_object or Opt_trace_array, then you add information to it with
441  add(), then the destructor closes the structure (we use RAII, Resource
442  Acquisition Is Initialization).
443 */
446  protected:
447  /**
448  @param ctx_arg Optimizer trace context for this structure
449  @param requires_key_arg whether this structure requires/forbids keys
450  for values put inside it (an object requires them, an
451  array forbids them)
452  @param key key if this structure is the value of a key/value pair,
453  NULL otherwise. This pointer must remain constant and
454  valid until the object is destroyed (to support
455  @ref saved_key).
456  @param feature optimizer feature to which this structure belongs
458  This constructor is never called directly, only from subclasses.
459  */
460  Opt_trace_struct(Opt_trace_context *ctx_arg, bool requires_key_arg,
461  const char *key, Opt_trace_context::feature_value feature)
462  : started(false) {
463  // A first inlined test
464  if (unlikely(ctx_arg->is_started())) {
465  // Tracing enabled: must fully initialize the structure.
466  do_construct(ctx_arg, requires_key_arg, key, feature);
467  }
468  /*
469  Otherwise, just leave "started" to false, it marks that the structure is
470  dummy.
471  */
472  }
474  if (unlikely(started)) do_destruct();
475  }
477  public:
478  /**
479  The exception to RAII: this function is an explicit way of ending a
480  structure before it goes out of scope. Don't use it unless RAII mandates
481  a new scope which mandates re-indenting lots of code lines.
482  */
483  void end() {
484  if (unlikely(started)) do_destruct();
485  }
487  /**
488  Adds a value (of string type) to the structure. A key is specified, so it
489  adds the key/value pair (the structure must thus be an object).
491  There are two "add_*" variants to add a string value.
492  If the value is 0-terminated and each character
493  - is ASCII 7-bit
494  - has ASCII code >=32 and is neither '"' nor '\\'
495  then add_alnum() should be used. That should be the case for all fixed
496  strings like add_alnum("cause", "cost").
497  Otherwise, add_utf8() should be used; it accepts any UTF8-encoded
498  character in 'value' and will escape characters which JSON requires (and
499  is thus slower than add_alnum()). It should be used for all strings which
500  we get from the server's objects (indeed a table's name, a WHERE
501  condition, may contain "strange" characters).
503  @param key key
504  @param value value
505  @returns a reference to the structure, useful for chaining like this:
506  @verbatim add(x,y).add(z,t).add(u,v) @endverbatim
508  String-related add() variants are named add_[something]():
509  - to avoid confusing the compiler between:
510  add(const char *value, size_t val_length) and
511  add(const char *key, ulonglong value)
512  - and because String::length() returns uint32 and not size_t, so for
513  add(str.ptr(), str.length())
514  compiler may pick
515  add(const char *key, double value) instead of
516  add(const char *value, size_t val_length).
517  */
518  Opt_trace_struct &add_alnum(const char *key, const char *value) {
519  if (likely(!started)) return *this;
520  return do_add(key, value, strlen(value), false);
521  }
523  /**
524  Adds a value (of string type) to the structure. No key is specified, so
525  it adds only the value (the structure must thus be an array).
526  @param value value
527  @returns a reference to the structure
528  */
530  if (likely(!started)) return *this;
531  return do_add(nullptr, value, strlen(value), false);
532  }
534  /**
535  Like add_alnum() but supports any UTF8 characters in 'value'.
536  Will "escape" 'value' to be JSON-compliant.
537  @param key key
538  @param value value
539  @param val_length length of string 'value'
540  */
541  Opt_trace_struct &add_utf8(const char *key, const char *value,
542  size_t val_length) {
543  if (likely(!started)) return *this;
544  return do_add(key, value, val_length, true);
545  }
547  /// Variant of add_utf8() for adding to an array (no key)
548  Opt_trace_struct &add_utf8(const char *value, size_t val_length) {
549  if (likely(!started)) return *this;
550  return do_add(nullptr, value, val_length, true);
551  }
553  /// Variant of add_utf8() where 'value' is 0-terminated
554  Opt_trace_struct &add_utf8(const char *key, const char *value) {
555  if (likely(!started)) return *this;
556  return do_add(key, value, strlen(value), true);
557  }
559  /// Variant of add_utf8() where 'value' is 0-terminated
561  if (likely(!started)) return *this;
562  return do_add(nullptr, value, strlen(value), true);
563  }
565  /**
566  Add a value (of Item type) to the structure. The Item should be a
567  condition (like a WHERE clause) which will be pretty-printed into the
568  trace. This is useful for showing condition transformations (equality
569  propagation etc).
570  @param key key
571  @param item the Item
572  @return a reference to the structure
573  */
574  Opt_trace_struct &add(const char *key, Item *item) {
575  if (likely(!started)) return *this;
576  return do_add(key, item);
577  }
579  if (likely(!started)) return *this;
580  return do_add(nullptr, item);
581  }
583  public:
584  Opt_trace_struct &add(const char *key, bool value) {
585  if (likely(!started)) return *this;
586  return do_add(key, value);
587  }
589  if (likely(!started)) return *this;
590  return do_add(nullptr, value);
591  }
592  Opt_trace_struct &add(const char *key, int value) {
593  if (likely(!started)) return *this;
594  return do_add(key, static_cast<longlong>(value));
595  }
597  if (likely(!started)) return *this;
598  return do_add(nullptr, static_cast<longlong>(value));
599  }
600  Opt_trace_struct &add(const char *key, uint value) {
601  if (likely(!started)) return *this;
602  return do_add(key, static_cast<ulonglong>(value));
603  }
605  if (likely(!started)) return *this;
606  return do_add(nullptr, static_cast<ulonglong>(value));
607  }
608  Opt_trace_struct &add(const char *key, ulong value) {
609  if (likely(!started)) return *this;
610  return do_add(key, static_cast<ulonglong>(value));
611  }
613  if (likely(!started)) return *this;
614  return do_add(nullptr, static_cast<ulonglong>(value));
615  }
617  if (likely(!started)) return *this;
618  return do_add(key, value);
619  }
621  if (likely(!started)) return *this;
622  return do_add(nullptr, value);
623  }
625  if (likely(!started)) return *this;
626  return do_add(key, value);
627  }
629  if (likely(!started)) return *this;
630  return do_add(nullptr, value);
631  }
632  Opt_trace_struct &add(const char *key, double value) {
633  if (likely(!started)) return *this;
634  return do_add(key, value);
635  }
637  if (likely(!started)) return *this;
638  return do_add(nullptr, value);
639  }
640  /// Adds a JSON null object (==Python's "None")
641  Opt_trace_struct &add_null(const char *key) {
642  if (likely(!started)) return *this;
643  return do_add_null(key);
644  }
645  /**
646  Helper to put the database/table name in an object.
647  @param tab TABLE* pointer
648  */
650  if (likely(!started)) return *this;
651  return do_add_utf8_table(tab);
652  }
653  /**
654  Helper to put the number of query_block in an object.
655  @param select_number number of query_block
656  */
658  return unlikely(select_number >= INT_MAX) ?
659  // Clearer than any huge number.
660  add_alnum("select#", "fake")
661  : add("select#", select_number);
662  }
663  /**
664  Add a value to the structure.
665  @param key key
666  @param cost the value of Cost_estimate
667  @return a reference to the structure
668  */
669  Opt_trace_struct &add(const char *key, const Cost_estimate &cost) {
670  if (likely(!started)) return *this;
671  return do_add(key, cost);
672  }
674  /**
675  Informs this structure that we are adding data (scalars, structures) to
676  it.
677  This is used only if sending to I_S.
678  @returns whether the structure was empty so far.
679  @note this is reserved for use by Opt_trace_stmt.
680  */
681  bool set_not_empty() {
682  const bool old_empty = empty;
683  empty = false;
684  return old_empty;
685  }
686  /**
687  Validates the key about to be added.
688  @note this is reserved for use by Opt_trace_stmt.
690  When adding a value (or array or object) to an array, or a key/value pair
691  to an object, we need to know this outer array or object.
693  It would be possible, when trying to add a key to an array, which is wrong
694  in JSON, or similarly when trying to add a value without any key to an
695  object, to catch it at compilation time, if the adder received, as
696  function parameter, the type of the structure (like @c
697  Opt_trace_array*). Then the @c add(key,val) call would not compile as
698  Opt_trace_array wouldn't feature it.
700  But as explained in comment of class Opt_trace_context we
701  cannot pass down the object, have to maintain a "current object or
702  array" in the Opt_trace_context context (pointer to an instance of
703  Opt_trace_struct), and the adder grabs it from the context.
705  As this current structure is of type "object or array", we cannot do
706  compile-time checks that only suitable functions are used. A call to @c
707  add(key,value) is necessarily legal for the compiler as the structure may
708  be an object, though it will be wrong in case the structure is actually
709  an array at run-time. Thus we have the risk of an untested particular
710  situation where the current structure is not an object (but an array)
711  though the code expected it to be one. This happens in practice, because
712  subqueries are evaluated in many possible places of code, not all of them
713  being known. Same happens, to a lesser extent, with calls to the range
714  optimizer.
715  So at run-time, in check_key(), we detect wrong usage, like adding a value
716  to an object without specifying a key, and then remove the unnecessary
717  key, or add an autogenerated key.
718  */
719  const char *check_key(const char *key);
721  private:
722  /// Not implemented, use add_alnum() instead.
723  Opt_trace_struct &add(const char *key, const char *value);
724  Opt_trace_struct &add(const char *key);
726  /// Full initialization. @sa Opt_trace_struct::Opt_trace_struct
727  void do_construct(Opt_trace_context *ctx, bool requires_key, const char *key,
729  /// Really does destruction
730  void do_destruct();
731  /**
732  Really adds to the object. @sa add().
734  @note add() has an up-front if(), hopefully inlined, so that in the
735  common case - tracing run-time disabled - we have no function call. If
736  tracing is enabled, we call do_add().
737  In a 20-table plan search (as in BUG#50595), the execution time was
738  decreased from 2.6 to 2.0 seconds thanks to this inlined-if trick.
740  @param key key
741  @param value value
742  @param val_length length of string 'value'
743  @param escape do JSON-compliant escaping of 'value'. If 'escape' is
744  false, 'value' should be ASCII. Otherwise, should be UTF8.
745  */
746  Opt_trace_struct &do_add(const char *key, const char *value,
747  size_t val_length, bool escape);
748  Opt_trace_struct &do_add(const char *key, Item *item);
749  Opt_trace_struct &do_add(const char *key, bool value);
750  Opt_trace_struct &do_add(const char *key, longlong value);
751  Opt_trace_struct &do_add(const char *key, ulonglong value);
752  Opt_trace_struct &do_add(const char *key, double value);
753  Opt_trace_struct &do_add_null(const char *key);
755  Opt_trace_struct &do_add(const char *key, const Cost_estimate &value);
757  Opt_trace_struct(const Opt_trace_struct &); ///< not defined
758  Opt_trace_struct &operator=(const Opt_trace_struct &); ///< not defined
760  bool started; ///< Whether the structure does tracing or is dummy
762  /**
763  Whether the structure requires/forbids keys for values inside it.
764  true: this is an object. false: this is an array.
766  @note The canonical way would be to not have such bool per instance, but
767  rather have a pure virtual member function
768  Opt_trace_struct::requires_key(), overloaded by Opt_trace_object
769  (returning true) and by Opt_trace_array (returning false). But
770  Opt_trace_object::requires_key() would not be accessible from
771  Opt_trace_struct::do_construct() (which would complicate coding), whereas
772  the bool is.
773  */
776  /**
777  Whether this structure caused tracing to be disabled in this statement
778  because belonging to a not-traced optimizer feature, in accordance with
779  the value of @@@@optimizer_trace_features.
780  */
782  bool empty; ///< @see set_not_empty()
783  Opt_trace_stmt *stmt; ///< Trace owning the structure
784  /// Key if the structure is the value of a key/value pair, NULL otherwise
785  const char *saved_key;
786 #ifndef NDEBUG
787  /**
788  Fixed-length prefix of previous key in this structure, if this structure
789  is an object. Serves to detect when adding two same consecutive keys to
790  an object, which would be wrong.
791  */
792  char previous_key[25];
793 #endif
794 };
796 /**
797  A JSON object (unordered set of key/value pairs).
798  Defines only a constructor, all the rest is inherited from
799  Opt_trace_struct.
800 */
802  public:
803  /**
804  Constructs an object. Key is specified, so the object is the value of a
805  key/value pair.
806  @param ctx context for this object
807  @param key key
808  @param feature optimizer feature to which this structure belongs
809  */
811  Opt_trace_context *ctx, const char *key,
813  : Opt_trace_struct(ctx, true, key, feature) {}
814  /**
815  Constructs an object. No key is specified, so the object is just a value
816  (serves for the single root object or for adding the object to an array).
817  @param ctx context for this object
818  @param feature optimizer feature to which this structure belongs
819  */
821  Opt_trace_context *ctx,
823  : Opt_trace_struct(ctx, true, nullptr, feature) {}
824 };
826 /**
827  A JSON array (ordered set of values).
828  Defines only a constructor, all the rest in inherited from
829  Opt_trace_struct.
830 */
832  public:
833  /**
834  Constructs an array. Key is specified, so the array is the value of a
835  key/value pair.
836  @param ctx context for this array
837  @param key key
838  @param feature optimizer feature to which this structure belongs
839  */
841  Opt_trace_context *ctx, const char *key,
843  : Opt_trace_struct(ctx, false, key, feature) {}
844  /**
845  Constructs an array. No key is specified, so the array is just a value
846  (serves for adding the object to an array).
847  @param ctx context for this array
848  @param feature optimizer feature to which this structure belongs
849  */
851  Opt_trace_context *ctx,
853  : Opt_trace_struct(ctx, false, nullptr, feature) {}
854 };
856 /**
857  Instantiate an instance of this class for specific cases where
858  optimizer trace, in a certain section of Optimizer code, should write only
859  to DBUG and not I_S. Example: see
860  Note that this class should rarely be used; the "feature" parameter of
861  Opt_trace_struct is a good alternative.
862 */
864  public:
865  /**
866  @param ctx_arg Context.
867  @param disable_arg Whether the instance should really disable
868  anything. If false, the object is dummy. If true,
869  tracing-to-I_S is disabled at construction and
870  re-enabled at destruction.
871  @details A dummy instance is there only for RAII reasons. Imagine we want
872  to do this:
873 @verbatim
874  {
875  if (x) disable tracing;
876  code;
877  } // tracing should be re-enabled here
878 @endverbatim
879  As we use RAII, we cannot put the instance declaration inside if(x):
880 @verbatim
881  {
882  if (x) Opt_trace_disable_I_S instance(ctx);
883  code;
884  }
885 @endverbatim
886  because it would be destroyed as soon as the if() block is left, so
887  tracing would be re-enabled before @c code;. It should rather be written
888  as:
889 @verbatim
890  {
891  Opt_trace_disable_I_S instance(ctx, x); // if !x, does nothing
892  code;
893  } // re-enabling happens here, if x is true
894 @endverbatim
895  */
896  Opt_trace_disable_I_S(Opt_trace_context *ctx_arg, bool disable_arg) {
897  if (disable_arg) {
898  ctx = ctx_arg;
900  } else
901  ctx = nullptr;
902  }
904  /// Destructor. Restores trace's "enabled" property to its previous value.
906  if (ctx != nullptr) ctx->restore_I_S();
907  }
909  private:
910  /** Context. Non-NULL if and only if this instance really does disabling */
914  const Opt_trace_disable_I_S &); // not defined
915 };
917 /**
918  @name Helpers connecting the optimizer trace to THD or Information Schema.
919 */
921 //@{
924  public:
925  /**
926  Instantiate this class to start tracing a THD's actions (generally at a
927  statement's start), and to set the "original" query (not transformed, as
928  sent by client) for the new trace. Destructor will end the trace.
930  If in a routine's instruction, there is no "query". To be helpful to the
931  user, we craft a query using the instruction's print(). We don't leave this
932  to the caller, because it would be inefficient if tracing is off.
934  @param thd_arg the THD
935  @param tbl list of tables read/written by the statement.
936  @param sql_command SQL command being prepared or executed
937  @param set_vars what variables are set by this command (only used if
938  sql_command is SQLCOM_SET_OPTION)
939  @param query query
940  @param query_length query's length
941  @param instr routine's instruction, if applicable; if so, 'query'
942  and 'query_length' above are ignored
943  @param query_charset charset which was used to encode this query
945  @note Each sub-statement is responsible for ending the trace which it
946  has started; this class achieves this by keeping some memory inside.
947  */
948  Opt_trace_start(THD *thd_arg, TABLE_LIST *tbl,
949  enum enum_sql_command sql_command,
950  List<set_var_base> *set_vars, const char *query,
951  size_t query_length, sp_printable *instr,
952  const CHARSET_INFO *query_charset);
955  private:
957  bool error; ///< whether trace start() had an error
958 };
960 class Query_block;
962 /**
963  Prints SELECT query to optimizer trace. It is not the original query (as in
964  @c Opt_trace_context::set_query()) but a printout of the parse tree
965  (Item-s).
966  @param thd the THD
967  @param query_block query's parse tree
968  @param trace_object Opt_trace_object to which the query will be added
969 */
970 void opt_trace_print_expanded_query(const THD *thd, Query_block *query_block,
971  Opt_trace_object *trace_object);
973 /**
974  If the security context is not that of the connected user, inform the trace
975  system that a privilege is missing. With one exception: see below.
977  @param thd the THD
979  This serves to eliminate the following issue.
980  Any information readable by a SELECT may theoretically end up in
981  the trace. And a SELECT may read information from other places than tables:
982  - from views (reading their bodies)
983  - from stored routines (reading their bodies)
984  - from files (reading their content), with LOAD_FILE()
985  - from the list of connections (reading their queries...), with
987  If the connected user has EXECUTE privilege on a routine which does a
988  security context change, the routine can retrieve information internally
989  (if allowed by the SUID context's privileges), and present only a portion
990  of it to the connected user. But with tracing on, all information is
991  possibly in the trace. So the connected user receives more information than
992  the routine's definer intended to provide. Fixing this issue would require
993  adding, near many privilege checks in the server, a new
994  optimizer-trace-specific check done against the connected user's context,
995  to verify that the connected user has the right to see the retrieved
996  information.
998  Instead, our chosen simpler solution is that if we see a security context
999  change where SUID user is not the connected user, we disable tracing. With
1000  only one safe exception: if the connected user has all global privileges
1001  (because then she/he can find any information anyway). By "all global
1002  privileges" we mean everything but WITH GRANT OPTION (that latter one isn't
1003  related to information gathering).
1005  Read access to I_S.OPTIMIZER_TRACE by another user than the connected user
1006  is restricted: @see fill_optimizer_trace_info().
1007 */
1010 /**
1011  If tracing is on, checks additional privileges for a view, to make sure
1012  that the user has the right to do SHOW CREATE VIEW. For that:
1013  - this function checks SHOW VIEW
1014  - SELECT is tested in opt_trace_disable_if_no_tables_access()
1015  - SELECT + SHOW VIEW is sufficient for SHOW CREATE VIEW.
1016  We also check underlying tables.
1017  If a privilege is missing, notifies the trace system.
1018  This function should be called when the view's underlying tables have not
1019  yet been merged.
1021  @param thd THD context
1022  @param view view to check
1023  @param underlying_tables underlying tables/views of 'view'
1024  */
1026  TABLE_LIST *underlying_tables);
1028 /**
1029  If tracing is on, checks additional privileges on a stored routine, to make
1030  sure that the user has the right to do SHOW CREATE PROCEDURE/FUNCTION. For
1031  that, we use the same checks as in those SHOW commands.
1032  If a privilege is missing, notifies the trace system.
1034  This function is not redundant with
1035  opt_trace_disable_if_no_security_context_access().
1036  Indeed, for a SQL SECURITY INVOKER routine, there is no context change, but
1037  we must still verify that the invoker can do SHOW CREATE.
1039  For triggers, see note in sp_head::execute_trigger().
1041  @param thd The THD
1042  @param sp routine to check
1043  */
1046 /**
1047  Fills information_schema.OPTIMIZER_TRACE with rows (one per trace)
1048  @retval 0 ok
1049  @retval 1 error
1050 */
1051 int fill_optimizer_trace_info(THD *thd, TABLE_LIST *tables, Item *);
1053 //@}
1055 /**
1056  Helper for defining query-transformation-related trace objects in one
1057  code line. This produces
1058  {
1059  "transformation": {
1060  "select#": @<select_number@>,
1061  "from": @<from@>,
1062  "to": @<to@>
1063  The objects are left open, so that one can add more to them (often a
1064  "chosen" property after making some computation). Objects get closed when
1065  going out of scope as usual.
1066  @param trace optimizer trace
1067  @param object_level0 name of the outer Opt_trace_object C++ object
1068  @param object_level1 name of the inner Opt_trace_object C++ object
1069  @param select_number number of the being-transformed Query_block
1070  @param from description of the before-transformation state
1071  @param to description of the after-transformation state
1072 */
1073 #define OPT_TRACE_TRANSFORM(trace, object_level0, object_level1, \
1074  select_number, from, to) \
1075  Opt_trace_object object_level0(trace); \
1076  Opt_trace_object object_level1(trace, "transformation"); \
1077  object_level1.add_select_number(select_number); \
1078  object_level1.add_alnum("from", from).add_alnum("to", to);
1080 #endif /* OPT_TRACE_INCLUDED */
Used to store optimizer cost estimates.
Definition: handler.h:3423
Base class that is used to represent any kind of expression in a relational query.
Definition: item.h:802
Definition: sql_list.h:431
A JSON array (ordered set of values).
Definition: opt_trace.h:831
Opt_trace_array(Opt_trace_context *ctx, const char *key, Opt_trace_context::feature_value feature=Opt_trace_context::MISC)
Constructs an array.
Definition: opt_trace.h:840
Opt_trace_array(Opt_trace_context *ctx, Opt_trace_context::feature_value feature=Opt_trace_context::MISC)
Constructs an array.
Definition: opt_trace.h:850
A per-session context which is always available at any point of execution, because in practice it's a...
Definition: opt_trace_context.h:89
void disable_I_S_for_this_and_children()
Temporarily disables I_S for this trace and its children.
Definition: opt_trace_context.h:256
Features' numeric values for @@optimizer_trace_features variable.
Definition: opt_trace_context.h:200
Anything unclassified, including the top object (thus, by "inheritance from parent",...
Definition: opt_trace_context.h:215
bool is_started() const
Returns whether there is a current trace.
Definition: opt_trace_context.h:143
void restore_I_S()
Restores I_S support to what it was before the previous call to disable_I_S_for_this_and_children().
Definition: opt_trace_context.h:265
Instantiate an instance of this class for specific cases where optimizer trace, in a certain section ...
Definition: opt_trace.h:863
Opt_trace_disable_I_S(const Opt_trace_disable_I_S &)
Destructor. Restores trace's "enabled" property to its previous value.
Definition: opt_trace.h:905
Opt_trace_disable_I_S & operator=(const Opt_trace_disable_I_S &)
Opt_trace_disable_I_S(Opt_trace_context *ctx_arg, bool disable_arg)
Definition: opt_trace.h:896
Opt_trace_context * ctx
Definition: opt_trace.h:911
Iterator over the list of remembered traces.
Definition: opt_trace.h:406
Opt_trace_iterator(Opt_trace_context *ctx)
bool at_end() const
Definition: opt_trace.h:427
void get_value(Opt_trace_info *info) const
Provides information about the trace on which the iterator is positioned.
const Opt_trace_stmt * cursor
trace which the iterator is positioned on
Definition: opt_trace.h:432
Opt_trace_context * ctx
Pointer to context, from which traces are retrieved.
Definition: opt_trace.h:431
void next()
Advances iterator to next trace.
long row_count
how many traces retrieved so far
Definition: opt_trace.h:433
A JSON object (unordered set of key/value pairs).
Definition: opt_trace.h:801
Opt_trace_object(Opt_trace_context *ctx, Opt_trace_context::feature_value feature=Opt_trace_context::MISC)
Constructs an object.
Definition: opt_trace.h:820
Opt_trace_object(Opt_trace_context *ctx, const char *key, Opt_trace_context::feature_value feature=Opt_trace_context::MISC)
Constructs an object.
Definition: opt_trace.h:810
Definition: opt_trace.h:923
Opt_trace_start(THD *thd_arg, TABLE_LIST *tbl, enum enum_sql_command sql_command, List< set_var_base > *set_vars, const char *query, size_t query_length, sp_printable *instr, const CHARSET_INFO *query_charset)
Instantiate this class to start tracing a THD's actions (generally at a statement's start),...
bool error
whether trace start() had an error
Definition: opt_trace.h:957
Opt_trace_context *const ctx
Definition: opt_trace.h:956
The trace of one statement.
Object and array are both "structured data" and have lots in common, so the Opt_trace_struct is a bas...
Definition: opt_trace.h:445
Opt_trace_struct & add(const char *key, const Cost_estimate &cost)
Add a value to the structure.
Definition: opt_trace.h:669
Opt_trace_struct & add(const char *key, longlong value)
Definition: opt_trace.h:616
Opt_trace_struct & add_alnum(const char *key, const char *value)
Adds a value (of string type) to the structure.
Definition: opt_trace.h:518
char previous_key[25]
Fixed-length prefix of previous key in this structure, if this structure is an object.
Definition: opt_trace.h:792
Opt_trace_struct & operator=(const Opt_trace_struct &)
not defined
Opt_trace_struct & add(ulong value)
Definition: opt_trace.h:612
const char * saved_key
Key if the structure is the value of a key/value pair, NULL otherwise.
Definition: opt_trace.h:785
Definition: opt_trace.h:473
Opt_trace_struct & add_utf8(const char *value)
Variant of add_utf8() where 'value' is 0-terminated.
Definition: opt_trace.h:560
Opt_trace_struct & add(longlong value)
Definition: opt_trace.h:620
void do_construct(Opt_trace_context *ctx, bool requires_key, const char *key, Opt_trace_context::feature_value feature)
Full initialization.
void end()
The exception to RAII: this function is an explicit way of ending a structure before it goes out of s...
Definition: opt_trace.h:483
void do_destruct()
Really does destruction.
Opt_trace_struct & add(const char *key, ulong value)
Definition: opt_trace.h:608
Opt_trace_struct & add(uint value)
Definition: opt_trace.h:604
Opt_trace_struct & add(Item *item)
Definition: opt_trace.h:578
Opt_trace_struct & add_utf8(const char *key, const char *value)
Variant of add_utf8() where 'value' is 0-terminated.
Definition: opt_trace.h:554
Opt_trace_struct & add_select_number(uint select_number)
Helper to put the number of query_block in an object.
Definition: opt_trace.h:657
Opt_trace_struct & add(const char *key, double value)
Definition: opt_trace.h:632
bool started
Whether the structure does tracing or is dummy.
Definition: opt_trace.h:760
Opt_trace_struct & do_add_utf8_table(const TABLE_LIST *tab)
Opt_trace_stmt * stmt
Trace owning the structure.
Definition: opt_trace.h:783
Opt_trace_struct & add_alnum(const char *value)
Adds a value (of string type) to the structure.
Definition: opt_trace.h:529
bool requires_key
Whether the structure requires/forbids keys for values inside it.
Definition: opt_trace.h:774
bool set_not_empty()
Informs this structure that we are adding data (scalars, structures) to it.
Definition: opt_trace.h:681
Opt_trace_struct & do_add(const char *key, const char *value, size_t val_length, bool escape)
Really adds to the object.
Opt_trace_struct & add(const char *key, const char *value)
Not implemented, use add_alnum() instead.
Opt_trace_struct & add(const char *key, bool value)
Definition: opt_trace.h:584
Opt_trace_struct & add(const char *key, Item *item)
Add a value (of Item type) to the structure.
Definition: opt_trace.h:574
Opt_trace_struct & add_null(const char *key)
Adds a JSON null object (==Python's "None")
Definition: opt_trace.h:641
Opt_trace_struct & add_utf8(const char *key, const char *value, size_t val_length)
Like add_alnum() but supports any UTF8 characters in 'value'.
Definition: opt_trace.h:541
const char * check_key(const char *key)
Validates the key about to be added.
Opt_trace_struct(Opt_trace_context *ctx_arg, bool requires_key_arg, const char *key, Opt_trace_context::feature_value feature)
Definition: opt_trace.h:460
bool has_disabled_I_S
Whether this structure caused tracing to be disabled in this statement because belonging to a not-tra...
Definition: opt_trace.h:781
Opt_trace_struct & add(int value)
Definition: opt_trace.h:596
Opt_trace_struct & add(const char *key)
Opt_trace_struct & add_utf8_table(const TABLE_LIST *tab)
Helper to put the database/table name in an object.
Definition: opt_trace.h:649
Opt_trace_struct & add(bool value)
Definition: opt_trace.h:588
Opt_trace_struct & add(ulonglong value)
Definition: opt_trace.h:628
Opt_trace_struct & add(const char *key, int value)
Definition: opt_trace.h:592
Opt_trace_struct(const Opt_trace_struct &)
not defined
Opt_trace_struct & add(const char *key, uint value)
Definition: opt_trace.h:600
Opt_trace_struct & add_utf8(const char *value, size_t val_length)
Variant of add_utf8() for adding to an array (no key)
Definition: opt_trace.h:548
Opt_trace_struct & add(const char *key, ulonglong value)
Definition: opt_trace.h:624
Opt_trace_struct & do_add_null(const char *key)
bool empty
Definition: opt_trace.h:782
Opt_trace_struct & add(double value)
Definition: opt_trace.h:636
This class represents a query block, aka a query specification, which is a query consisting of a SELE...
Definition: sql_lex.h:1123
For each client connection we create a separate thread with THD serving as a thread/connection descri...
Definition: sql_class.h:821
A base class for everything that can be set with SET command.
Definition: set_var.h:428
sp_head represents one instance of a stored program.
Definition: sp_head.h:379
sp_printable defines an interface which should be implemented if a class wants report some internal i...
Definition: sp_instr.h:65
Dialog Client Authentication nullptr
Header for compiler-dependent features.
bool unlikely(bool expr)
Definition: my_compiler.h:55
bool likely(bool expr)
Definition: my_compiler.h:54
Some integer typedefs for easier portability.
unsigned long long int ulonglong
Definition: my_inttypes.h:55
long long int longlong
Definition: my_inttypes.h:54
Definition: my_sqlcommand.h:45
static char * query
Log info(cout, "NOTE")
static std::string escape(const std::string &str)
Escapes (only) apostrophes.
void opt_trace_disable_if_no_stored_proc_func_access(THD *thd, sp_head *sp)
If tracing is on, checks additional privileges on a stored routine, to make sure that the user has th...
int fill_optimizer_trace_info(THD *thd, TABLE_LIST *tables, Item *)
Fills information_schema.OPTIMIZER_TRACE with rows (one per trace)
void opt_trace_disable_if_no_view_access(THD *thd, TABLE_LIST *view, TABLE_LIST *underlying_tables)
If tracing is on, checks additional privileges for a view, to make sure that the user has the right t...
void opt_trace_print_expanded_query(const THD *thd, Query_block *query_block, Opt_trace_object *trace_object)
Prints SELECT query to optimizer trace.
void opt_trace_disable_if_no_security_context_access(THD *thd)
If the security context is not that of the connected user, inform the trace system that a privilege i...
This contains the declaration of class Opt_trace_context, which is needed to declare THD.
const string value("\"Value\"")
required string key
Definition: replication_asynchronous_connection_failover.proto:59
Definition: m_ctype.h:354
User-visible information about a trace.
Definition: opt_trace.h:376
size_t missing_bytes
How many bytes this trace is missing (for traces which were truncated because of @@optimizer-trace-ma...
Definition: opt_trace.h:397
size_t trace_length
length of trace string / String containing original query. 0-termination: like trace_ptr.
Definition: opt_trace.h:388
const CHARSET_INFO * query_charset
charset of query string
Definition: opt_trace.h:392
const char * query_ptr
Definition: opt_trace.h:390
const char * trace_ptr
String containing trace.
Definition: opt_trace.h:387
bool missing_priv
whether user lacks privilege to see this trace
Definition: opt_trace.h:398
size_t query_length
length of query string
Definition: opt_trace.h:391
Definition: table.h:2694
unsigned int uint