MySQL  8.0.23
Source Code Documentation
Go to the documentation of this file.
1 /* Copyright (c) 2011, 2020, Oracle and/or its affiliates. All rights reserved.
3  This program is free software; you can redistribute it and/or modify
4  it under the terms of the GNU General Public License, version 2.0,
5  as published by the Free Software Foundation.
7  This program is also distributed with certain software (including
8  but not limited to OpenSSL) that is licensed under separate terms,
9  as designated in a particular file or component or in included license
10  documentation. The authors of MySQL hereby grant you an additional
11  permission to link the program and your derivative works with the
12  separately licensed software that they have included with MySQL.
14  This program is distributed in the hope that it will be useful,
15  but WITHOUT ANY WARRANTY; without even the implied warranty of
17  GNU General Public License, version 2.0, for more details.
19  You should have received a copy of the GNU General Public License
20  along with this program; if not, write to the Free Software
21  Foundation, Inc., 51 Franklin St, Fifth Floor, Boston, MA 02110-1301 USA */
26 #include "my_config.h"
28 #include <limits.h>
29 #include <string.h>
30 #include <sys/types.h>
32 #include "m_ctype.h"
33 #include "my_compiler.h"
34 #include "my_inttypes.h"
35 #include "my_sqlcommand.h" // enum_sql_command
36 #include "sql/opt_trace_context.h" // Opt_trace_context
38 class Cost_estimate;
39 class Item;
40 class THD;
41 class set_var_base;
42 class sp_head;
43 class sp_printable;
44 struct TABLE_LIST;
45 template <class T>
46 class List;
48 /**
49  @file sql/opt_trace.h
50  API for the Optimizer trace (WL#5257)
51 */
53 /**
54  @page PAGE_OPT_TRACE The Optimizer Trace
56  @section INTRODUCTION Introduction
58  This optimizer trace is aimed at producing output, which is readable by
59  humans and by programs, to aid understanding of decisions and actions taken
60  by the MySQL Optimizer.
62  @section OUTPUT_FORMAT Output format
64  The chosen output format is JSON (JavaScript Object Notation).
65  In JSON there are:
66  @li "objects" (unordered set of key-value pairs); equivalent to Python's
67  dictionary or Perl's associative array or hash or STL's hash_map.
68  @li "arrays" (ordered set of values); equivalent to Python's and Perl's list
69  or STL's vector.
70  @li "values": a value can be a string, number, boolean, null,
71  which we all call "scalars", or be an object, array.
73  For example (explanations after "<<" are not part of output):
74 @verbatim
75  { << start of top object
76  "first_name": "Gustave", << key/value pair (value is string)
77  "last_name": "Eiffel", << key/value pair (value is string)
78  "born": 1832, << key/value pair (value is integer)
79  "contributions_to": [ << key/value pair (value is array)
80  { << 1st item of array is an object (a building)
81  "name": "Eiffel tower",
82  "location": Paris
83  }, << end of 1st item of array
84  {
85  "name": "Liberty statue",
86  "location": "New York"
87  } << end of 2nd item of array
88  ] << end of array
89  } << end of top object
90 @endverbatim
91  For more details, have a look at the syntax at
92  Note that indentation and newlines are superfluous, useful only for
93  human-readability.
94  Note also that there is nothing like a "named object": an object, array or
95  value has no name; but if it is the value of a key/value pair in an
96  enclosing, outer object, then the key can be seen as the inner object's
97  "name".
99  @section USER_ENABLE_TRACING How a user enables/views the trace
101 @verbatim
102  SET SESSION OPTIMIZER_TRACE="enabled=on"; # enable tracing
103  <statement to trace>; # like SELECT, EXPLAIN SELECT, UPDATE, DELETE...
104  SELECT * FROM information_schema.OPTIMIZER_TRACE;
105  [ repeat last two steps at will ]
106  SET SESSION OPTIMIZER_TRACE="enabled=off"; # disable tracing
107 @endverbatim
109  @c SELECT and @c EXPLAIN SELECT produce the same trace. But there are
110  exceptions regarding subqueries because the two commands treat subqueries
111  differently, for example in
112 @verbatim
113  SELECT ... WHERE x IN (subq1) AND y IN (subq2)
114 @endverbatim
115  SELECT terminates after executing the first subquery if the related IN
116  predicate is false, so we won't see @c JOIN::optimize() tracing for subq2;
117  whereas EXPLAIN SELECT analyzes all subqueries (see loop at the end of
118  @c select_describe()).
120  @section USER_SELECT_TRACING_STATEMENTS How a user traces only certain
121 statements
123  When tracing is in force, each SQL statement generates a trace; more
124  exactly, so does any of
125  SELECT,
128  UPDATE/DELETE and their multi-table variants,
129  SET (unless it manipulates @@@@optimizer_trace),
130  DO,
131  DECLARE/CASE/IF/RETURN (stored routines language elements),
132  CALL.
133  If a command above is prepared and executed in separate steps, preparation
134  and execution are separately traced.
135  By default each new trace overwrites the previous trace. Thus, if a
136  statement contains sub-statements (example: invokes stored procedures,
137  stored functions, triggers), the top statement and sub-statements each
138  generate traces, but at the execution's end only the last sub-statement's
139  trace is visible.
140  If the user wants to see the trace of another sub-statement, she/he can
141  enable/disable tracing around the desired sub-statement, but this requires
142  editing the routine's code, which may not be possible. Another solution is
143  to use
144 @verbatim
145  SET optimizer_trace_offset=<OFFSET>, optimizer_trace_limit=<LIMIT>
146 @endverbatim
147  where OFFSET is a signed integer, and LIMIT is a positive integer.
148  The effect of this SET is the following:
150  @li all remembered traces are cleared
152  @li a later SELECT on OPTIMIZER_TRACE returns the first LIMIT traces of
153  the OFFSET oldest remembered traces (if OFFSET >= 0), or the first LIMIT
154  traces of the -OFFSET newest remembered traces (if OFFSET < 0).
156  For example,
157  a combination of OFFSET=-1 and LIMIT=1 will make the last trace be shown (as
158  is default), OFFSET=-2 and LIMIT=1 will make the next-to-last be shown,
159  OFFSET=-5 and LIMIT=5 will make the last five traces be shown. Such negative
160  OFFSET can be useful when one knows that the interesting sub-statements are
161  the few last ones of a stored routine, like this:
162 @verbatim
163  SET optimizer_trace_offset=-5, optimizer_trace_limit=5;
164  CALL stored_routine(); # more than 5 sub-statements in this routine
165  SELECT * FROM information_schema.OPTIMIZER_TRACE; # see only last 5 traces
166 @endverbatim
167  On the opposite, a positive OFFSET can be useful when one knows that the
168  interesting sub-statements are the few first ones of a stored routine.
170  The more those two variables are accurately adjusted, the less memory is
171  used. For example, OFFSET=0 and LIMIT=5 will use memory to remember 5
172  traces, so if only the three first are needed, OFFSET=0 and LIMIT=3 is
173  better (tracing stops after the LIMITth trace, so the 4th and 5th trace are
174  not created and don't take up memory). A stored routine may have a loop
175  which executes many sub-statements and thus generates many traces, which
176  would use a lot of memory; proper OFFSET and LIMIT can restrict tracing to
177  one iteration of the loop for example. This also gains speed, as tracing a
178  sub-statement impacts performance.
180  If OFFSET>=0, only LIMIT traces are kept in memory. If OFFSET<0, that is not
181  true: instead, (-OFFSET) traces are kept in memory; indeed even if LIMIT is
182  smaller than (-OFFSET), so excludes the last statement, the last statement
183  must still be traced because it will be inside LIMIT after executing one
184  more statement (remember than OFFSET<0 is counted from the end: the "window"
185  slides as more statements execute).
187  Such memory and speed gains are the reason why optimizer_trace_offset/limit,
188  which are restrictions at the trace producer level, are offered. They are
189  better than using
190 @verbatim
192 @endverbatim
193  which is a restriction on the trace consumer level, which saves almost
194  nothing.
196  @section USER_SELECT_TRACING_FEATURES How a user traces only certain
197  optimizer features
199 @verbatim
200  SET OPTIMIZER_TRACE_FEATURES="feature1=on|off,...";
201 @endverbatim
202  where "feature1" is one optimizer feature. For example "greedy_search": a
203  certain Opt_trace_array at the start of @c
204  Optimize_table_order::choose_table_order() has a flag "GREEDY_SEARCH" passed
205  to its constructor: this means that if the user has turned tracing of greedy
206  search off, this array will not be written to the I_S trace, neither will
207  any children structures. All this disabled "trace chunk" will be replaced by
208  an ellipsis "...".
210  @section DEV_ADDS_TRACING How a developer adds tracing to a function
212  Check @c Opt_trace* usage in @c advance_sj_state():
214 @verbatim
215  Opt_trace_array trace_choices(trace, "semijoin_strategy_choice");
216 @endverbatim
218  This creates an array for key "semijoin_strategy_choice". We are going to
219  list possible semijoin strategy choices.
221 @verbatim
222  Opt_trace_object trace_one_strategy(trace);
223 @endverbatim
225  This creates an object without key (normal, it's in an array). This
226  object will describe one single strategy choice.
228 @verbatim
229  trace_one_strategy.add_alnum("strategy", "FirstMatch");
230 @endverbatim
232  This adds a key/value pair to the just-created object: key is "strategy",
233  value is "FirstMatch". This is the strategy to be described in the
234  just-created object.
236 @verbatim
237  trace_one_strategy.add("cost", *current_read_time).
238  add("records", *current_record_count);
239  trace_one_strategy.add("chosen", (pos->sj_strategy == SJ_OPT_FIRST_MATCH));
240 @endverbatim
242  This adds 3 key/value pairs: cost of strategy, number of records produced
243  by this strategy, and whether this strategy is chosen.
245  After that, there is similar code for other semijoin strategies.
247  The resulting trace piece (seen in @c information_schema.OPTIMIZER_TRACE) is
248 @verbatim
249  "semijoin_strategy_choice": [
250  {
251  "strategy": "FirstMatch",
252  "cost": 1,
253  "records": 1,
254  "chosen": true
255  },
256  {
257  "strategy": "DuplicatesWeedout",
258  "cost": 1.1,
259  "records": 1,
260  "duplicate_tables_left": false,
261  "chosen": false
262  }
263  ]
264 @endverbatim
266  For more output examples, check result files of the opt_trace suite in
267  @c mysql-test.
269  Feature can be un-compiled with @code cmake -DOPTIMIZER_TRACE=0 @endcode.
271  @section WITH_DBUG Interaction between trace and DBUG
273  We don't want to have to duplicate code like this:
274 @verbatim
275  DBUG_PRINT("info",("cost %g",cost));
276  Opt_trace_object(thd->opt_trace).add("cost",cost);
277 @endverbatim
279  Thus, any optimizer trace operation, *even* if tracing is run-time disabled,
280  has an implicit DBUG_PRINT("opt",...) inside. This way, only the
281  second line above is needed, and several DBUG_PRINT() could be removed from
282  the Optimizer code.
283  When tracing is run-time disabled, in a debug binary, traces are still
284  created in order to catch the @c add() calls and write their text to DBUG,
285  but those traces are not visible into INFORMATION_SCHEMA.OPTIMIZER_TRACE: we
286  then say that they "don't support I_S".
287  A debug binary without optimizer trace compiled in, will intentionally not
288  compile.
290  Because opening an object or array, or add()-ing to it, writes to DBUG
291  immediately, a key/value pair and its outer object may be 100 lines
292  apart in the DBUG log.
294  @section ADDING_TRACING Guidelines for adding tracing
296  @li Try to limit the number of distinct "words". For example, when
297  describing an optimizer's decision, the words "chosen" (true/false value,
298  tells whether we are choosing the said optimization), "cause" (free text
299  value, tells why we are making this choice, when it's not obvious)
300  can and should often be used. Having a restricted vocabulary helps
301  consistency. Use "row" instead of "record". Use "tmp" instead of
302  "temporary".
304  @li Use only simple characters for key names: a-ZA-Z_#, and no space. '#'
305  serves to denote a number, like in "select#" .
307  @li Keep in mind than in an object, keys are not ordered; an application may
308  parse the JSON output and output it again with keys order changed; thus
309  when order matters, use an array (which may imply having anonymous objects
310  as items of the array, with keys inside the anonymous objects, see how it's
311  done in @c JOIN::optimize()). Keep in mind that in an object keys should
312  be unique, an application may lose duplicate keys.
314  @section OOM_HANDLING Handling of "out-of-memory" errors
316  All memory allocations (with exceptions: see below) in the Optimizer trace
317  use @c my_error() to report errors, which itself calls @c
318  error_handler_hook. It is the responsibility of the API user to set up a
319  proper @c error_handler_hook which will alert her/him of the OOM
320  problem. When in the server, this is already the case (@c error_handler_hook
321  is @c my_message_sql() which makes the statement fail).
322  Note that the debug binary may crash if OOM (OOM can cause syntax
323  errors...).
325  @section TRACE_SECURITY Description of trace-induced security checks.
327  A trace exposes information. For example if one does SELECT on a view, the
328  trace contains the view's body. So, the user should be allowed to see the
329  trace only if she/he has privilege to see the body, i.e. privilege to do
331  There are similar issues with stored procedures, functions, triggers.
333  We implement this by doing additional checks on SQL objects when tracing is
334  on:
335  @li stored procedures, functions, triggers: checks are done when executing
336  those objects
337  @li base tables and views.
339  Base tables or views are listed in some @c LEX::query_tables.
340  The LEX may be of the executing statement (statement executed by
341  @c mysql_execute_command(), or by
342  @c sp_lex_keeper::reset_lex_and_exec_core()), we check this LEX in the
343  constructor of Opt_trace_start.
344  Or it may be a LEX describing a view, we check this LEX when
345  opening the view (@c open_and_read_view()).
347  Those checks are greatly simplified by disabling traces in case of security
348  context changes. @see opt_trace_disable_if_no_security_context_access().
350  Those checks must be done with the security context of the connected
351  user. Checks with the SUID context would be useless: assume the design is
352  that the basic user does not have DML privileges on tables, but only
353  EXECUTE on SUID-highly-privileged routines (which implement _controlled_
354  _approved_ DMLs): then the SUID context would successfully pass all
355  additional privilege checks, routine would generate tracing, and the
356  connected user would view the trace after the routine's execution, seeing
357  secret information.
359  @section NEXT What a developer should read next
361  The documentation of those classes, in order
362 @code
363  Opt_trace_context
364  Opt_trace_context_impl
365  Opt_trace_stmt
366  Opt_trace_struct
367  Opt_trace_object
368  Opt_trace_array
369 @endcode
370  and then @ref opt_trace.h as a whole.
371 */
373 class Opt_trace_stmt; // implementation detail local to
375 /**
376  User-visible information about a trace. @sa Opt_trace_iterator.
377 */
379  /**
380  String containing trace.
381  If trace has been end()ed, this is 0-terminated, which is only to aid
382  debugging or unit testing; this property is not relied upon in normal
383  server usage.
384  If trace has not been ended, this is not 0-terminated. That rare case can
385  happen when a substatement reads OPTIMIZER_TRACE (at that stage, the top
386  statement is still executing so its trace is not ended yet, but may still
387  be read by the sub-statement).
388  */
389  const char *trace_ptr;
390  size_t trace_length; ///< length of trace string
391  //// String containing original query. 0-termination: like trace_ptr.
392  const char *query_ptr;
393  size_t query_length; ///< length of query string
394  const CHARSET_INFO *query_charset; ///< charset of query string
395  /**
396  How many bytes this trace is missing (for traces which were truncated
397  because of @@@@optimizer-trace-max-mem-size).
398  */
400  bool missing_priv; ///< whether user lacks privilege to see this trace
401 };
403 /**
404  Iterator over the list of remembered traces.
405  @note due to implementation, the list must not change during an
406  iterator's lifetime, or results may be unexpected (no crash though).
407 */
409  public:
410  /**
411  @param ctx context
412  */
415  void next(); ///< Advances iterator to next trace.
417  /**
418  Provides information about the trace on which the iterator is
419  positioned.
420  @param[out] info information returned.
421  The usage pattern is
422  1) instantiate the iterator
423  2) test at_end(), if false: call get_value() and then next()
424  3) repeat (2) until at_end() is true.
425  */
426  void get_value(Opt_trace_info *info) const;
428  /// @returns whether iterator is positioned to the end.
429  bool at_end() const { return cursor == nullptr; }
431  private:
432  /// Pointer to context, from which traces are retrieved
434  const Opt_trace_stmt *cursor; ///< trace which the iterator is positioned on
435  long row_count; ///< how many traces retrieved so far
436 };
438 /**
439  Object and array are both "structured data" and have lots in common, so the
440  Opt_trace_struct is a base class for them.
441  When you want to add a structure to the trace, you create an instance of
442  Opt_trace_object or Opt_trace_array, then you add information to it with
443  add(), then the destructor closes the structure (we use RAII, Resource
444  Acquisition Is Initialization).
445 */
448  protected:
449  /**
450  @param ctx_arg Optimizer trace context for this structure
451  @param requires_key_arg whether this structure requires/forbids keys
452  for values put inside it (an object requires them, an
453  array forbids them)
454  @param key key if this structure is the value of a key/value pair,
455  NULL otherwise. This pointer must remain constant and
456  valid until the object is destroyed (to support
457  @ref saved_key).
458  @param feature optimizer feature to which this structure belongs
460  This constructor is never called directly, only from subclasses.
461  */
462  Opt_trace_struct(Opt_trace_context *ctx_arg, bool requires_key_arg,
463  const char *key, Opt_trace_context::feature_value feature)
464  : started(false) {
465  // A first inlined test
466  if (unlikely(ctx_arg->is_started())) {
467  // Tracing enabled: must fully initialize the structure.
468  do_construct(ctx_arg, requires_key_arg, key, feature);
469  }
470  /*
471  Otherwise, just leave "started" to false, it marks that the structure is
472  dummy.
473  */
474  }
476  if (unlikely(started)) do_destruct();
477  }
479  public:
480  /**
481  The exception to RAII: this function is an explicit way of ending a
482  structure before it goes out of scope. Don't use it unless RAII mandates
483  a new scope which mandates re-indenting lots of code lines.
484  */
485  void end() {
486  if (unlikely(started)) do_destruct();
487  }
489  /**
490  Adds a value (of string type) to the structure. A key is specified, so it
491  adds the key/value pair (the structure must thus be an object).
493  There are two "add_*" variants to add a string value.
494  If the value is 0-terminated and each character
495  - is ASCII 7-bit
496  - has ASCII code >=32 and is neither '"' nor '\\'
497  then add_alnum() should be used. That should be the case for all fixed
498  strings like add_alnum("cause", "cost").
499  Otherwise, add_utf8() should be used; it accepts any UTF8-encoded
500  character in 'value' and will escape characters which JSON requires (and
501  is thus slower than add_alnum()). It should be used for all strings which
502  we get from the server's objects (indeed a table's name, a WHERE
503  condition, may contain "strange" characters).
505  @param key key
506  @param value value
507  @returns a reference to the structure, useful for chaining like this:
508  @verbatim add(x,y).add(z,t).add(u,v) @endverbatim
510  String-related add() variants are named add_[something]():
511  - to avoid confusing the compiler between:
512  add(const char *value, size_t val_length) and
513  add(const char *key, ulonglong value)
514  - and because String::length() returns uint32 and not size_t, so for
515  add(str.ptr(), str.length())
516  compiler may pick
517  add(const char *key, double value) instead of
518  add(const char *value, size_t val_length).
519  */
520  Opt_trace_struct &add_alnum(const char *key, const char *value) {
521  if (likely(!started)) return *this;
522  return do_add(key, value, strlen(value), false);
523  }
525  /**
526  Adds a value (of string type) to the structure. No key is specified, so
527  it adds only the value (the structure must thus be an array).
528  @param value value
529  @returns a reference to the structure
530  */
532  if (likely(!started)) return *this;
533  return do_add(nullptr, value, strlen(value), false);
534  }
536  /**
537  Like add_alnum() but supports any UTF8 characters in 'value'.
538  Will "escape" 'value' to be JSON-compliant.
539  @param key key
540  @param value value
541  @param val_length length of string 'value'
542  */
543  Opt_trace_struct &add_utf8(const char *key, const char *value,
544  size_t val_length) {
545  if (likely(!started)) return *this;
546  return do_add(key, value, val_length, true);
547  }
549  /// Variant of add_utf8() for adding to an array (no key)
550  Opt_trace_struct &add_utf8(const char *value, size_t val_length) {
551  if (likely(!started)) return *this;
552  return do_add(nullptr, value, val_length, true);
553  }
555  /// Variant of add_utf8() where 'value' is 0-terminated
556  Opt_trace_struct &add_utf8(const char *key, const char *value) {
557  if (likely(!started)) return *this;
558  return do_add(key, value, strlen(value), true);
559  }
561  /// Variant of add_utf8() where 'value' is 0-terminated
563  if (likely(!started)) return *this;
564  return do_add(nullptr, value, strlen(value), true);
565  }
567  /**
568  Add a value (of Item type) to the structure. The Item should be a
569  condition (like a WHERE clause) which will be pretty-printed into the
570  trace. This is useful for showing condition transformations (equality
571  propagation etc).
572  @param key key
573  @param item the Item
574  @return a reference to the structure
575  */
576  Opt_trace_struct &add(const char *key, Item *item) {
577  if (likely(!started)) return *this;
578  return do_add(key, item);
579  }
581  if (likely(!started)) return *this;
582  return do_add(nullptr, item);
583  }
585  public:
586  Opt_trace_struct &add(const char *key, bool value) {
587  if (likely(!started)) return *this;
588  return do_add(key, value);
589  }
591  if (likely(!started)) return *this;
592  return do_add(nullptr, value);
593  }
594  Opt_trace_struct &add(const char *key, int value) {
595  if (likely(!started)) return *this;
596  return do_add(key, static_cast<longlong>(value));
597  }
599  if (likely(!started)) return *this;
600  return do_add(nullptr, static_cast<longlong>(value));
601  }
602  Opt_trace_struct &add(const char *key, uint value) {
603  if (likely(!started)) return *this;
604  return do_add(key, static_cast<ulonglong>(value));
605  }
607  if (likely(!started)) return *this;
608  return do_add(nullptr, static_cast<ulonglong>(value));
609  }
610  Opt_trace_struct &add(const char *key, ulong value) {
611  if (likely(!started)) return *this;
612  return do_add(key, static_cast<ulonglong>(value));
613  }
615  if (likely(!started)) return *this;
616  return do_add(nullptr, static_cast<ulonglong>(value));
617  }
619  if (likely(!started)) return *this;
620  return do_add(key, value);
621  }
623  if (likely(!started)) return *this;
624  return do_add(nullptr, value);
625  }
627  if (likely(!started)) return *this;
628  return do_add(key, value);
629  }
631  if (likely(!started)) return *this;
632  return do_add(nullptr, value);
633  }
634  Opt_trace_struct &add(const char *key, double value) {
635  if (likely(!started)) return *this;
636  return do_add(key, value);
637  }
639  if (likely(!started)) return *this;
640  return do_add(nullptr, value);
641  }
642  /// Adds a 64-bit integer to trace, in hexadecimal format
644  if (likely(!started)) return *this;
645  return do_add_hex(key, value);
646  }
648  if (likely(!started)) return *this;
649  return do_add_hex(nullptr, value);
650  }
651  /// Adds a JSON null object (==Python's "None")
652  Opt_trace_struct &add_null(const char *key) {
653  if (likely(!started)) return *this;
654  return do_add_null(key);
655  }
656  /**
657  Helper to put the database/table name in an object.
658  @param tab TABLE* pointer
659  */
661  if (likely(!started)) return *this;
662  return do_add_utf8_table(tab);
663  }
664  /**
665  Helper to put the number of select_lex in an object.
666  @param select_number number of select_lex
667  */
669  return unlikely(select_number >= INT_MAX) ?
670  // Clearer than any huge number.
671  add_alnum("select#", "fake")
672  : add("select#", select_number);
673  }
674  /**
675  Add a value to the structure.
676  @param key key
677  @param cost the value of Cost_estimate
678  @return a reference to the structure
679  */
680  Opt_trace_struct &add(const char *key, const Cost_estimate &cost) {
681  if (likely(!started)) return *this;
682  return do_add(key, cost);
683  }
685  /**
686  Informs this structure that we are adding data (scalars, structures) to
687  it.
688  This is used only if sending to I_S.
689  @returns whether the structure was empty so far.
690  @note this is reserved for use by Opt_trace_stmt.
691  */
692  bool set_not_empty() {
693  const bool old_empty = empty;
694  empty = false;
695  return old_empty;
696  }
697  /**
698  Validates the key about to be added.
699  @note this is reserved for use by Opt_trace_stmt.
701  When adding a value (or array or object) to an array, or a key/value pair
702  to an object, we need to know this outer array or object.
704  It would be possible, when trying to add a key to an array, which is wrong
705  in JSON, or similarly when trying to add a value without any key to an
706  object, to catch it at compilation time, if the adder received, as
707  function parameter, the type of the structure (like @c
708  Opt_trace_array*). Then the @c add(key,val) call would not compile as
709  Opt_trace_array wouldn't feature it.
711  But as explained in comment of class Opt_trace_context we
712  cannot pass down the object, have to maintain a "current object or
713  array" in the Opt_trace_context context (pointer to an instance of
714  Opt_trace_struct), and the adder grabs it from the context.
716  As this current structure is of type "object or array", we cannot do
717  compile-time checks that only suitable functions are used. A call to @c
718  add(key,value) is necessarily legal for the compiler as the structure may
719  be an object, though it will be wrong in case the structure is actually
720  an array at run-time. Thus we have the risk of an untested particular
721  situation where the current structure is not an object (but an array)
722  though the code expected it to be one. This happens in practice, because
723  subqueries are evaluated in many possible places of code, not all of them
724  being known. Same happens, to a lesser extent, with calls to the range
725  optimizer.
726  So at run-time, in check_key(), we detect wrong usage, like adding a value
727  to an object without specifying a key, and then remove the unnecessary
728  key, or add an autogenerated key.
729  */
730  const char *check_key(const char *key);
732  private:
733  /// Not implemented, use add_alnum() instead.
734  Opt_trace_struct &add(const char *key, const char *value);
735  Opt_trace_struct &add(const char *key);
737  /// Full initialization. @sa Opt_trace_struct::Opt_trace_struct
738  void do_construct(Opt_trace_context *ctx, bool requires_key, const char *key,
740  /// Really does destruction
741  void do_destruct();
742  /**
743  Really adds to the object. @sa add().
745  @note add() has an up-front if(), hopefully inlined, so that in the
746  common case - tracing run-time disabled - we have no function call. If
747  tracing is enabled, we call do_add().
748  In a 20-table plan search (as in BUG#50595), the execution time was
749  decreased from 2.6 to 2.0 seconds thanks to this inlined-if trick.
751  @param key key
752  @param value value
753  @param val_length length of string 'value'
754  @param escape do JSON-compliant escaping of 'value'. If 'escape' is
755  false, 'value' should be ASCII. Otherwise, should be UTF8.
756  */
757  Opt_trace_struct &do_add(const char *key, const char *value,
758  size_t val_length, bool escape);
759  Opt_trace_struct &do_add(const char *key, Item *item);
760  Opt_trace_struct &do_add(const char *key, bool value);
761  Opt_trace_struct &do_add(const char *key, longlong value);
762  Opt_trace_struct &do_add(const char *key, ulonglong value);
763  Opt_trace_struct &do_add(const char *key, double value);
764  Opt_trace_struct &do_add_hex(const char *key, uint64 value);
765  Opt_trace_struct &do_add_null(const char *key);
767  Opt_trace_struct &do_add(const char *key, const Cost_estimate &value);
769  Opt_trace_struct(const Opt_trace_struct &); ///< not defined
770  Opt_trace_struct &operator=(const Opt_trace_struct &); ///< not defined
772  bool started; ///< Whether the structure does tracing or is dummy
774  /**
775  Whether the structure requires/forbids keys for values inside it.
776  true: this is an object. false: this is an array.
778  @note The canonical way would be to not have such bool per instance, but
779  rather have a pure virtual member function
780  Opt_trace_struct::requires_key(), overloaded by Opt_trace_object
781  (returning true) and by Opt_trace_array (returning false). But
782  Opt_trace_object::requires_key() would not be accessible from
783  Opt_trace_struct::do_construct() (which would complicate coding), whereas
784  the bool is.
785  */
788  /**
789  Whether this structure caused tracing to be disabled in this statement
790  because belonging to a not-traced optimizer feature, in accordance with
791  the value of @@@@optimizer_trace_features.
792  */
794  bool empty; ///< @see set_not_empty()
795  Opt_trace_stmt *stmt; ///< Trace owning the structure
796  /// Key if the structure is the value of a key/value pair, NULL otherwise
797  const char *saved_key;
798 #ifndef DBUG_OFF
799  /**
800  Fixed-length prefix of previous key in this structure, if this structure
801  is an object. Serves to detect when adding two same consecutive keys to
802  an object, which would be wrong.
803  */
804  char previous_key[25];
805 #endif
806 };
808 /**
809  A JSON object (unordered set of key/value pairs).
810  Defines only a constructor, all the rest is inherited from
811  Opt_trace_struct.
812 */
814  public:
815  /**
816  Constructs an object. Key is specified, so the object is the value of a
817  key/value pair.
818  @param ctx context for this object
819  @param key key
820  @param feature optimizer feature to which this structure belongs
821  */
823  Opt_trace_context *ctx, const char *key,
825  : Opt_trace_struct(ctx, true, key, feature) {}
826  /**
827  Constructs an object. No key is specified, so the object is just a value
828  (serves for the single root object or for adding the object to an array).
829  @param ctx context for this object
830  @param feature optimizer feature to which this structure belongs
831  */
833  Opt_trace_context *ctx,
835  : Opt_trace_struct(ctx, true, nullptr, feature) {}
836 };
838 /**
839  A JSON array (ordered set of values).
840  Defines only a constructor, all the rest in inherited from
841  Opt_trace_struct.
842 */
844  public:
845  /**
846  Constructs an array. Key is specified, so the array is the value of a
847  key/value pair.
848  @param ctx context for this array
849  @param key key
850  @param feature optimizer feature to which this structure belongs
851  */
853  Opt_trace_context *ctx, const char *key,
855  : Opt_trace_struct(ctx, false, key, feature) {}
856  /**
857  Constructs an array. No key is specified, so the array is just a value
858  (serves for adding the object to an array).
859  @param ctx context for this array
860  @param feature optimizer feature to which this structure belongs
861  */
863  Opt_trace_context *ctx,
865  : Opt_trace_struct(ctx, false, nullptr, feature) {}
866 };
868 /**
869  Instantiate an instance of this class for specific cases where
870  optimizer trace, in a certain section of Optimizer code, should write only
871  to DBUG and not I_S. Example: see
872  Note that this class should rarely be used; the "feature" parameter of
873  Opt_trace_struct is a good alternative.
874 */
876  public:
877  /**
878  @param ctx_arg Context.
879  @param disable_arg Whether the instance should really disable
880  anything. If false, the object is dummy. If true,
881  tracing-to-I_S is disabled at construction and
882  re-enabled at destruction.
883  @details A dummy instance is there only for RAII reasons. Imagine we want
884  to do this:
885 @verbatim
886  {
887  if (x) disable tracing;
888  code;
889  } // tracing should be re-enabled here
890 @endverbatim
891  As we use RAII, we cannot put the instance declaration inside if(x):
892 @verbatim
893  {
894  if (x) Opt_trace_disable_I_S instance(ctx);
895  code;
896  }
897 @endverbatim
898  because it would be destroyed as soon as the if() block is left, so
899  tracing would be re-enabled before @c code;. It should rather be written
900  as:
901 @verbatim
902  {
903  Opt_trace_disable_I_S instance(ctx, x); // if !x, does nothing
904  code;
905  } // re-enabling happens here, if x is true
906 @endverbatim
907  */
908  Opt_trace_disable_I_S(Opt_trace_context *ctx_arg, bool disable_arg) {
909  if (disable_arg) {
910  ctx = ctx_arg;
912  } else
913  ctx = nullptr;
914  }
916  /// Destructor. Restores trace's "enabled" property to its previous value.
918  if (ctx != nullptr) ctx->restore_I_S();
919  }
921  private:
922  /** Context. Non-NULL if and only if this instance really does disabling */
926  const Opt_trace_disable_I_S &); // not defined
927 };
929 /**
930  @name Helpers connecting the optimizer trace to THD or Information Schema.
931 */
933 //@{
936  public:
937  /**
938  Instantiate this class to start tracing a THD's actions (generally at a
939  statement's start), and to set the "original" query (not transformed, as
940  sent by client) for the new trace. Destructor will end the trace.
942  If in a routine's instruction, there is no "query". To be helpful to the
943  user, we craft a query using the instruction's print(). We don't leave this
944  to the caller, because it would be inefficient if tracing is off.
946  @param thd_arg the THD
947  @param tbl list of tables read/written by the statement.
948  @param sql_command SQL command being prepared or executed
949  @param set_vars what variables are set by this command (only used if
950  sql_command is SQLCOM_SET_OPTION)
951  @param query query
952  @param query_length query's length
953  @param instr routine's instruction, if applicable; if so, 'query'
954  and 'query_length' above are ignored
955  @param query_charset charset which was used to encode this query
957  @note Each sub-statement is responsible for ending the trace which it
958  has started; this class achieves this by keeping some memory inside.
959  */
960  Opt_trace_start(THD *thd_arg, TABLE_LIST *tbl,
961  enum enum_sql_command sql_command,
962  List<set_var_base> *set_vars, const char *query,
963  size_t query_length, sp_printable *instr,
964  const CHARSET_INFO *query_charset);
967  private:
969  bool error; ///< whether trace start() had an error
970 };
972 class SELECT_LEX;
974 /**
975  Prints SELECT query to optimizer trace. It is not the original query (as in
976  @c Opt_trace_context::set_query()) but a printout of the parse tree
977  (Item-s).
978  @param thd the THD
979  @param select_lex query's parse tree
980  @param trace_object Opt_trace_object to which the query will be added
981 */
982 void opt_trace_print_expanded_query(const THD *thd, SELECT_LEX *select_lex,
983  Opt_trace_object *trace_object);
985 /**
986  If the security context is not that of the connected user, inform the trace
987  system that a privilege is missing. With one exception: see below.
989  @param thd the THD
991  This serves to eliminate the following issue.
992  Any information readable by a SELECT may theoretically end up in
993  the trace. And a SELECT may read information from other places than tables:
994  - from views (reading their bodies)
995  - from stored routines (reading their bodies)
996  - from files (reading their content), with LOAD_FILE()
997  - from the list of connections (reading their queries...), with
999  If the connected user has EXECUTE privilege on a routine which does a
1000  security context change, the routine can retrieve information internally
1001  (if allowed by the SUID context's privileges), and present only a portion
1002  of it to the connected user. But with tracing on, all information is
1003  possibly in the trace. So the connected user receives more information than
1004  the routine's definer intended to provide. Fixing this issue would require
1005  adding, near many privilege checks in the server, a new
1006  optimizer-trace-specific check done against the connected user's context,
1007  to verify that the connected user has the right to see the retrieved
1008  information.
1010  Instead, our chosen simpler solution is that if we see a security context
1011  change where SUID user is not the connected user, we disable tracing. With
1012  only one safe exception: if the connected user has all global privileges
1013  (because then she/he can find any information anyway). By "all global
1014  privileges" we mean everything but WITH GRANT OPTION (that latter one isn't
1015  related to information gathering).
1017  Read access to I_S.OPTIMIZER_TRACE by another user than the connected user
1018  is restricted: @see fill_optimizer_trace_info().
1019 */
1022 /**
1023  If tracing is on, checks additional privileges for a view, to make sure
1024  that the user has the right to do SHOW CREATE VIEW. For that:
1025  - this function checks SHOW VIEW
1026  - SELECT is tested in opt_trace_disable_if_no_tables_access()
1027  - SELECT + SHOW VIEW is sufficient for SHOW CREATE VIEW.
1028  We also check underlying tables.
1029  If a privilege is missing, notifies the trace system.
1030  This function should be called when the view's underlying tables have not
1031  yet been merged.
1033  @param thd THD context
1034  @param view view to check
1035  @param underlying_tables underlying tables/views of 'view'
1036  */
1038  TABLE_LIST *underlying_tables);
1040 /**
1041  If tracing is on, checks additional privileges on a stored routine, to make
1042  sure that the user has the right to do SHOW CREATE PROCEDURE/FUNCTION. For
1043  that, we use the same checks as in those SHOW commands.
1044  If a privilege is missing, notifies the trace system.
1046  This function is not redundant with
1047  opt_trace_disable_if_no_security_context_access().
1048  Indeed, for a SQL SECURITY INVOKER routine, there is no context change, but
1049  we must still verify that the invoker can do SHOW CREATE.
1051  For triggers, see note in sp_head::execute_trigger().
1053  @param thd The THD
1054  @param sp routine to check
1055  */
1058 /**
1059  Fills information_schema.OPTIMIZER_TRACE with rows (one per trace)
1060  @retval 0 ok
1061  @retval 1 error
1062 */
1063 int fill_optimizer_trace_info(THD *thd, TABLE_LIST *tables, Item *);
1065 //@}
1067 /**
1068  Helper for defining query-transformation-related trace objects in one
1069  code line. This produces
1070  {
1071  "transformation": {
1072  "select#": @<select_number@>,
1073  "from": @<from@>,
1074  "to": @<to@>
1075  The objects are left open, so that one can add more to them (often a
1076  "chosen" property after making some computation). Objects get closed when
1077  going out of scope as usual.
1078  @param trace optimizer trace
1079  @param object_level0 name of the outer Opt_trace_object C++ object
1080  @param object_level1 name of the inner Opt_trace_object C++ object
1081  @param select_number number of the being-transformed SELECT_LEX
1082  @param from description of the before-transformation state
1083  @param to description of the after-transformation state
1084 */
1085 #define OPT_TRACE_TRANSFORM(trace, object_level0, object_level1, \
1086  select_number, from, to) \
1087  Opt_trace_object object_level0(trace); \
1088  Opt_trace_object object_level1(trace, "transformation"); \
1089  object_level1.add_select_number(select_number); \
1090  object_level1.add_alnum("from", from).add_alnum("to", to);
1092 #endif /* OPT_TRACE_INCLUDED */
Opt_trace_struct(Opt_trace_context *ctx_arg, bool requires_key_arg, const char *key, Opt_trace_context::feature_value feature)
Definition: opt_trace.h:462
bool unlikely(bool expr)
Definition: my_compiler.h:55
Opt_trace_context * ctx
Definition: opt_trace.h:923
Definition: item.h:775
const CHARSET_INFO * query_charset
charset of query string
Definition: opt_trace.h:394
Definition: sql_class.h:807
Definition: opt_trace.h:935
Opt_trace_disable_I_S(const Opt_trace_disable_I_S &)
Anything unclassified, including the top object (thus, by "inheritance from parent",...
Definition: opt_trace_context.h:214
void disable_I_S_for_this_and_children()
Temporarily disables I_S for this trace and its children.
Definition: opt_trace_context.h:255
void opt_trace_disable_if_no_stored_proc_func_access(THD *thd, sp_head *sp)
If tracing is on, checks additional privileges on a stored routine, to make sure that the user has th...
Opt_trace_struct & add(const char *key, Item *item)
Add a value (of Item type) to the structure.
Definition: opt_trace.h:576
const Opt_trace_stmt * cursor
trace which the iterator is positioned on
Definition: opt_trace.h:434
void restore_I_S()
Restores I_S support to what it was before the previous call to disable_I_S_for_this_and_children().
Definition: opt_trace_context.h:264
sp_head represents one instance of a stored program.
Definition: sp_head.h:379
bool missing_priv
whether user lacks privilege to see this trace
Definition: opt_trace.h:400
A base class for everything that can be set with SET command.
Definition: set_var.h:361
Definition: m_ctype.h:354
Dialog Client Authentication nullptr
uint64_t uint64
Definition: my_inttypes.h:68
void opt_trace_disable_if_no_security_context_access(THD *thd)
If the security context is not that of the connected user, inform the trace system that a privilege i...
void do_destruct()
Really does destruction.
Opt_trace_struct & add_alnum(const char *key, const char *value)
Adds a value (of string type) to the structure.
Definition: opt_trace.h:520
Definition: opt_trace_context.h:88
bool error
whether trace start() had an error
Definition: opt_trace.h:969
Features' numeric values for @@optimizer_trace_features variable.
Definition: opt_trace_context.h:199
long long int longlong
Definition: my_inttypes.h:54
Opt_trace_struct & add_utf8(const char *key, const char *value, size_t val_length)
Like add_alnum() but supports any UTF8 characters in 'value'.
Definition: opt_trace.h:543
A JSON array (ordered set of values).
Definition: opt_trace.h:843
bool empty
Definition: opt_trace.h:794
const string value("\"Value\"")
Opt_trace_iterator(Opt_trace_context *ctx)
Opt_trace_object(Opt_trace_context *ctx, const char *key, Opt_trace_context::feature_value feature=Opt_trace_context::MISC)
Constructs an object.
Definition: opt_trace.h:822
Opt_trace_array(Opt_trace_context *ctx, const char *key, Opt_trace_context::feature_value feature=Opt_trace_context::MISC)
Constructs an array.
Definition: opt_trace.h:852
sp_printable defines an interface which should be implemented if a class wants report some internal i...
Definition: sp_instr.h:64
bool is_started() const
Returns whether there is a current trace.
Definition: opt_trace_context.h:142
Opt_trace_struct & add_alnum(const char *value)
Adds a value (of string type) to the structure.
Definition: opt_trace.h:531
Opt_trace_struct & add(bool value)
Definition: opt_trace.h:590
#define true
Definition: config_static.h:44
size_t query_length
length of query string
Definition: opt_trace.h:393
Opt_trace_struct & add(ulong value)
Definition: opt_trace.h:614
static const char * key
Definition: suite_stubs.c:14
void opt_trace_print_expanded_query(const THD *thd, SELECT_LEX *select_lex, Opt_trace_object *trace_object)
Prints SELECT query to optimizer trace.
Opt_trace_struct & add_select_number(uint select_number)
Helper to put the number of select_lex in an object.
Definition: opt_trace.h:668
Opt_trace_struct & operator=(const Opt_trace_struct &)
not defined
Object and array are both "structured data" and have lots in common, so the Opt_trace_struct is a bas...
Definition: opt_trace.h:447
Opt_trace_struct & add(const char *key, bool value)
Definition: opt_trace.h:586
Opt_trace_struct & add(ulonglong value)
Definition: opt_trace.h:630
Definition: opt_trace.h:475
Opt_trace_struct & add_hex(const char *key, uint64 value)
Adds a 64-bit integer to trace, in hexadecimal format.
Definition: opt_trace.h:643
Opt_trace_struct & add(double value)
Definition: opt_trace.h:638
Opt_trace_struct & add_utf8(const char *value, size_t val_length)
Variant of add_utf8() for adding to an array (no key)
Definition: opt_trace.h:550
size_t trace_length
length of trace string / String containing original query. 0-termination: like trace_ptr.
Definition: opt_trace.h:390
Opt_trace_struct & do_add(const char *key, const char *value, size_t val_length, bool escape)
Really adds to the object.
Opt_trace_struct & add(longlong value)
Definition: opt_trace.h:622
long row_count
how many traces retrieved so far
Definition: opt_trace.h:435
bool requires_key
Whether the structure requires/forbids keys for values inside it.
Definition: opt_trace.h:786
Definition: my_sqlcommand.h:45
Opt_trace_struct & add(const char *key)
Instantiate an instance of this class for specific cases where optimizer trace, in a certain section ...
Definition: opt_trace.h:875
Opt_trace_struct & add(const char *key, const Cost_estimate &cost)
Add a value to the structure.
Definition: opt_trace.h:680
Opt_trace_context *const ctx
Definition: opt_trace.h:968
bool set_not_empty()
Informs this structure that we are adding data (scalars, structures) to it.
Definition: opt_trace.h:692
bool at_end() const
Definition: opt_trace.h:429
unsigned int uint
bool has_disabled_I_S
Whether this structure caused tracing to be disabled in this statement because belonging to a not-tra...
Definition: opt_trace.h:793
void opt_trace_disable_if_no_view_access(THD *thd, TABLE_LIST *view, TABLE_LIST *underlying_tables)
If tracing is on, checks additional privileges for a view, to make sure that the user has the right t...
Opt_trace_object(Opt_trace_context *ctx, Opt_trace_context::feature_value feature=Opt_trace_context::MISC)
Constructs an object.
Definition: opt_trace.h:832
Opt_trace_struct & add(int value)
Definition: opt_trace.h:598
Opt_trace_context * ctx
Pointer to context, from which traces are retrieved.
Definition: opt_trace.h:433
static std::string escape(const std::string &str)
Escapes (only) apostrophes.
Iterator over the list of remembered traces.
Definition: opt_trace.h:408
Definition: table.h:2541
Opt_trace_struct & do_add_utf8_table(const TABLE_LIST *tab)
Opt_trace_struct & add(const char *key, uint value)
Definition: opt_trace.h:602
Opt_trace_struct & add(Item *item)
Definition: opt_trace.h:580
size_t missing_bytes
How many bytes this trace is missing (for traces which were truncated because of @@optimizer-trace-ma...
Definition: opt_trace.h:399
Used to store optimizer cost estimates.
Definition: handler.h:3340
Opt_trace_struct & add_hex(uint64 value)
Definition: opt_trace.h:647
Opt_trace_struct(const Opt_trace_struct &)
not defined
void next()
Advances iterator to next trace.
User-visible information about a trace.
Definition: opt_trace.h:378
Opt_trace_struct & add(const char *key, longlong value)
Definition: opt_trace.h:618
static char * query
Opt_trace_struct & add_utf8(const char *key, const char *value)
Variant of add_utf8() where 'value' is 0-terminated.
Definition: opt_trace.h:556
Opt_trace_struct & do_add_hex(const char *key, uint64 value)
Opt_trace_disable_I_S & operator=(const Opt_trace_disable_I_S &)
Opt_trace_start(THD *thd_arg, TABLE_LIST *tbl, enum enum_sql_command sql_command, List< set_var_base > *set_vars, const char *query, size_t query_length, sp_printable *instr, const CHARSET_INFO *query_charset)
Instantiate this class to start tracing a THD's actions (generally at a statement's start),...
Opt_trace_stmt * stmt
Trace owning the structure.
Definition: opt_trace.h:795
Opt_trace_struct & add_utf8_table(const TABLE_LIST *tab)
Helper to put the database/table name in an object.
Definition: opt_trace.h:660
Opt_trace_struct & add(const char *key, ulong value)
Definition: opt_trace.h:610
A JSON object (unordered set of key/value pairs).
Definition: opt_trace.h:813
bool started
Whether the structure does tracing or is dummy.
Definition: opt_trace.h:772
unsigned long long int ulonglong
Definition: my_inttypes.h:55
Opt_trace_struct & add_null(const char *key)
Adds a JSON null object (==Python's "None")
Definition: opt_trace.h:652
Definition: auth_common.h:57
int fill_optimizer_trace_info(THD *thd, TABLE_LIST *tables, Item *)
Fills information_schema.OPTIMIZER_TRACE with rows (one per trace)
const char * trace_ptr
String containing trace.
Definition: opt_trace.h:389
Opt_trace_struct & add(uint value)
Definition: opt_trace.h:606
Opt_trace_disable_I_S(Opt_trace_context *ctx_arg, bool disable_arg)
Definition: opt_trace.h:908
void get_value(Opt_trace_info *info) const
Provides information about the trace on which the iterator is positioned.
void end()
The exception to RAII: this function is an explicit way of ending a structure before it goes out of s...
Definition: opt_trace.h:485
This class represents a query block, aka a query specification, which is a query consisting of a SELE...
Definition: sql_lex.h:1098
const char * check_key(const char *key)
Validates the key about to be added.
Opt_trace_struct & add(const char *key, const char *value)
Not implemented, use add_alnum() instead.
const char * saved_key
Key if the structure is the value of a key/value pair, NULL otherwise.
Definition: opt_trace.h:797
char previous_key[25]
Fixed-length prefix of previous key in this structure, if this structure is an object.
Definition: opt_trace.h:804
Opt_trace_struct & add(const char *key, ulonglong value)
Definition: opt_trace.h:626
Opt_trace_struct & add_utf8(const char *value)
Variant of add_utf8() where 'value' is 0-terminated.
Definition: opt_trace.h:562
Log info(cout, "NOTE")
const char * query_ptr
Definition: opt_trace.h:392
void do_construct(Opt_trace_context *ctx, bool requires_key, const char *key, Opt_trace_context::feature_value feature)
Full initialization.
Opt_trace_struct & add(const char *key, double value)
Definition: opt_trace.h:634
Opt_trace_array(Opt_trace_context *ctx, Opt_trace_context::feature_value feature=Opt_trace_context::MISC)
Constructs an array.
Definition: opt_trace.h:862
bool likely(bool expr)
Definition: my_compiler.h:54
Opt_trace_struct & add(const char *key, int value)
Definition: opt_trace.h:594
#define false
Definition: config_static.h:43
Opt_trace_struct & do_add_null(const char *key)
Destructor. Restores trace's "enabled" property to its previous value.
Definition: opt_trace.h:917