MySQL 8.0.32
Source Code Documentation
bootstrapper.h
Go to the documentation of this file.
1/* Copyright (c) 2014, 2022, Oracle and/or its affiliates.
2
3 This program is free software; you can redistribute it and/or modify
4 it under the terms of the GNU General Public License, version 2.0,
5 as published by the Free Software Foundation.
6
7 This program is also distributed with certain software (including
8 but not limited to OpenSSL) that is licensed under separate terms,
9 as designated in a particular file or component or in included license
10 documentation. The authors of MySQL hereby grant you an additional
11 permission to link the program and your derivative works with the
12 separately licensed software that they have included with MySQL.
13
14 This program is distributed in the hope that it will be useful,
15 but WITHOUT ANY WARRANTY; without even the implied warranty of
16 MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
17 GNU General Public License, version 2.0, for more details.
18
19 You should have received a copy of the GNU General Public License
20 along with this program; if not, write to the Free Software
21 Foundation, Inc., 51 Franklin St, Fifth Floor, Boston, MA 02110-1301 USA */
22
23#ifndef DD__BOOTSTRAPPER_INCLUDED
24#define DD__BOOTSTRAPPER_INCLUDED
25
26#include <sys/types.h>
27
28#include "sql/dd/impl/system_registry.h" // dd::System_tables
29#include "sql/dd/string_type.h" // dd::String_type
30#include "sql/handler.h" // dict_init_mode_t
31
32class THD;
33
34/**
35 Data dictionary initialization.
36
37 The data dictionary is initialized whenever the mysqld process starts.
38 We distinguish between the first time start and the subsequent normal
39 restarts/upgrades, as explained below. However, there are three main
40 design principles that should be elaborated first.
41
42 1. Two-step process: The dictionary initialization is implemented as
43 a two step process. First, scaffolding is built to prepare the
44 synchronization with persistent storage, then, the actual synchronization
45 is done. The way this is done depends on the context, and is different
46 for first time start and the subsequent restarts.
47
48 2. Use SQL: The initialization uses SQL to build the scaffolding. This
49 means that we execute SQL statements to create the dictionary tables.
50 Since this is done at a stage where the physical tables either do not
51 exist yet, or are not known, we must instrument the DDL execution to
52 create the physical counterpart of the tables only on first time start.
53 The goal is to keep the instrumentation at a minimum.
54
55 3. Fake caching: As a consequence of keeping instrumentation at a minimum,
56 we provide uniform behavior of the caching layer in the data dictionary
57 also in the scaffolding phase. This means that as seen from the outside,
58 dictionary objects can be retrieved from the cache. Internally, below the
59 caching layer, the objects are only kept in a separate buffer until all
60 the required scaffolding is built. At that point, we can start using the
61 underlying physical tables, depending on the circumstances:
62
63 - For first time start (initialization), we can flush the meta data
64 generated in the scaffolding phase, to the DD tables.
65 - For ordinary restart, we can use the scaffolding to open the physical
66 tables, and then sync up the real meta data that is stored persistently.
67 - For upgrade, we first build scaffolding based on the actual DD tables,
68 then we create the target DD tables, migrate the meta data from the old
69 to the new tables, and finally switch from old to new tables
70 atomically by means of DML on the DD tables. This means that we update
71 the schema ids in the DD tables directly instead of executing
72 'RENAME TABLE', which would do auto commit and thus break atomicity.
73
74 After the scaffolding has been flushed or synced, what should be left is
75 a collection of the core DD meta data objects. This collection is located
76 in the storage adapter, and allows the DD cache to evict core DD objects
77 in the same way as other DD objects.
78
79 Please note that dictionary initialization is only a small part of server
80 initialization. There is a lot going on before and after dictionary
81 initialization while starting the server.
82
83 Please see more elaborated descriptions for the initialize() and restart()
84 methods below.
85*/
86
87namespace dd {
88class Dictionary_impl;
89
90namespace bootstrap {
91
92/**
93 Initialize the dictionary while starting the server for the first time.
94
95 At this point, the DDSE has been initialized as a normal plugin. The
96 dictionary initialization proceeds as follows:
97
98 1. Preparation phase
99
100 1.1 Call dict_init() to initialize the DDSE. This will make the predefined
101 tablespaces be created physically, and their meta data be returned to
102 the SQL layer along with the meta data for the DD tables required by
103 the DDSE. The tables are not yet created physically.
104 1.2 Prepare the dd::Tablespace objects reflecting the predefined tablespace
105 objects and add them to the core registry in the storage adapter.
106
107 2. Scaffolding phase
108
109 2.1 Create and use the dictionary schema by executing SQL statements.
110 The schema is created physically since this is the first time start,
111 and the meta data is generated and stored in the core registry of
112 the storage adapter without being written to disk.
113 2.2 Create tables by executing SQL statements. Like for the schema, the
114 tables are created physically, and the meta data is generated
115 and stored in the core registry without being written to disk.
116 This is done to prepare enough meta data to actually be able to
117 open the DD tables.
118
119 3. Synchronization phase
120
121 3.1 Store meta data for the DD schema, tablespace and tables, i.e., the DD
122 objects that were generated in the scaffolding phase, and make sure the
123 IDs are maintained when the objects are stored.
124 3.2 Populate the DD tables which have some predefined static contents to
125 be inserted. This is, e.g., relevant for the 'catalogs' table, which
126 only has a single default entry in it. Dynamic contents is added in
127 other ways, e.g. by storing generated DD objects (see above) or by
128 inserting data from other sources (see re-population of character sets
129 in the context of server restart below).
130 3.3 Store various properties of the DD tables, including the SE private data,
131 a representation of the DDL statement used to create the table etc.
132 3.4 Verify that the dictionary objects representing the core DD table meta
133 data are present in the core registry of the storage adapter. If an
134 object representing the meta data of a core DD table is not available,
135 then we loose access to the DD tables, and we will not be able to handle
136 cache misses or updates to the meta data.
137 3.5 Update the version numbers that are stored, e.g. the DD version and the
138 current mysqld server version.
139
140 @param thd Thread context.
141
142 @return Upon failure, return true, otherwise false.
143*/
144
145bool initialize(THD *thd);
146
147/**
148 Initialize the dictionary while restarting the server.
149
150 At this point, the DDSE has been initialized as a normal plugin. The
151 dictionary initialization proceeds as follows:
152
153 1. Preparation phase
154
155 1.1 Call dict_init() to initialize the DDSE. This will retrieve the meta data
156 of the predefined tablespaces and the DD tables required by the DDSE.
157 Both the tables and the tablespaces are already created physically, the
158 point here is just to get hold of enough meta data to start using the DD.
159 1.2 Prepare the dd::Tablespace objects reflecting the predefined tablespace
160 objects and add them to the core registry in the storage adapter.
161
162 2. Scaffolding phase
163
164 2.1 Create and use the dictionary schema by executing SQL statements.
165 The schema is not created physically, but the meta data is generated
166 and stored in the core registry without being written to disk.
167 2.2 Create tables by executing SQL statements. Like for the schema, the
168 tables are not created physically, but the meta data is generated
169 and stored in the core registry without being written to disk.
170 This is done to prepare enough meta data to actually be able to
171 open the DD tables. The SQL DDL statements are either retrieved from
172 the table definitions that are part of the server binary (for restart),
173 or from one of the DD tables (for upgrade).
174
175 3. Synchronization phase
176
177 3.1 Read meta data for the DD tables from the DD tables. Here, we use the
178 meta data from the scaffolding phase for the schema, tablespace and the
179 DD tables to open the physical DD tables. We read the stored objects,
180 and update the in-memory copies in the core registry with the real meta
181 data from the objects that are retrieved form persistent storage. Finally,
182 we flush the tables to empty the table definition cache to make sure the
183 table share structures for the DD tables are re-created based on the
184 actual meta data that was read from disk rather than the temporary meta
185 data from the scaffolding phase.
186 3.2 If this is a restart with a new DD version, we must upgrade the DD
187 tables. In that case, we create the new target DD tables in a temporary
188 schema, migrate the meta data to the new tables, and then do DML on the
189 DD tables to make sure the new DD tables will be used instead of the old
190 ones. This DML involves changing the schema ids directly in the DD tables,
191 and updating the meta data stored in the 'dd_properties' DD table.
192 This will make sure the switch from the old to the new tables is
193 atomic. After this is done, we will reset the DD cache and start over
194 the initialization from step 1.2. Then, the new DD tables will be used,
195 and a normal restart will be done.
196 3.3 Re-populate character sets and collations: The character set and
197 collation information is read from files and added to a server
198 internal data structure when the server starts. This data structure is,
199 in turn, used to populate the corresponding DD tables. The tables must
200 be re-populated on each server start if new character sets or collations
201 have been added. However, we can not do this if in read only mode.
202 3.4 Verify that the dictionary objects representing the core DD table meta
203 data are present in the core registry of the storage adapter. If an
204 object representing the meta data of a core DD table is not available,
205 then we loose access to the DD tables, and we will not be able to handle
206 cache misses or updates to the meta data.
207 3.5 If an upgrade was done, the persistent version numbers are updated,
208 e.g. the DD version and the current mysqld server version.
209
210 @param thd Thread context.
211
212 @return Upon failure, return true, otherwise false.
213*/
214
215bool restart(THD *thd);
216
217/**
218 Iterate through all the plugins, and store IS table meta data
219 into dictionary, once during MySQL server bootstrap.
220
221 @param thd Thread context.
222
223 @return Upon failure, return true, otherwise false.
224*/
226
227/**
228 Initialization and verification of dictionary objects
229 after upgrade, similar to what is done after normal server
230 restart.
231
232 @param thd Thread context
233*/
235
236/**
237 This function is used in case of crash during upgrade.
238 It tries to initialize dictionary and calls DDSE_dict_recover.
239 InnoDB should do the recovery and empty undo log. Upgrade
240 process will do the cleanup and exit.
241
242 @param thd Thread context.
243*/
245
246/**
247 Initialize InnoDB for
248 - creating new data directory : InnoDB creates system tablespace and
249 dictionary tablespace.
250 - normal server restart. : Verifies existence of system and dictionary
251 tablespaces.
252 - in place upgrade : Verifies existence of system tablespace and
253 create dictionary tablespace.
254
255 @param thd Thread context.
256 @param dict_init_mode mode to initialize InnoDB
257 @param version Dictionary version.
258
259 @return Upon failure, return true, otherwise false.
260*/
261bool DDSE_dict_init(THD *thd, dict_init_mode_t dict_init_mode, uint version);
262
263/**
264 Create mysql schema. Create dictionary tables inside InnoDB.
265 Create entry for dictionary tables inside dictionary tables.
266 Add hard coded data to dictionary tables.
267 Create Foreign key constraint on dictionary tables.
268
269 This function is used in both cases, new data directory initialization
270 and in place upgrade.
271
272 @param thd Thread context.
273 @param is_dd_upgrade Flag to indicate if it is in place upgrade.
274 @param d Dictionary instance
275
276 @return Upon failure, return true, otherwise false.
277
278*/
279bool initialize_dictionary(THD *thd, bool is_dd_upgrade, Dictionary_impl *d);
280
281} // namespace bootstrap
282
283/**
284 This function creates the meta data of the predefined tablespaces.
285
286 @param thd Thread context.
287*/
289
290/**
291 Executes SQL queries to create and use the dictionary schema.
292
293 @param thd Thread context.
294*/
295bool create_dd_schema(THD *thd);
296
297/**
298 During --initialize, we create the dd_properties table. During restart,
299 create its meta data, and use it to open and read its contents.
300
301 @param thd Thread context.
302*/
304
305/**
306 Predicate to check if a table type is a non-inert DD or a DDSE table.
307
308 @param table_type Type as defined in the System_tables registry.
309 @returns true if the table is a non-inert DD or DDSE table,
310 false otherwise
311*/
313
314/**
315 Execute SQL statements to create the DD tables.
316
317 The tables created here will be a subset of the target DD tables for this
318 DD version. This function is called in the following four cases:
319
320 1. When a server is started the first time, with --initialize. Then, we
321 will iterate over all target tables and create them. This will also
322 make them be created physically in the DDSE.
323 2. When a server is restarted, and the data directory contains a dictionary
324 with the same DD version as the target DD version of the starting server.
325 In this case, we will iterate over all target tables and create them,
326 using the target table SQL DDL definitions. This is done only to create
327 the meta data, though; the tables will not be created physically in the
328 DDSE since they already exist. But we need to create the meta data to be
329 able top open them.
330 3. When a server is restarted, and the data directory was last used by a
331 more recent MRU within the same GA with a higher target DD version.
332 This is considered a 'minor downgrade'. In this case, the restarting
333 server will continue to run using the more recent DD version. This is
334 possible since only a subset of DD changes are allowed in a DD upgrade
335 that can also be downgraded. However, it means that we must create the
336 meta data reflecting the *actual* tables, not the target tables. So in
337 this case, we iterate over the target tables, but execute the DDL
338 statements of the actual tables. We get these statements from the
339 'dd_properties' table, where the more recent MRU has stored them.
340 4. When a server is restarted, and the data directory was last used by a
341 server with a DD version from which the starting server can upgrade. In
342 this case, this function is called three times:
343
344 - The first time, we need to create the meta data reflecting the actual
345 tables in the persistent DD. This is needed to be able to open the DD
346 tables and read the data. This is similar to use case 3. above.
347 - The second time, we create the tables that are modified in the new DD
348 version. Here, the tables are also created physically in the DDSE.
349 In this case, the 'create_set' specifies which subset of the target
350 tables should be created. After this stage, we replace the meta data
351 in 'dd_properties' by new meta data reflecting the modified tables. We
352 also replace the version numbers to make sure a new restart will use
353 the upgraded DD.
354 - The third time, we do the same as in case 2 above. This is basically
355 the same as a shutdown and restart of the server after upgrade was
356 completed.
357
358 @param thd Thread context.
359 @param create_set Subset of the target tables which should be created
360 during upgrade.
361
362 @returns false if success, otherwise true.
363*/
364bool create_tables(THD *thd, const std::set<String_type> *create_set);
365
366/**
367 Acquire the DD schema, tablespace and table objects. Read the persisted
368 objects from the DD tables, and replace the contents of the core
369 registry in the storage adapter
370
371 @param thd Thread context.
372*/
373bool sync_meta_data(THD *thd);
374
375/**
376 Update properties in the DD_properties table. Note that upon failure, we
377 will rollback, whereas upon success, commit will be delayed.
378
379 @param thd Thread context.
380 @param create_set A set of table names created/modified in
381 this version of DD.
382 @param remove_set A set of table names removed in this
383 version of DD.
384 @param target_table_schema_name Schema name in which the final changes are
385 required.
386
387 @return Upon failure, return true, otherwise false.
388*/
389bool update_properties(THD *thd, const std::set<String_type> *create_set,
390 const std::set<String_type> *remove_set,
391 const String_type &target_table_schema_name);
392
393/**
394 Updates the DD Version in the DD_properties table to the current version.
395 This function is used during initialize and during server upgrade.
396
397 @param thd Thread context.
398 @param is_dd_upgrade_57 Flag to indicate if it is an upgrade from 5.7.
399
400 @return Upon failure, return true, otherwise false.
401*/
402bool update_versions(THD *thd, bool is_dd_upgrade_57);
403
404} // namespace dd
405#endif // DD__BOOTSTRAPPER_INCLUDED
For each client connection we create a separate thread with THD serving as a thread/connection descri...
Definition: sql_lexer_thd.h:33
Definition: dictionary_impl.h:62
Types
Definition: system_registry.h:347
Definition: bootstrap.cc:69
bool setup_dd_objects_and_collations(THD *thd)
Initialization and verification of dictionary objects after upgrade, similar to what is done after no...
Definition: bootstrapper.cc:980
bool initialize(THD *thd)
Initialize the dictionary while starting the server for the first time.
Definition: bootstrapper.cc:877
bool DDSE_dict_init(THD *thd, dict_init_mode_t dict_init_mode, uint version)
Initialize InnoDB for.
Definition: bootstrapper.cc:726
void recover_innodb_upon_upgrade(THD *thd)
This function is used in case of crash during upgrade.
Definition: bootstrapper.cc:957
bool restart(THD *thd)
Initialize the dictionary while restarting the server.
Definition: bootstrapper.cc:910
bool initialize_dictionary(THD *thd, bool is_dd_upgrade_57, Dictionary_impl *d)
Create mysql schema.
Definition: bootstrapper.cc:820
bool store_plugin_IS_table_metadata(THD *thd)
Iterate through all the plugins, and store IS table meta data into dictionary, once during MySQL serv...
The version of the current data dictionary table definitions.
Definition: dictionary_client.h:42
void store_predefined_tablespace_metadata(THD *thd)
This function creates the meta data of the predefined tablespaces.
Definition: bootstrapper.cc:1017
bool update_versions(THD *thd, bool is_dd_upgrade_57)
Updates the DD Version in the DD_properties table to the current version.
Definition: bootstrapper.cc:1723
bool create_tables(THD *thd, const std::set< String_type > *create_set)
Execute SQL statements to create the DD tables.
Definition: bootstrapper.cc:1246
bool initialize_dd_properties(THD *thd)
During –initialize, we create the dd_properties table.
Definition: bootstrapper.cc:1074
bool create_dd_schema(THD *thd)
Executes SQL queries to create and use the dictionary schema.
Definition: bootstrapper.cc:1064
Char_string_template< String_type_allocator > String_type
Definition: string_type.h:50
bool sync_meta_data(THD *thd)
Acquire the DD schema, tablespace and table objects.
Definition: bootstrapper.cc:1301
bool update_properties(THD *thd, const std::set< String_type > *create_set, const std::set< String_type > *remove_set, const String_type &target_table_schema_name)
Update properties in the DD_properties table.
Definition: bootstrapper.cc:1610
bool is_non_inert_dd_or_ddse_table(System_tables::Types table_type)
Predicate to check if a table type is a non-inert DD or a DDSE table.
Definition: bootstrapper.cc:1239
required uint64 version
Definition: replication_group_member_actions.proto:40
dict_init_mode_t
Mode for initializing the data dictionary.
Definition: handler.h:1858
unsigned int uint
Definition: uca-dump.cc:29