MySQL 9.0.0
Source Code Documentation
bootstrapper.h
Go to the documentation of this file.
1/* Copyright (c) 2014, 2024, Oracle and/or its affiliates.
2
3 This program is free software; you can redistribute it and/or modify
4 it under the terms of the GNU General Public License, version 2.0,
5 as published by the Free Software Foundation.
6
7 This program is designed to work with certain software (including
8 but not limited to OpenSSL) that is licensed under separate terms,
9 as designated in a particular file or component or in included license
10 documentation. The authors of MySQL hereby grant you an additional
11 permission to link the program and your derivative works with the
12 separately licensed software that they have either included with
13 the program or referenced in the documentation.
14
15 This program is distributed in the hope that it will be useful,
16 but WITHOUT ANY WARRANTY; without even the implied warranty of
17 MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
18 GNU General Public License, version 2.0, for more details.
19
20 You should have received a copy of the GNU General Public License
21 along with this program; if not, write to the Free Software
22 Foundation, Inc., 51 Franklin St, Fifth Floor, Boston, MA 02110-1301 USA */
23
24#ifndef DD__BOOTSTRAPPER_INCLUDED
25#define DD__BOOTSTRAPPER_INCLUDED
26
27#include <sys/types.h>
28
29#include "sql/dd/impl/system_registry.h" // dd::System_tables
30#include "sql/dd/string_type.h" // dd::String_type
31#include "sql/handler.h" // dict_init_mode_t
32
33class THD;
34
35/**
36 Data dictionary initialization.
37
38 The data dictionary is initialized whenever the mysqld process starts.
39 We distinguish between the first time start and the subsequent normal
40 restarts/upgrades, as explained below. However, there are three main
41 design principles that should be elaborated first.
42
43 1. Two-step process: The dictionary initialization is implemented as
44 a two step process. First, scaffolding is built to prepare the
45 synchronization with persistent storage, then, the actual synchronization
46 is done. The way this is done depends on the context, and is different
47 for first time start and the subsequent restarts.
48
49 2. Use SQL: The initialization uses SQL to build the scaffolding. This
50 means that we execute SQL statements to create the dictionary tables.
51 Since this is done at a stage where the physical tables either do not
52 exist yet, or are not known, we must instrument the DDL execution to
53 create the physical counterpart of the tables only on first time start.
54 The goal is to keep the instrumentation at a minimum.
55
56 3. Fake caching: As a consequence of keeping instrumentation at a minimum,
57 we provide uniform behavior of the caching layer in the data dictionary
58 also in the scaffolding phase. This means that as seen from the outside,
59 dictionary objects can be retrieved from the cache. Internally, below the
60 caching layer, the objects are only kept in a separate buffer until all
61 the required scaffolding is built. At that point, we can start using the
62 underlying physical tables, depending on the circumstances:
63
64 - For first time start (initialization), we can flush the meta data
65 generated in the scaffolding phase, to the DD tables.
66 - For ordinary restart, we can use the scaffolding to open the physical
67 tables, and then sync up the real meta data that is stored persistently.
68 - For upgrade, we first build scaffolding based on the actual DD tables,
69 then we create the target DD tables, migrate the meta data from the old
70 to the new tables, and finally switch from old to new tables
71 atomically by means of DML on the DD tables. This means that we update
72 the schema ids in the DD tables directly instead of executing
73 'RENAME TABLE', which would do auto commit and thus break atomicity.
74
75 After the scaffolding has been flushed or synced, what should be left is
76 a collection of the core DD meta data objects. This collection is located
77 in the storage adapter, and allows the DD cache to evict core DD objects
78 in the same way as other DD objects.
79
80 Please note that dictionary initialization is only a small part of server
81 initialization. There is a lot going on before and after dictionary
82 initialization while starting the server.
83
84 Please see more elaborated descriptions for the initialize() and restart()
85 methods below.
86*/
87
88namespace dd {
89class Dictionary_impl;
90
91namespace bootstrap {
92
93/**
94 Initialize the dictionary while starting the server for the first time.
95
96 At this point, the DDSE has been initialized as a normal plugin. The
97 dictionary initialization proceeds as follows:
98
99 1. Preparation phase
100
101 1.1 Call dict_init() to initialize the DDSE. This will make the predefined
102 tablespaces be created physically, and their meta data be returned to
103 the SQL layer along with the meta data for the DD tables required by
104 the DDSE. The tables are not yet created physically.
105 1.2 Prepare the dd::Tablespace objects reflecting the predefined tablespace
106 objects and add them to the core registry in the storage adapter.
107
108 2. Scaffolding phase
109
110 2.1 Create and use the dictionary schema by executing SQL statements.
111 The schema is created physically since this is the first time start,
112 and the meta data is generated and stored in the core registry of
113 the storage adapter without being written to disk.
114 2.2 Create tables by executing SQL statements. Like for the schema, the
115 tables are created physically, and the meta data is generated
116 and stored in the core registry without being written to disk.
117 This is done to prepare enough meta data to actually be able to
118 open the DD tables.
119
120 3. Synchronization phase
121
122 3.1 Store meta data for the DD schema, tablespace and tables, i.e., the DD
123 objects that were generated in the scaffolding phase, and make sure the
124 IDs are maintained when the objects are stored.
125 3.2 Populate the DD tables which have some predefined static contents to
126 be inserted. This is, e.g., relevant for the 'catalogs' table, which
127 only has a single default entry in it. Dynamic contents is added in
128 other ways, e.g. by storing generated DD objects (see above) or by
129 inserting data from other sources (see re-population of character sets
130 in the context of server restart below).
131 3.3 Store various properties of the DD tables, including the SE private data,
132 a representation of the DDL statement used to create the table etc.
133 3.4 Verify that the dictionary objects representing the core DD table meta
134 data are present in the core registry of the storage adapter. If an
135 object representing the meta data of a core DD table is not available,
136 then we loose access to the DD tables, and we will not be able to handle
137 cache misses or updates to the meta data.
138 3.5 Update the version numbers that are stored, e.g. the DD version and the
139 current mysqld server version.
140
141 @param thd Thread context.
142
143 @return Upon failure, return true, otherwise false.
144*/
145
146bool initialize(THD *thd);
147
148/**
149 Initialize the dictionary while restarting the server.
150
151 At this point, the DDSE has been initialized as a normal plugin. The
152 dictionary initialization proceeds as follows:
153
154 1. Preparation phase
155
156 1.1 Call dict_init() to initialize the DDSE. This will retrieve the meta data
157 of the predefined tablespaces and the DD tables required by the DDSE.
158 Both the tables and the tablespaces are already created physically, the
159 point here is just to get hold of enough meta data to start using the DD.
160 1.2 Prepare the dd::Tablespace objects reflecting the predefined tablespace
161 objects and add them to the core registry in the storage adapter.
162
163 2. Scaffolding phase
164
165 2.1 Create and use the dictionary schema by executing SQL statements.
166 The schema is not created physically, but the meta data is generated
167 and stored in the core registry without being written to disk.
168 2.2 Create tables by executing SQL statements. Like for the schema, the
169 tables are not created physically, but the meta data is generated
170 and stored in the core registry without being written to disk.
171 This is done to prepare enough meta data to actually be able to
172 open the DD tables. The SQL DDL statements are either retrieved from
173 the table definitions that are part of the server binary (for restart),
174 or from one of the DD tables (for upgrade).
175
176 3. Synchronization phase
177
178 3.1 Read meta data for the DD tables from the DD tables. Here, we use the
179 meta data from the scaffolding phase for the schema, tablespace and the
180 DD tables to open the physical DD tables. We read the stored objects,
181 and update the in-memory copies in the core registry with the real meta
182 data from the objects that are retrieved form persistent storage. Finally,
183 we flush the tables to empty the table definition cache to make sure the
184 table share structures for the DD tables are re-created based on the
185 actual meta data that was read from disk rather than the temporary meta
186 data from the scaffolding phase.
187 3.2 If this is a restart with a new DD version, we must upgrade the DD
188 tables. In that case, we create the new target DD tables in a temporary
189 schema, migrate the meta data to the new tables, and then do DML on the
190 DD tables to make sure the new DD tables will be used instead of the old
191 ones. This DML involves changing the schema ids directly in the DD tables,
192 and updating the meta data stored in the 'dd_properties' DD table.
193 This will make sure the switch from the old to the new tables is
194 atomic. After this is done, we will reset the DD cache and start over
195 the initialization from step 1.2. Then, the new DD tables will be used,
196 and a normal restart will be done.
197 3.3 Re-populate character sets and collations: The character set and
198 collation information is read from files and added to a server
199 internal data structure when the server starts. This data structure is,
200 in turn, used to populate the corresponding DD tables. The tables must
201 be re-populated on each server start if new character sets or collations
202 have been added. However, we can not do this if in read only mode.
203 3.4 Verify that the dictionary objects representing the core DD table meta
204 data are present in the core registry of the storage adapter. If an
205 object representing the meta data of a core DD table is not available,
206 then we loose access to the DD tables, and we will not be able to handle
207 cache misses or updates to the meta data.
208 3.5 If an upgrade was done, the persistent version numbers are updated,
209 e.g. the DD version and the current mysqld server version.
210
211 @param thd Thread context.
212
213 @return Upon failure, return true, otherwise false.
214*/
215bool restart_dictionary(THD *thd);
216
217/**
218 Iterate through all the plugins, and store IS table meta data
219 into dictionary, once during MySQL server bootstrap.
220
221 @param thd Thread context.
222
223 @return Upon failure, return true, otherwise false.
224*/
226
227/**
228 Initialization and verification of dictionary objects
229 after upgrade, similar to what is done after normal server
230 restart.
231
232 @param thd Thread context
233*/
235
236/**
237 This function is used in case of crash during upgrade.
238 It tries to initialize dictionary and calls DDSE_dict_recover.
239 InnoDB should do the recovery and empty undo log. Upgrade
240 process will do the cleanup and exit.
241
242 @param thd Thread context.
243*/
245
246/**
247 Initialize InnoDB for
248 - creating new data directory : InnoDB creates system tablespace and
249 dictionary tablespace.
250 - normal server restart. : Verifies existence of system and dictionary
251 tablespaces.
252 - in place upgrade : Verifies existence of system tablespace and
253 create dictionary tablespace.
254
255 @param thd Thread context.
256 @param dict_init_mode mode to initialize InnoDB
257 @param version Dictionary version.
258
259 @return Upon failure, return true, otherwise false.
260*/
261bool DDSE_dict_init(THD *thd, dict_init_mode_t dict_init_mode, uint version);
262
263/**
264 Create mysql schema. Create dictionary tables inside InnoDB.
265 Create entry for dictionary tables inside dictionary tables.
266 Add hard coded data to dictionary tables.
267 Create Foreign key constraint on dictionary tables.
268
269 This function is used in both cases, new data directory initialization
270 and in place upgrade.
271
272 @param thd Thread context.
273 @param d Dictionary instance
274
275 @return Upon failure, return true, otherwise false.
276
277*/
279
280} // namespace bootstrap
281
282/**
283 This function creates the meta data of the predefined tablespaces.
284
285 @param thd Thread context.
286*/
288
289/**
290 Executes SQL queries to create and use the dictionary schema.
291
292 @param thd Thread context.
293*/
294bool create_dd_schema(THD *thd);
295
296/**
297 During --initialize, we create the dd_properties table. During restart,
298 create its meta data, and use it to open and read its contents.
299
300 @param thd Thread context.
301*/
303
304/**
305 Predicate to check if a table type is a non-inert DD or a DDSE table.
306
307 @param table_type Type as defined in the System_tables registry.
308 @returns true if the table is a non-inert DD or DDSE table,
309 false otherwise
310*/
312
313/**
314 Execute SQL statements to create the DD tables.
315
316 The tables created here will be a subset of the target DD tables for this
317 DD version. This function is called in the following four cases:
318
319 1. When a server is started the first time, with --initialize. Then, we
320 will iterate over all target tables and create them. This will also
321 make them be created physically in the DDSE.
322 2. When a server is restarted, and the data directory contains a dictionary
323 with the same DD version as the target DD version of the starting server.
324 In this case, we will iterate over all target tables and create them,
325 using the target table SQL DDL definitions. This is done only to create
326 the meta data, though; the tables will not be created physically in the
327 DDSE since they already exist. But we need to create the meta data to be
328 able top open them.
329 3. When a server is restarted, and the data directory was last used by a
330 more recent MRU within the same GA with a higher target DD version.
331 This is considered a 'minor downgrade'. In this case, the restarting
332 server will continue to run using the more recent DD version. This is
333 possible since only a subset of DD changes are allowed in a DD upgrade
334 that can also be downgraded. However, it means that we must create the
335 meta data reflecting the *actual* tables, not the target tables. So in
336 this case, we iterate over the target tables, but execute the DDL
337 statements of the actual tables. We get these statements from the
338 'dd_properties' table, where the more recent MRU has stored them.
339 4. When a server is restarted, and the data directory was last used by a
340 server with a DD version from which the starting server can upgrade. In
341 this case, this function is called three times:
342
343 - The first time, we need to create the meta data reflecting the actual
344 tables in the persistent DD. This is needed to be able to open the DD
345 tables and read the data. This is similar to use case 3. above.
346 - The second time, we create the tables that are modified in the new DD
347 version. Here, the tables are also created physically in the DDSE.
348 In this case, the 'create_set' specifies which subset of the target
349 tables should be created. After this stage, we replace the meta data
350 in 'dd_properties' by new meta data reflecting the modified tables. We
351 also replace the version numbers to make sure a new restart will use
352 the upgraded DD.
353 - The third time, we do the same as in case 2 above. This is basically
354 the same as a shutdown and restart of the server after upgrade was
355 completed.
356
357 @param thd Thread context.
358 @param create_set Subset of the target tables which should be created
359 during upgrade.
360
361 @returns false if success, otherwise true.
362*/
363bool create_tables(THD *thd, const std::set<String_type> *create_set);
364
365/**
366 Acquire the DD schema, tablespace and table objects. Read the persisted
367 objects from the DD tables, and replace the contents of the core
368 registry in the storage adapter
369
370 @param thd Thread context.
371*/
372bool sync_meta_data(THD *thd);
373
374/**
375 Update properties in the DD_properties table. Note that upon failure, we
376 will rollback, whereas upon success, commit will be delayed.
377
378 @param thd Thread context.
379 @param create_set A set of table names created/modified in
380 this version of DD.
381 @param remove_set A set of table names removed in this
382 version of DD.
383 @param target_table_schema_name Schema name in which the final changes are
384 required.
385
386 @return Upon failure, return true, otherwise false.
387*/
388bool update_properties(THD *thd, const std::set<String_type> *create_set,
389 const std::set<String_type> *remove_set,
390 const String_type &target_table_schema_name);
391
392/**
393 Updates the DD Version in the DD_properties table to the current version.
394 This function is used during initialize and during server upgrade.
395
396 @param thd Thread context.
397
398 @return Upon failure, return true, otherwise false.
399*/
400bool update_versions(THD *thd);
401
402} // namespace dd
403#endif // DD__BOOTSTRAPPER_INCLUDED
For each client connection we create a separate thread with THD serving as a thread/connection descri...
Definition: sql_lexer_thd.h:36
Definition: dictionary_impl.h:63
Types
Definition: system_registry.h:348
Definition: bootstrap.cc:71
bool setup_dd_objects_and_collations(THD *thd)
Initialization and verification of dictionary objects after upgrade, similar to what is done after no...
Definition: bootstrapper.cc:995
bool restart_dictionary(THD *thd)
Initialize the dictionary while restarting the server.
Definition: bootstrapper.cc:901
bool initialize(THD *thd)
Initialize the dictionary while starting the server for the first time.
Definition: bootstrapper.cc:868
bool DDSE_dict_init(THD *thd, dict_init_mode_t dict_init_mode, uint version)
Initialize InnoDB for.
Definition: bootstrapper.cc:733
bool initialize_dictionary(THD *thd, Dictionary_impl *d)
Create mysql schema.
Definition: bootstrapper.cc:837
void recover_innodb_upon_upgrade(THD *thd)
This function is used in case of crash during upgrade.
Definition: bootstrapper.cc:972
bool store_plugin_IS_table_metadata(THD *thd)
Iterate through all the plugins, and store IS table meta data into dictionary, once during MySQL serv...
The version of the current data dictionary table definitions.
Definition: dictionary_client.h:43
void store_predefined_tablespace_metadata(THD *thd)
This function creates the meta data of the predefined tablespaces.
Definition: bootstrapper.cc:1032
bool create_tables(THD *thd, const std::set< String_type > *create_set)
Execute SQL statements to create the DD tables.
Definition: bootstrapper.cc:1377
bool update_versions(THD *thd)
Updates the DD Version in the DD_properties table to the current version.
Definition: bootstrapper.cc:1854
bool initialize_dd_properties(THD *thd)
During –initialize, we create the dd_properties table.
Definition: bootstrapper.cc:1136
bool create_dd_schema(THD *thd)
Executes SQL queries to create and use the dictionary schema.
Definition: bootstrapper.cc:1079
Char_string_template< String_type_allocator > String_type
Definition: string_type.h:51
bool sync_meta_data(THD *thd)
Acquire the DD schema, tablespace and table objects.
Definition: bootstrapper.cc:1432
bool update_properties(THD *thd, const std::set< String_type > *create_set, const std::set< String_type > *remove_set, const String_type &target_table_schema_name)
Update properties in the DD_properties table.
Definition: bootstrapper.cc:1741
bool is_non_inert_dd_or_ddse_table(System_tables::Types table_type)
Predicate to check if a table type is a non-inert DD or a DDSE table.
Definition: bootstrapper.cc:1370
required uint64 version
Definition: replication_group_member_actions.proto:41
dict_init_mode_t
Mode for initializing the data dictionary.
Definition: handler.h:1874