MySQL 8.2.0
Source Code Documentation
log0files_capacity.h
Go to the documentation of this file.
1/*****************************************************************************
2
3Copyright (c) 2019, 2023, Oracle and/or its affiliates.
4
5This program is free software; you can redistribute it and/or modify it under
6the terms of the GNU General Public License, version 2.0, as published by the
7Free Software Foundation.
8
9This program is also distributed with certain software (including but not
10limited to OpenSSL) that is licensed under separate terms, as designated in a
11particular file or component or in included license documentation. The authors
12of MySQL hereby grant you an additional permission to link the program and
13your derivative works with the separately licensed software that they have
14included with MySQL.
15
16This program is distributed in the hope that it will be useful, but WITHOUT
17ANY WARRANTY; without even the implied warranty of MERCHANTABILITY or FITNESS
18FOR A PARTICULAR PURPOSE. See the GNU General Public License, version 2.0,
19for more details.
20
21You should have received a copy of the GNU General Public License along with
22this program; if not, write to the Free Software Foundation, Inc.,
2351 Franklin St, Fifth Floor, Boston, MA 02110-1301 USA
24
25*****************************************************************************/
26
27/**************************************************/ /**
28 @file include/log0files_capacity.h
29
30 Redo log management of capacity.
31
32 *******************************************************/
33
34#ifndef log0files_capacity_h
35#define log0files_capacity_h
36
37/* Log_files_dict */
38#include "log0files_dict.h"
39
40/* log_t&, lsn_t */
41#include "log0types.h"
42
43/* os_offset_t */
44#include "os0file.h"
45
46/** Responsible for the redo log capacity computations. Computes size for the
47next log file that will be created. Tracks the redo resize operation when the
48innodb_redo_log_capacity gets changed. Computes maximum ages for dirty pages,
49which are then used by page cleaner coordinator.
50
51@remarks When downsize is started, the limits for ages are decreased, forcing
52page cleaners to flush more dirty pages then. File sizes for new redo files
53are adjusted accordingly, so they could always be effectively used to hold
54the whole existing redo log data (for the given current logical size). */
56 public:
57 /** Initialize on discovered set of redo log files (empty set if new redo
58 is being created).
59 @param[in] files in-memory dictionary of existing files
60 @param[in] current_logical_size current logical size of data in redo,
61 or 0 if new redo is being created
62 @param[in] current_checkpoint_age current checkpoint age */
63 void initialize(const Log_files_dict &files, lsn_t current_logical_size,
64 lsn_t current_checkpoint_age);
65
66 /** Updates all internal limits according to the provided parameters.
67 If there are any values outside this class, on which computations of
68 limits depend on, they should be explicitly provided here, except the
69 server variables (srv_thread_concurrency, srv_redo_log_capacity_used).
70 @param[in] files in-memory dictionary of files
71 @param[in] current_logical_size current logical size of data
72 in the redo log; it depends
73 directly on the oldest redo
74 log consumer
75 @param[in] current_checkpoint_age current checkpoint age */
76 void update(const Log_files_dict &files, lsn_t current_logical_size,
77 lsn_t current_checkpoint_age);
78
79 /** @return true iff resize-down is pending */
80 bool is_resizing_down() const;
81
82 /** Provides maximum limitation for space occupied on disk.
83 @note This value changes only during calls to @see update or @see initialize.
84 @return maximum allowed size on disk, in bytes */
86
87 /** If a redo downsize is in progress, it is the targeted value for the
88 current_physical_capacity() (is equal if there is no resize in progress).
89 It is set to srv_redo_log_capacity_used when @see update is called.
90 @note This value changes only during calls to @see update or @see initialize.
91 @return targeted physical capacity, in bytes */
93
94 /** Soft limit for logical capacity of the redo log. When the log writer
95 exceeds this limitation, all user threads are paused during log_free_check()
96 calls and message is emitted to the error log. The log writer can still
97 continue to write until it reaches the hard limit for logical capacity
98 (value returned by hard_logical_capacity()).
99 @note This value changes only during calls to @see update or @see initialize.
100 @return limitation for free space in the redo log for user threads */
102
103 /** Hard limit for logical capacity of the redo log. This limitation includes
104 "extra_writer_margin" that belongs to the log writer thread. The log writer
105 does not to exceed this limit. If space isn't reclaimed after 1 sec wait,
106 it writes only as much as possible or crashes the InnoDB.
107 @note This value changes only during calls to @see update or @see initialize.
108 @return limitation for free space in the redo log for the log writer thread */
110
111 /** Once checkpoint age exceeds this value, the flushing of pages starts to
112 be adaptive. The adaptive page flushing is becoming more and more aggressive
113 in the following range: adaptive_flush_min_age()..adaptive_flush_max_age().
114 @note This value changes only during calls to @see update or @see initialize.
115 @note Note that it must hold:
116 adaptive_flush_min_age() < adaptive_flush_max_age() <=
117 soft_logical_capacity().
118 @remarks
119 The diagram below shows how flushing / checkpointing becomes more aggressive
120 when the age of the oldest modified page gets increased:
121
122 adaptive_flush_min_age adaptive_flush_max_age aggressive_checkpoint_min_age
123 | | |
124 -------!------------------------!---------------------!----------------->age
125 regular adaptive flushing aggressive flushing aggr. checkpoints
126 @return limitation to start adaptive flushing */
128
129 /** Once checkpoint age exceeds that value, the flushing of pages is the most
130 aggressive possible since then. For more details @see adaptive_flush_min_age.
131 @note This value changes only during calls to @see update or @see initialize.
132 @return limitation to start furious flushing */
134
135 /** Once checkpoint age exceeds that value, the log checkpointer thread keeps
136 writing checkpoints aggressively (whatever the progress of last_checkpoint_lsn
137 would it make). Before that happens, checkpoints could be written periodically
138 (for more details @see adaptive_flush_min_age).
139 @note This value changes only during calls to @see update or @see initialize.
140 @note It holds: adaptive_flush_max_age() < aggressive_checkpoint_min_age().
141 @return limitation to start aggressive checkpointing */
143
144 /** Provides size of the next redo log file that will be created. The initial
145 value becomes set during a call to @see initialize. Since then, it changes
146 only when innodb_redo_log_capacity is changed, during a call to @see update.
147 @note Does not depend on whether the file actually might be created or not.
148 It is log_files_governor's responsibility not to exceed the physical capacity.
149 @remarks
150 The strategy used by the Log_files_capacity, guarantees that next redo log
151 file should always be possible to be created. That's because:
152 1. The next file size is always chosen as:
153 innodb_redo_log_capacity / LOG_N_FILES.
154 1. The logical capacity of the redo log is limited to:
155 (LOG_N_FILES - 2) / LOG_N_FILES * m_current_physical_capacity.
156 2. The m_current_physical_capacity is changed only after resize is finished,
157 and the resize is considered finished only when:
158 - all redo log files have size <= innodb_redo_log_capacity / LOG_N_FILES,
159 - and the logical size of the redo log can fit physical size of
160 LOG_N_FILES - 2 redo files, which guarantees that at most
161 LOG_N_FILES - 1 redo files will ever need to exist (consider scenario
162 in which oldest_lsn is at the very end of the oldest redo files and
163 newest_lsn is at the very beginning of the newest redo file if you
164 are curious why -2 is there instead of -1).
165 @return file size suggested for next log file to create */
167
168 /** Computes size of a next redo log file that would be chosen for a given
169 physical capacity.
170 @param[in] physical_capacity physical capacity assumed for computation
171 @return file size suggested for next log file to create */
172 static os_offset_t next_file_size(os_offset_t physical_capacity);
173
174 /** Provides margin which might be used ahead of the newest lsn to create
175 a next file earlier if needed (it will be created as unused redo file).
176 @note This value changes only during calls to @see update or @see initialize.
177 @return the maximum allowed margin ahead of the newest lsn to be reserved */
179
180 /** Computes margin which might be used ahead of the newest lsn to create
181 a next file earlier if needed (it will be created as unused redo file).
182 The computation is done for a given physical capacity.
183 @param[in] physical_capacity physical capacity assumed for computation
184 @return the maximum allowed margin ahead of the newest lsn to be reserved */
185 static lsn_t next_file_earlier_margin(os_offset_t physical_capacity);
186
187 /** Computes hard logical capacity, that corresponds to the provided
188 soft logical capacity of the redo log (@see soft_logical_capacity()).
189 @param[in] soft_logical_capacity logical capacity for user threads,
190 used in log_free_check() calls
191 @return hard logical capacity */
194
195 /** Computes soft logical capacity, that corresponds to the provided
196 hard logical capacity of the redo log (@see hard_logical_capacity()).
197 @param[in] hard_logical_capacity logical capacity for the log writer
198 @return soft logical capacity */
200
201 /** Computes hard logical capacity, that corresponds to the provided
202 physical capacity of the redo log (@see hard_logical_capacity()).
203 @param[in] physical_capacity physical capacity for the redo log
204 @return hard logical capacity */
206 os_offset_t physical_capacity);
207
208 /** Computes maximum age of dirty pages up to which there is no sync flush
209 enforced on page cleaners. This is a smaller value than soft logical capacity,
210 because sync flush must be started earlier than log_free_check() calls begin
211 to stop user threads.
212 @param[in] soft_logical_capacity logical capacity for user threads,
213 used in log_free_check() calls
214 @return maximum age of dirty pages before the sync flush is started */
217
218 /** Computes soft logical capacity, that corresponds to the provided
219 maximum age of dirty pages up to which there is no sync flush enforced
220 on page cleaners. This is a larger value than the provided maximum age,
221 because sync flush must be started earlier than log_free_check() calls
222 begin to stop user threads.
223 @param[in] adaptive_flush_max_age maximum age of dirty page without
224 sync flush started
225 @return soft logical capacity */
228
229 private:
230 /** @see m_exposed */
231 struct Exposed {
232 /** Value returned by @see soft_logical_capacity */
234
235 /** Value returned by @see hard_logical_capacity */
237
238 /** Value returned by @see adaptive_flush_min_age */
240
241 /** Value returned by @see adaptive_flush_max_age */
243
244 /** Value returned by @see aggressive_checkpoint_min_age */
246 };
247
248 /** Cache for values returned by getters in this object, which otherwise
249 would need to be computed on-demand. These values do not have impact on
250 state updates of this object.
251 @note Updated only during calls to @see initialize and @see update. */
253
254 /** This is limitation for space on disk we are never allowed to exceed.
255 This is the guard of disk space - current size of all log files on disk
256 is always not greater than this value.
257 @note Updated only during calls to @see initialize and @see update. */
259
260 /** Goal we are trying to achieve for m_current_physical_capacity when
261 resize operation is in progress, else: equal to m_current_physical_capacity.
262 During startup (when srv_is_being_started is true) it stays equal to the
263 m_current_physical_capacity (which is then computed for discovered log files).
264 After startup, it's set to srv_redo_log_capacity_used by calls to
265 @see update. */
267
268 /** Current resize direction. When user decides to resize down the redo log,
269 it becomes Log_resize_mode::RESIZING_DOWN until the resize is finished or
270 user decides to stop it (providing other capacity). Note, that resize is not
271 started during startup (when srv_is_being_started is true).
272 @note Updated only during calls to @see initialize and @see update. */
274
275 /** Cancels current resize operation immediately.
276 @remarks If the innodb_redo_log_capacity is changed when there is a previous
277 redo resize in progress, the previous resize is first cancelled. */
278 void cancel_resize();
279
280 /** Updates m_target_physical_capacity (reading srv_redo_log_capacity_used)
281 and possibly starts a new downsize operation. Might also update:
282 m_resize_mode, m_current_physical_capacity. */
283 void update_target();
284
285 /** Checks if target of the resize is reached, with regards to the criteria
286 based on the current logical size of the redo.
287 @param[in] current_logical_size current logical size of the redo log
288 @return true iff the target is reached */
289 bool is_target_reached_for_logical_size(lsn_t current_logical_size) const;
290
291 /** Checks if target of the resize is reached, with regards to the criteria
292 based on the current physical size of existing log files (excludes unused).
293 @param[in] current_physical_size total size of existing redo log files,
294 excluding unused (spare) files
295 @return true iff the target is reached */
297 os_offset_t current_physical_size) const;
298
299 /** Checks if target of the resize is reached, with regards to the criteria
300 based on the largest existing redo file.
301 @param[in] files in-memory dictionary of existing files
302 @return true iff the target is reached */
304
305 /** Checks if target of the resize is reached.
306 @param[in] files in-memory dictionary of existing files
307 @param[in] current_logical_size current logical size of the redo log
308 @return true iff the target is reached */
310 lsn_t current_logical_size) const;
311
312 /** Checks if target of the resize is reached with regards to all criteria
313 and updates the m_resize_mode, m_current_physical_capacity when that happens
314 (marking the resize operation as finished).
315 @param[in] files in-memory dictionary of existing files
316 @param[in] current_logical_size current logical size of the redo log */
318 lsn_t current_logical_size);
319
320 /** Updates value of server status variable: innodb_redo_log_resize_status. */
322
323 /** Updates values of server status variables:
324 innodb_redo_log_capacity_resized, innodb_redo_log_logical_size,
325 innodb_redo_log_physical_size, innodb_redo_log_resize_status.
326 @param[in] files in-memory dictionary of existing files
327 @param[in] current_logical_size current logical size of the redo log */
328 void update_status_variables(const Log_files_dict &files,
329 lsn_t current_logical_size);
330
331 /** Updates cached and exposed values related to the logical redo capacity:
332 - @see m_soft_logical_capacity
333 - @see m_hard_logical_capacity
334 - @see m_agressive_checkpoint_min_age
335 - @see m_adaptive_flush_min_age
336 - @see m_adaptive_flush_max_age
337 @param[in] current_logical_size current logical size of the redo log */
338 void update_exposed(lsn_t current_logical_size);
339
340 /** Computes suggested value for the current hard logical capacity.
341 @remarks This becomes non-trivial when the redo log is being resized down,
342 because this method is supposed to follow the checkpoint age then.
343 On the other hand, when the redo log is not being resized down, this method
344 computes the hard logical capacity by using simple math based on the current
345 physical capacity only (ie. ignoring the current checkpoint age).
346 @param[in] current_checkpoint_age current checkpoint age,
347 used only when resizing down
348 @return suggested value for current hard logical capacity */
349 lsn_t get_suggested_hard_logical_capacity(lsn_t current_checkpoint_age) const;
350};
351
352/** Retrieves limitations determined by the current state of log.m_capacity.
353These values are retrieved atomically (are consistent with each other).
354@param[in] log redo log
355@param[out] limit_for_free_check soft capacity of the redo decreased by
356 the current free check margin; this is
357 limit for size of redo until which the
358 log_free_check calls do not force waits
359@param[out] limit_for_dirty_page_age limit for the oldest dirty page until
360 which the async (adaptive) flushing is
361 not forced to be started (it might be
362 started if turned on explicitly by the
363 innodb_adaptive_flushing); note that
364 computation of this value include doing
365 the subtraction of the current log free
366 check margin */
368 lsn_t &limit_for_free_check,
369 lsn_t &limit_for_dirty_page_age);
370
371#endif /* !log0files_capacity_h */
Responsible for the redo log capacity computations.
Definition: log0files_capacity.h:55
void update_if_target_reached(const Log_files_dict &files, lsn_t current_logical_size)
Checks if target of the resize is reached with regards to all criteria and updates the m_resize_mode,...
Definition: log0files_capacity.cc:370
bool is_target_reached_for_physical_size(os_offset_t current_physical_size) const
Checks if target of the resize is reached, with regards to the criteria based on the current physical...
Definition: log0files_capacity.cc:341
lsn_t get_suggested_hard_logical_capacity(lsn_t current_checkpoint_age) const
Computes suggested value for the current hard logical capacity.
Definition: log0files_capacity.cc:454
lsn_t next_file_earlier_margin() const
Provides margin which might be used ahead of the newest lsn to create a next file earlier if needed (...
Definition: log0files_capacity.cc:553
bool is_target_reached_for_max_file_size(const Log_files_dict &files) const
Checks if target of the resize is reached, with regards to the criteria based on the largest existing...
Definition: log0files_capacity.cc:354
void update_exposed(lsn_t current_logical_size)
Updates cached and exposed values related to the logical redo capacity:
Definition: log0files_capacity.cc:474
static lsn_t hard_logical_capacity_for_physical(os_offset_t physical_capacity)
Computes hard logical capacity, that corresponds to the provided physical capacity of the redo log (.
Definition: log0files_capacity.cc:272
Log_resize_mode m_resize_mode
Current resize direction.
Definition: log0files_capacity.h:273
void initialize(const Log_files_dict &files, lsn_t current_logical_size, lsn_t current_checkpoint_age)
Initialize on discovered set of redo log files (empty set if new redo is being created).
Definition: log0files_capacity.cc:150
lsn_t soft_logical_capacity() const
Soft limit for logical capacity of the redo log.
Definition: log0files_capacity.cc:504
os_offset_t target_physical_capacity() const
If a redo downsize is in progress, it is the targeted value for the current_physical_capacity() (is e...
Definition: log0files_capacity.cc:524
Exposed m_exposed
Cache for values returned by getters in this object, which otherwise would need to be computed on-dem...
Definition: log0files_capacity.h:252
void update_status_variables(const Log_files_dict &files, lsn_t current_logical_size)
Updates values of server status variables: innodb_redo_log_capacity_resized, innodb_redo_log_logical_...
Definition: log0files_capacity.cc:388
bool is_resizing_down() const
Definition: log0files_capacity.cc:520
static lsn_t sync_flush_logical_capacity_for_soft(lsn_t soft_logical_capacity)
Computes maximum age of dirty pages up to which there is no sync flush enforced on page cleaners.
Definition: log0files_capacity.cc:446
static lsn_t soft_logical_capacity_for_hard(lsn_t hard_logical_capacity)
Computes soft logical capacity, that corresponds to the provided hard logical capacity of the redo lo...
Definition: log0files_capacity.cc:430
static lsn_t guess_soft_logical_capacity_for_sync_flush(lsn_t adaptive_flush_max_age)
Computes soft logical capacity, that corresponds to the provided maximum age of dirty pages up to whi...
Definition: log0files_capacity.cc:438
lsn_t aggressive_checkpoint_min_age() const
Once checkpoint age exceeds that value, the log checkpointer thread keeps writing checkpoints aggress...
Definition: log0files_capacity.cc:516
bool is_target_reached_for_logical_size(lsn_t current_logical_size) const
Checks if target of the resize is reached, with regards to the criteria based on the current logical ...
Definition: log0files_capacity.cc:334
void update_target()
Updates m_target_physical_capacity (reading srv_redo_log_capacity_used) and possibly starts a new dow...
Definition: log0files_capacity.cc:225
void update(const Log_files_dict &files, lsn_t current_logical_size, lsn_t current_checkpoint_age)
Updates all internal limits according to the provided parameters.
Definition: log0files_capacity.cc:189
static lsn_t guess_hard_logical_capacity_for_soft(lsn_t soft_logical_capacity)
Computes hard logical capacity, that corresponds to the provided soft logical capacity of the redo lo...
Definition: log0files_capacity.cc:422
bool is_target_reached_for_resizing_down(const Log_files_dict &files, lsn_t current_logical_size) const
Checks if target of the resize is reached.
Definition: log0files_capacity.cc:362
os_offset_t m_target_physical_capacity
Goal we are trying to achieve for m_current_physical_capacity when resize operation is in progress,...
Definition: log0files_capacity.h:266
os_offset_t current_physical_capacity() const
Provides maximum limitation for space occupied on disk.
Definition: log0files_capacity.cc:528
lsn_t hard_logical_capacity() const
Hard limit for logical capacity of the redo log.
Definition: log0files_capacity.cc:500
void cancel_resize()
Cancels current resize operation immediately.
Definition: log0files_capacity.cc:214
lsn_t adaptive_flush_min_age() const
Once checkpoint age exceeds this value, the flushing of pages starts to be adaptive.
Definition: log0files_capacity.cc:508
os_offset_t m_current_physical_capacity
This is limitation for space on disk we are never allowed to exceed.
Definition: log0files_capacity.h:258
void update_resize_status_variable()
Updates value of server status variable: innodb_redo_log_resize_status.
Definition: log0files_capacity.cc:405
lsn_t adaptive_flush_max_age() const
Once checkpoint age exceeds that value, the flushing of pages is the most aggressive possible since t...
Definition: log0files_capacity.cc:512
os_offset_t next_file_size() const
Provides size of the next redo log file that will be created.
Definition: log0files_capacity.cc:532
In-memory dictionary of meta data of existing log files.
Definition: log0files_dict.h:56
void log_files_capacity_get_limits(const log_t &log, lsn_t &limit_for_free_check, lsn_t &limit_for_dirty_page_age)
Retrieves limitations determined by the current state of log.m_capacity.
Definition: log0files_capacity.cc:123
In-memory dictionary of log files (keeps their meta data).
Redo log basic types.
Log_resize_mode
Direction of resize operation.
Definition: log0types.h:192
@ NONE
No pending resize.
std::atomic< lsn_t > atomic_lsn_t
Alias for atomic based on lsn_t.
Definition: log0types.h:81
uint64_t lsn_t
Type used for all log sequence number storage and arithmetic.
Definition: log0types.h:62
The interface to the operating system file io.
uint64_t os_offset_t
File offset in bytes.
Definition: os0file.h:86
Definition: log0files_capacity.h:231
atomic_lsn_t m_agressive_checkpoint_min_age
Value returned by.
Definition: log0files_capacity.h:245
atomic_lsn_t m_hard_logical_capacity
Value returned by.
Definition: log0files_capacity.h:236
atomic_lsn_t m_soft_logical_capacity
Value returned by.
Definition: log0files_capacity.h:233
atomic_lsn_t m_adaptive_flush_max_age
Value returned by.
Definition: log0files_capacity.h:242
atomic_lsn_t m_adaptive_flush_min_age
Value returned by.
Definition: log0files_capacity.h:239
Redo log - single data structure with state of the redo log system.
Definition: log0sys.h:76