MySQL 9.0.0
Source Code Documentation
lock0latches.h
Go to the documentation of this file.
1/*****************************************************************************
2
3Copyright (c) 2020, 2024, Oracle and/or its affiliates.
4
5This program is free software; you can redistribute it and/or modify it under
6the terms of the GNU General Public License, version 2.0, as published by the
7Free Software Foundation.
8
9This program is designed to work with certain software (including
10but not limited to OpenSSL) that is licensed under separate terms,
11as designated in a particular file or component or in included license
12documentation. The authors of MySQL hereby grant you an additional
13permission to link the program and your derivative works with the
14separately licensed software that they have either included with
15the program or referenced in the documentation.
16
17This program is distributed in the hope that it will be useful, but WITHOUT
18ANY WARRANTY; without even the implied warranty of MERCHANTABILITY or FITNESS
19FOR A PARTICULAR PURPOSE. See the GNU General Public License, version 2.0,
20for more details.
21
22You should have received a copy of the GNU General Public License along with
23this program; if not, write to the Free Software Foundation, Inc.,
2451 Franklin St, Fifth Floor, Boston, MA 02110-1301 USA
25
26*****************************************************************************/
27#ifndef lock0latches_h
28#define lock0latches_h
29
30#include "dict0types.h"
31#include "sync0sharded_rw.h"
32#include "ut0cpu_cache.h"
33#include "ut0mutex.h"
34
35/* Forward declarations */
36struct dict_table_t;
37class page_id_t;
38
39namespace locksys {
40/**
41The class which handles the logic of latching of lock_sys queues themselves.
42The lock requests for table locks and record locks are stored in queues, and to
43allow concurrent operations on these queues, we need a mechanism to latch these
44queues in safe and quick fashion.
45In the past we had a single latch which protected access to all of them.
46Now, we use more granular approach.
47In extreme, one could imagine protecting each queue with a separate latch.
48To avoid having too many latch objects, and having to create and remove them on
49demand, we use a more conservative approach.
50The queues are grouped into a fixed number of shards, and each shard is
51protected by its own mutex.
52
53However, there are several rare events in which we need to "stop the world" -
54latch all queues, to prevent any activity inside lock-sys.
55One way to accomplish this would be to simply latch all the shards one by one,
56but it turns out to be way too slow in debug runs, where such "stop the world"
57events are very frequent due to lock_sys validation.
58
59To allow for efficient latching of everything, we've introduced a global_latch,
60which is a read-write latch.
61Most of the time, we operate on one or two shards, in which case it is
62sufficient to s-latch the global_latch and then latch shard's mutex.
63For the "stop the world" operations, we x-latch the global_latch, which prevents
64any other thread from latching any shard.
65
66However, it turned out that on ARM architecture, the default implementation of
67read-write latch (rw_lock_t) is too slow because increments and decrements of
68the number of s-latchers is implemented as read-update-try-to-write loop, which
69means multiple threads try to modify the same cache line disrupting each other.
70Therefore, we use a sharded version of read-write latch (Sharded_rw_lock), which
71internally uses multiple instances of rw_lock_t, spreading the load over several
72cache lines. Note that this sharding is a technical internal detail of the
73global_latch, which for all other purposes can be treated as a single entity.
74
75This his how this conceptually looks like:
76```
77 [ global latch ]
78 |
79 v
80 [table shard 1] ... [table shard 512] [page shard 1] ... [page shard 512]
81
82```
83
84So, for example access two queues for two records involves following steps:
851. s-latch the global_latch
862. identify the 2 pages to which the records belong
873. identify the lock_sys 2 hash cells which contain the queues for given pages
884. identify the 2 shard ids which contain these two cells
895. latch mutexes for the two shards in the order of their addresses
90
91All of the steps above (except 2, as we usually know the page already) are
92accomplished with the help of single line:
93
94 locksys::Shard_latches_guard guard{*block_a, *block_b};
95
96And to "stop the world" one can simply x-latch the global latch by using:
97
98 locksys::Global_exclusive_latch_guard guard{};
99
100This class does not expose too many public functions, as the intention is to
101rather use friend guard classes, like the Shard_latches_guard demonstrated.
102*/
103class Latches {
104 private:
105 using Lock_mutex = ib_mutex_t;
106
107 /** A helper wrapper around Shared_rw_lock which simplifies:
108 - lifecycle by providing constructor and destructor, and
109 - s-latching and s-unlatching by keeping track of the shard id used for
110 spreading the contention.
111 There must be at most one instance of this class (the one in the lock_sys), as
112 it uses thread_local-s to remember which shard of sharded rw lock was used by
113 this thread to perform s-latching (so, hypothetical other instances would
114 share this field, overwriting it and leading to errors). */
116 /** The actual rw_lock implementation doing the heavy lifting */
118
119 /** The value used for m_shard_id to indicate that current thread did not
120 s-latch any of the rw_lock's shards */
121 static constexpr size_t NOT_IN_USE = std::numeric_limits<size_t>::max();
122
123 /** The id of the rw_lock's shard which this thread has s-latched, or
124 NOT_IN_USE if it has not s-latched any*/
125 static thread_local size_t m_shard_id;
126
127 public:
130 bool try_x_lock(ut::Location location) {
131 return rw_lock.try_x_lock(location);
132 }
133 /** Checks if there is a thread requesting an x-latch waiting for our
134 thread to release its s-latch.
135 Must be called while holding an s-latch.
136 @return true iff there is an x-latcher blocked by our s-latch. */
140 }
141 void x_lock(ut::Location location) { rw_lock.x_lock(location); }
143 void s_lock(ut::Location location) {
145 m_shard_id = rw_lock.s_lock(location);
146 }
147 void s_unlock() {
151 }
152#ifdef UNIV_DEBUG
153 bool x_own() const { return rw_lock.x_own(); }
154 bool s_own() const {
156 }
157#endif
158 };
159
161
162 /** Number of page shards, and also number of table shards.
163 Must be a power of two */
164 static constexpr size_t SHARDS_COUNT = 512;
165
166 /*
167 Functions related to sharding by page (containing records to lock).
168
169 This must be done in such a way that two pages which share a single lock
170 queue fall into the same shard. We accomplish this by reusing hash function
171 used to determine lock queue, and then group multiple queues into single
172 shard.
173 */
175 /** Each shard is protected by a separate mutex. Mutexes are padded to avoid
176 false sharing issues with cache. */
178 /**
179 Identifies the page shard which contains record locks for records from the
180 given page.
181 @param[in] page_id The space_id and page_no of the page
182 @return Integer in the range [0..lock_sys_t::SHARDS_COUNT)
183 */
184 static size_t get_shard(const page_id_t &page_id);
185
186 public:
187 Page_shards();
188 ~Page_shards();
189
190 /**
191 Returns the mutex which (together with the global latch) protects the page
192 shard which contains record locks for records from the given page.
193 @param[in] page_id The space_id and page_no of the page
194 @return The mutex responsible for the shard containing the page
195 */
196 const Lock_mutex &get_mutex(const page_id_t &page_id) const;
197
198 /**
199 Returns the mutex which (together with the global latch) protects the page
200 shard which contains record locks for records from the given page.
201 @param[in] page_id The space_id and page_no of the page
202 @return The mutex responsible for the shard containing the page
203 */
204 Lock_mutex &get_mutex(const page_id_t &page_id);
205 };
206
207 /*
208 Functions related to sharding by table
209
210 We identify tables by their id. Each table has its own lock queue, so we
211 simply group several such queues into single shard.
212 */
214 /** Each shard is protected by a separate mutex. Mutexes are padded to avoid
215 false sharing issues with cache. */
217 /**
218 Identifies the table shard which contains locks for the given table.
219 @param[in] table_id The id of the table
220 @return Integer in the range [0..lock_sys_t::SHARDS_COUNT)
221 */
222 static size_t get_shard(const table_id_t table_id);
223
224 public:
225 Table_shards();
227
228 /** Returns the mutex which (together with the global latch) protects the
229 table shard which contains table locks for the given table.
230 @param[in] table_id The id of the table
231 @return The mutex responsible for the shard containing the table
232 */
233 Lock_mutex &get_mutex(const table_id_t table_id);
234
235 /** Returns the mutex which (together with the global latch) protects the
236 table shard which contains table locks for the given table.
237 @param[in] table_id The id of the table
238 @return The mutex responsible for the shard containing the table
239 */
240 const Lock_mutex &get_mutex(const table_id_t table_id) const;
241
242 /** Returns the mutex which (together with the global latch) protects the
243 table shard which contains table locks for the given table.
244 @param[in] table The table
245 @return The mutex responsible for the shard containing the table
246 */
247 const Lock_mutex &get_mutex(const dict_table_t &table) const;
248 };
249
250 /** padding to prevent other memory update hotspots from residing on the same
251 memory cache line */
253
255
257
259
260 public:
261 /* You should use following RAII guards to modify the state of Latches. */
267
268 /** You should not use this functionality in new code.
269 Instead use Global_exclusive_latch_guard.
270 This is intended only to be use within lock0* module, thus this class is only
271 accessible through lock0priv.h.
272 It is only used by lock_rec_fetch_page() as a workaround. */
274
275 /** For some reason clang 6.0.0 and 7.0.0 (but not 8.0.0) fail at linking
276 stage if we completely omit the destructor declaration, or use
277
278 ~Latches() = default;
279
280 This might be somehow connected to one of these:
281 https://bugs.llvm.org/show_bug.cgi?id=28280
282 https://github.com/android/ndk/issues/143
283 https://reviews.llvm.org/D45898
284 So, this declaration is just to make clang 6.0.0 and 7.0.0 happy.
285 */
286#if defined(__clang__) && (__clang_major__ < 8)
287 ~Latches() {} // NOLINT(modernize-use-equals-default)
288#else
289 ~Latches() = default;
290#endif
291
292#ifdef UNIV_DEBUG
293 /**
294 Tests if lock_sys latch is exclusively owned by the current thread.
295 @return true iff the current thread owns exclusive global lock_sys latch
296 */
298
299 /**
300 Tests if lock_sys latch is owned in shared mode by the current thread.
301 @return true iff the current thread owns shared global lock_sys latch
302 */
303 bool owns_shared_global_latch() const { return global_latch.s_own(); }
304
305 /**
306 Tests if given page shard can be safely accessed by the current thread.
307 @param[in] page_id The space_id and page_no of the page
308 @return true iff the current thread owns exclusive global lock_sys latch or
309 both a shared global lock_sys latch and mutex protecting the page shard
310 */
311 bool owns_page_shard(const page_id_t &page_id) const {
313 (page_shards.get_mutex(page_id).is_owned() &&
315 }
316
317 /**
318 Tests if given table shard can be safely accessed by the current thread.
319 @param table the table
320 @return true iff the current thread owns exclusive global lock_sys latch or
321 both a shared global lock_sys latch and mutex protecting the table shard
322 */
325 (table_shards.get_mutex(table).is_owned() &&
327 }
328#endif /* UNIV_DEBUG */
329};
330} // namespace locksys
331
332#endif /* lock0latches_h */
Rw-lock with very fast, highly concurrent s-lock but slower x-lock.
Definition: sync0sharded_rw.h:63
bool s_own(size_t shard_no) const
Definition: sync0sharded_rw.h:143
void x_unlock()
Definition: sync0sharded_rw.h:138
size_t s_lock(ut::Location location)
Definition: sync0sharded_rw.h:96
void s_unlock(size_t shard_no)
Definition: sync0sharded_rw.h:103
bool x_own() const
Definition: sync0sharded_rw.h:147
bool try_x_lock(ut::Location location)
Tries to obtain exclusive latch - similar to x_lock(), but non-blocking, and thus can fail.
Definition: sync0sharded_rw.h:120
bool is_x_blocked_by_s(size_t shard_no)
Checks if there is a thread requesting an x-latch waiting for threads to release their s-latches on g...
Definition: sync0sharded_rw.h:111
void x_lock(ut::Location location)
Definition: sync0sharded_rw.h:132
A RAII helper which latches global_latch in exclusive mode during constructor, and unlatches it durin...
Definition: lock0guards.h:40
A RAII helper which tries to exclusively latch the global_lach in constructor and unlatches it,...
Definition: lock0guards.h:51
A RAII helper which latches global_latch in shared mode during constructor, and unlatches it during d...
Definition: lock0guards.h:71
Definition: lock0latches.h:174
Page_shards()
Definition: lock0latches.cc:99
static size_t get_shard(const page_id_t &page_id)
Identifies the page shard which contains record locks for records from the given page.
Definition: lock0latches.cc:36
~Page_shards()
Definition: lock0latches.cc:105
const Lock_mutex & get_mutex(const page_id_t &page_id) const
Returns the mutex which (together with the global latch) protects the page shard which contains recor...
Definition: lock0latches.cc:54
Padded_mutex mutexes[SHARDS_COUNT]
Each shard is protected by a separate mutex.
Definition: lock0latches.h:177
Definition: lock0latches.h:213
static size_t get_shard(const table_id_t table_id)
Identifies the table shard which contains locks for the given table.
Definition: lock0latches.cc:66
Table_shards()
Definition: lock0latches.cc:111
Padded_mutex mutexes[SHARDS_COUNT]
Each shard is protected by a separate mutex.
Definition: lock0latches.h:216
Lock_mutex & get_mutex(const table_id_t table_id)
Returns the mutex which (together with the global latch) protects the table shard which contains tabl...
Definition: lock0latches.cc:75
~Table_shards()
Definition: lock0latches.cc:117
A helper wrapper around Shared_rw_lock which simplifies:
Definition: lock0latches.h:115
bool is_x_blocked_by_our_s()
Checks if there is a thread requesting an x-latch waiting for our thread to release its s-latch.
Definition: lock0latches.h:137
Sharded_rw_lock rw_lock
The actual rw_lock implementation doing the heavy lifting.
Definition: lock0latches.h:117
Unique_sharded_rw_lock()
Definition: lock0latches.cc:89
void x_lock(ut::Location location)
Definition: lock0latches.h:141
bool s_own() const
Definition: lock0latches.h:154
static constexpr size_t NOT_IN_USE
The value used for m_shard_id to indicate that current thread did not s-latch any of the rw_lock's sh...
Definition: lock0latches.h:121
static thread_local size_t m_shard_id
The id of the rw_lock's shard which this thread has s-latched, or NOT_IN_USE if it has not s-latched ...
Definition: lock0latches.h:125
bool x_own() const
Definition: lock0latches.h:153
void s_lock(ut::Location location)
Definition: lock0latches.h:143
bool try_x_lock(ut::Location location)
Definition: lock0latches.h:130
void s_unlock()
Definition: lock0latches.h:147
~Unique_sharded_rw_lock()
Definition: lock0latches.cc:97
void x_unlock()
Definition: lock0latches.h:142
The class which handles the logic of latching of lock_sys queues themselves.
Definition: lock0latches.h:103
~Latches()=default
For some reason clang 6.0.0 and 7.0.0 (but not 8.0.0) fail at linking stage if we completely omit the...
Page_shards page_shards
Definition: lock0latches.h:256
static constexpr size_t SHARDS_COUNT
Number of page shards, and also number of table shards.
Definition: lock0latches.h:164
char pad1[ut::INNODB_CACHE_LINE_SIZE]
padding to prevent other memory update hotspots from residing on the same memory cache line
Definition: lock0latches.h:252
ib_mutex_t Lock_mutex
Definition: lock0latches.h:105
bool owns_table_shard(const dict_table_t &table) const
Tests if given table shard can be safely accessed by the current thread.
Definition: lock0latches.h:323
Unique_sharded_rw_lock global_latch
Definition: lock0latches.h:254
bool owns_shared_global_latch() const
Tests if lock_sys latch is owned in shared mode by the current thread.
Definition: lock0latches.h:303
bool owns_page_shard(const page_id_t &page_id) const
Tests if given page shard can be safely accessed by the current thread.
Definition: lock0latches.h:311
bool owns_exclusive_global_latch() const
Tests if lock_sys latch is exclusively owned by the current thread.
Definition: lock0latches.h:297
Table_shards table_shards
Definition: lock0latches.h:258
A RAII helper which latches the mutex protecting given shard during constructor, and unlatches it dur...
Definition: lock0guards.h:91
A RAII helper which latches the mutexes protecting specified shards for the duration of its scope.
Definition: lock0guards.h:141
Definition: lock0priv.h:1098
Page identifier.
Definition: buf0types.h:207
Data dictionary global types.
ib_id_t table_id_t
Table or partition identifier (unique within an InnoDB instance).
Definition: dict0types.h:232
static PFS_engine_table_share_proxy table
Definition: pfs.cc:61
Definition: lock0guards.h:34
constexpr size_t INNODB_CACHE_LINE_SIZE
CPU cache line size.
Definition: ut0cpu_cache.h:41
Data structure for a database table.
Definition: dict0mem.h:1909
Definition: ut0core.h:36
The sharded read-write lock (for threads).
Utilities related to CPU cache.
#define ut_ad(EXPR)
Debug assertion.
Definition: ut0dbg.h:105
Policy based mutexes.