MySQL 8.0.30
Source Code Documentation
ut0new.h
Go to the documentation of this file.
1/*****************************************************************************
2
3Copyright (c) 2014, 2022, Oracle and/or its affiliates.
4
5This program is free software; you can redistribute it and/or modify it under
6the terms of the GNU General Public License, version 2.0, as published by the
7Free Software Foundation.
8
9This program is also distributed with certain software (including but not
10limited to OpenSSL) that is licensed under separate terms, as designated in a
11particular file or component or in included license documentation. The authors
12of MySQL hereby grant you an additional permission to link the program and
13your derivative works with the separately licensed software that they have
14included with MySQL.
15
16This program is distributed in the hope that it will be useful, but WITHOUT
17ANY WARRANTY; without even the implied warranty of MERCHANTABILITY or FITNESS
18FOR A PARTICULAR PURPOSE. See the GNU General Public License, version 2.0,
19for more details.
20
21You should have received a copy of the GNU General Public License along with
22this program; if not, write to the Free Software Foundation, Inc.,
2351 Franklin St, Fifth Floor, Boston, MA 02110-1301 USA
24
25*****************************************************************************/
26
27/** @file include/ut0new.h
28 Dynamic memory allocation routines and custom allocators specifically
29 crafted to support memory instrumentation through performance schema memory
30 engine (PFS).
31 */
32
33/** This file contains a set of libraries providing overloads for regular
34 dynamic allocation routines which allow for opt-in memory instrumentation
35 through performance schema memory engine (PFS).
36
37 In particular, _no_ regular dynamic allocation routines shall be used given
38 that the end goal of instrumentation through PFS is system observability
39 and resource control. In practice this means that we are off the chances to
40 use _any_ standard means of allocating the memory and that we have to
41 provide and re-implement our own PFS-aware variants ourselves.
42
43 This does not only apply to direct memory allocation through malloc or new
44 but also to data structures that may allocate dynamic memory under the hood,
45 like the ones from STL. For that reason, STL data structures shall always
46 be used with PFS-enabled custom memory allocator. STL algorithms OTOH
47 also _may_ allocate dynamic memory but they do not provide customization
48 point for user-code to provide custom memory allocation mechanism so there's
49 nothing that we can do about it.
50
51 Furthermore, facilities that allow safer memory management such as
52 std::unique_ptr, std::shared_ptr and their respective std::make_unique and
53 std::make_shared functions also have to be re-implemented as such so that
54 they become PFS-aware.
55
56 Following is the list of currently implemented PFS-enabled dynamic
57 allocation overloads and associated facilities:
58 * Primitive allocation functions:
59 * ut::malloc
60 * ut::zalloc
61 * ut::realloc
62 * ut::free
63 * ut::{malloc | zalloc | realloc}_withkey
64 * Primitive allocation functions for types with extended alignment:
65 * ut::aligned_alloc
66 * ut::aligned_zalloc
67 * ut::aligned_free
68 * ut::{aligned_alloc | aligned_zalloc}_withkey
69 * Primitive allocation functions for page-aligned allocations:
70 * ut::malloc_page
71 * ut::malloc_page_withkey
72 * ut::free_page
73 * Primitive allocation functions for large (huge) page aligned
74 allocations:
75 * ut::malloc_large_page
76 * ut::malloc_large_page_withkey
77 * ut::free_large_page
78 * Primitive allocation functions for large (huge) aligned allocations with
79 fallback to page-aligned allocations:
80 * ut::malloc_large_page(fallback_to_normal_page_t)
81 * ut::malloc_large_page_withkey(fallback_to_normal_page_t)
82 * ut::free_large_page(fallback_to_normal_page_t)
83 * Overloads for C++ new and delete syntax:
84 * ut::new_
85 * ut::new_arr
86 * ut::{new_ | new_arr_}_withkey
87 * ut::delete_
88 * ut::delete_arr
89 * Overloads for C++ new and delete syntax for types with extended
90 alignment:
91 * ut::aligned_new
92 * ut::aligned_new_arr
93 * ut::{aligned_new_ | aligned_new_arr_}_withkey
94 * ut::aligned_delete
95 * ut::aligned_delete_arr
96 * Custom memory allocators:
97 * ut::allocator
98 * Overloads for std::unique_ptr and std::shared_ptr factory functions
99 * ut::make_unique
100 * ut::make_unique_aligned
101 * ut::make_shared
102 * ut::make_shared_aligned
103 _withkey variants from above are the PFS-enabled dynamic allocation
104 overloads.
105
106 Usages of PFS-enabled library functions are trying to resemble already
107 familiar syntax as close as possible. For concrete examples please see
108 particular function documentation but in general it applies that ::foo(x)
109 becomes ut::foo(x) or ut::foo_withkey(key, x) where foo is some allocation
110 function listed above and key is PFS key to instrument the allocation with.
111*/
112
113#ifndef ut0new_h
114#define ut0new_h
115
116#include <algorithm>
117#include <cerrno>
118#include <cstddef>
119#include <cstdlib>
120#include <cstring>
121#include <limits>
122#include <list>
123#include <map>
124#include <memory>
125#include <set>
126#include <type_traits> /* std::is_trivially_default_constructible */
127#include <unordered_set>
128
129#include "my_basename.h"
132#include "mysql/psi/psi_memory.h"
133
134namespace ut {
135/** Can be used to extract pointer and size of the allocation provided by the
136OS. It is a low level information, and is needed only to call low level
137memory-related OS functions. */
139 /** A pointer returned by the OS allocator. */
140 void *base_ptr;
141 /** The size of allocation that OS performed. */
143};
144} // namespace ut
145
146#include "detail/ut0new.h"
147#include "os0proc.h"
148#include "os0thread.h"
149#include "univ.i"
150#include "ut0byte.h" /* ut_align */
151#include "ut0cpu_cache.h"
152#include "ut0dbg.h"
153#include "ut0ut.h"
154
155namespace ut {
156
157/** Light-weight and type-safe wrapper around the PSI_memory_key
158 that eliminates the possibility of introducing silent bugs
159 through the course of implicit conversions and makes them
160 show up as compile-time errors.
161
162 Without this wrapper it was possible to say:
163 aligned_alloc_withkey(10*sizeof(int), key, 64))
164 Which would unfortunately compile just fine but it would silently
165 introduce a bug because it confuses the order of 10*sizeof(int) and
166 key input arguments. Both of them are unsigned types.
167
168 With the wrapper, aligned_alloc_withkey(10*sizeof(int), key, 64)) now
169 results with a compile-time error and the only proper way to accomplish
170 the original intent is to use PSI_memory_key_t wrapper like so:
171 aligned_alloc_withkey(PSI_memory_key_t{key}, 10*sizeof(int), 64))
172
173 Or by making use of the convenience function to create one:
174 aligned_alloc_withkey(make_psi_memory_key(key), 10*sizeof(int), 64))
175*/
178 PSI_memory_key operator()() const { return m_key; }
180};
181
182/** Convenience helper function to create type-safe representation of
183 PSI_memory_key.
184
185 @param[in] key PSI memory key to be held in type-safe PSI_memory_key_t.
186 @return PSI_memory_key_t which wraps the given PSI_memory_key
187 */
189 return PSI_memory_key_t(key);
190}
191
192} // namespace ut
193
194/** Maximum number of retries to allocate memory. */
195extern const size_t alloc_max_retries;
196
197/** Keys for registering allocations with performance schema.
198Pointers to these variables are supplied to PFS code via the pfs_info[]
199array and the PFS code initializes them via PSI_MEMORY_CALL(register_memory)().
200mem_key_other and mem_key_std are special in the following way.
201* If the caller has not provided a key and the file name of the caller is
202 unknown, then mem_key_std will be used. This happens only when called from
203 within std::* containers.
204* If the caller has not provided a key and the file name of the caller is
205 known, but is not amongst the predefined names (see ut_new_boot()) then
206 mem_key_other will be used. Generally this should not happen and if it
207 happens then that means that the list of predefined names must be extended.
208Keep this list alphabetically sorted. */
213/** Memory key for clone */
228/* Please obey alphabetical order in the definitions above. */
229
230/** Setup the internal objects needed for ut::*_withkey() to operate.
231This must be called before the first call to ut::*_withkey(). */
232void ut_new_boot();
233
234/** Setup the internal objects needed for ut::*_withkey() to operate.
235This must be called before the first call to ut::*_withkey(). This
236version of function might be called several times and it will
237simply skip all calls except the first one, during which the
238initialization will happen. */
239void ut_new_boot_safe();
240
241#ifdef UNIV_PFS_MEMORY
242
243/** List of filenames that allocate memory and are instrumented via PFS. */
244static constexpr const char *auto_event_names[] = {
245 /* Keep this list alphabetically sorted. */
246 "api0api",
247 "api0misc",
248 "btr0btr",
249 "btr0cur",
250 "btr0load",
251 "btr0pcur",
252 "btr0sea",
253 "btr0types",
254 "buf",
255 "buf0buddy",
256 "buf0buf",
257 "buf0checksum",
258 "buf0dblwr",
259 "buf0dump",
260 "buf0flu",
261 "buf0lru",
262 "buf0rea",
263 "buf0stats",
264 "buf0types",
265 "checksum",
266 "crc32",
267 "create",
268 "data0data",
269 "data0type",
270 "data0types",
271 "db0err",
272 "ddl0buffer",
273 "ddl0builder",
274 "ddl0ctx",
275 "ddl0ddl",
276 "ddl0file-reader",
277 "ddl0loader",
278 "ddl0merge",
279 "ddl0rtree",
280 "ddl0par-scan",
281 "dict",
282 "dict0boot",
283 "dict0crea",
284 "dict0dd",
285 "dict0dict",
286 "dict0load",
287 "dict0mem",
288 "dict0priv",
289 "dict0sdi",
290 "dict0stats",
291 "dict0stats_bg",
292 "dict0types",
293 "dyn0buf",
294 "dyn0types",
295 "eval0eval",
296 "eval0proc",
297 "fil0fil",
298 "fil0types",
299 "file",
300 "fsp0file",
301 "fsp0fsp",
302 "fsp0space",
303 "fsp0sysspace",
304 "fsp0types",
305 "fts0ast",
306 "fts0blex",
307 "fts0config",
308 "fts0fts",
309 "fts0opt",
310 "fts0pars",
311 "fts0plugin",
312 "fts0priv",
313 "fts0que",
314 "fts0sql",
315 "fts0tlex",
316 "fts0tokenize",
317 "fts0types",
318 "fts0vlc",
319 "fut0fut",
320 "fut0lst",
321 "gis0geo",
322 "gis0rtree",
323 "gis0sea",
324 "gis0type",
325 "ha0ha",
326 "ha0storage",
327 "ha_innodb",
328 "ha_innopart",
329 "ha_prototypes",
330 "handler0alter",
331 "hash0hash",
332 "i_s",
333 "ib0mutex",
334 "ibuf0ibuf",
335 "ibuf0types",
336 "lexyy",
337 "lob0lob",
338 "lock0iter",
339 "lock0lock",
340 "lock0prdt",
341 "lock0priv",
342 "lock0types",
343 "lock0wait",
344 "log0log",
345 "log0recv",
346 "log0write",
347 "mach0data",
348 "mem",
349 "mem0mem",
350 "memory",
351 "mtr0log",
352 "mtr0mtr",
353 "mtr0types",
354 "os0atomic",
355 "os0event",
356 "os0file",
357 "os0numa",
358 "os0once",
359 "os0proc",
360 "os0thread",
361 "page",
362 "page0cur",
363 "page0page",
364 "page0size",
365 "page0types",
366 "page0zip",
367 "pars0grm",
368 "pars0lex",
369 "pars0opt",
370 "pars0pars",
371 "pars0sym",
372 "pars0types",
373 "que0que",
374 "que0types",
375 "read0read",
376 "read0types",
377 "rec",
378 "rem0cmp",
379 "rem0rec",
380 "rem0types",
381 "row0ext",
382 "row0ft",
383 "row0import",
384 "row0ins",
385 "row0log",
386 "row0mysql",
387 "row0purge",
388 "row0quiesce",
389 "row0row",
390 "row0sel",
391 "row0types",
392 "row0uins",
393 "row0umod",
394 "row0undo",
395 "row0upd",
396 "row0vers",
397 "sess0sess",
398 "srv0conc",
399 "srv0mon",
400 "srv0srv",
401 "srv0start",
402 "srv0tmp",
403 "sync0arr",
404 "sync0debug",
405 "sync0policy",
406 "sync0sharded_rw",
407 "sync0rw",
408 "sync0sync",
409 "sync0types",
410 "trx0i_s",
411 "trx0purge",
412 "trx0rec",
413 "trx0roll",
414 "trx0rseg",
415 "trx0sys",
416 "trx0trx",
417 "trx0types",
418 "trx0undo",
419 "trx0xa",
420 "usr0sess",
421 "usr0types",
422 "ut",
423 "ut0byte",
424 "ut0counter",
425 "ut0crc32",
426 "ut0dbg",
427 "ut0link_buf",
428 "ut0list",
429 "ut0lock_free_hash",
430 "ut0lst",
431 "ut0mem",
432 "ut0mutex",
433 "ut0new",
434 "ut0pool",
435 "ut0rbt",
436 "ut0rnd",
437 "ut0sort",
438 "ut0stage",
439 "ut0ut",
440 "ut0vec",
441 "ut0wqueue",
442 "zipdecompress",
443};
444
445static constexpr size_t n_auto = UT_ARR_SIZE(auto_event_names);
448
449/** gcc 5 fails to evalutate costexprs at compile time. */
450#if defined(__GNUG__) && (__GNUG__ == 5)
451
452/** Compute whether a string begins with a given prefix, compile-time.
453@param[in] a first string, taken to be zero-terminated
454@param[in] b second string (prefix to search for)
455@param[in] b_len length in bytes of second string
456@param[in] index character index to start comparing at
457@return whether b is a prefix of a */
458constexpr bool ut_string_begins_with(const char *a, const char *b, size_t b_len,
459 size_t index = 0) {
460 return (index == b_len || (a[index] == b[index] &&
461 ut_string_begins_with(a, b, b_len, index + 1)));
462}
463
464/** Find the length of the filename without its file extension.
465@param[in] file filename, with extension but without directory
466@param[in] index character index to start scanning for extension
467 separator at
468@return length, in bytes */
469constexpr size_t ut_len_without_extension(const char *file, size_t index = 0) {
470 return ((file[index] == '\0' || file[index] == '.')
471 ? index
472 : ut_len_without_extension(file, index + 1));
473}
474
475/** Retrieve a memory key (registered with PFS), given the file name of the
476caller.
477@param[in] file portion of the filename - basename, with extension
478@param[in] len length of the filename to check for
479@param[in] index index of first PSI key to check
480@return registered memory key or PSI_NOT_INSTRUMENTED if not found */
482 size_t len,
483 size_t index = 0) {
484 return ((index == n_auto)
487 ? auto_event_keys[index]
488 : ut_new_get_key_by_base_file(file, len, index + 1)));
489}
490
491/** Retrieve a memory key (registered with PFS), given the file name of
492the caller.
493@param[in] file portion of the filename - basename, with extension
494@return registered memory key or PSI_NOT_INSTRUMENTED if not found */
495constexpr PSI_memory_key ut_new_get_key_by_file(const char *file) {
497}
498
499#define UT_NEW_THIS_FILE_PSI_KEY ut_new_get_key_by_file(MY_BASENAME)
500
501#else /* __GNUG__ == 5 */
502
503/** Compute whether a string begins with a given prefix, compile-time.
504@param[in] a first string, taken to be zero-terminated
505@param[in] b second string (prefix to search for)
506@param[in] b_len length in bytes of second string
507@return whether b is a prefix of a */
508constexpr bool ut_string_begins_with(const char *a, const char *b,
509 size_t b_len) {
510 for (size_t i = 0; i < b_len; ++i) {
511 if (a[i] != b[i]) {
512 return false;
513 }
514 }
515 return true;
516}
517
518/** Find the length of the filename without its file extension.
519@param[in] file filename, with extension but without directory
520@return length, in bytes */
521constexpr size_t ut_len_without_extension(const char *file) {
522 for (size_t i = 0;; ++i) {
523 if (file[i] == '\0' || file[i] == '.') {
524 return i;
525 }
526 }
527}
528
529/** Retrieve a memory key (registered with PFS), given the file name of the
530caller.
531@param[in] file portion of the filename - basename, with extension
532@param[in] len length of the filename to check for
533@return index to registered memory key or -1 if not found */
534constexpr int ut_new_get_key_by_base_file(const char *file, size_t len) {
535 for (size_t i = 0; i < n_auto; ++i) {
537 return static_cast<int>(i);
538 }
539 }
540 return -1;
541}
542
543/** Retrieve a memory key (registered with PFS), given the file name of
544the caller.
545@param[in] file portion of the filename - basename, with extension
546@return index to memory key or -1 if not found */
547constexpr int ut_new_get_key_by_file(const char *file) {
549}
550
551// Sending an expression through a template variable forces the compiler to
552// evaluate the expression at compile time (constexpr in itself has no such
553// guarantee, only that the compiler is allowed).
554template <int Value>
556 static constexpr int value = Value;
557};
558
559#define UT_NEW_THIS_FILE_PSI_INDEX \
560 (force_constexpr<ut_new_get_key_by_file(MY_BASENAME)>::value)
561
562#define UT_NEW_THIS_FILE_PSI_KEY \
563 (UT_NEW_THIS_FILE_PSI_INDEX == -1 \
564 ? ut::make_psi_memory_key(PSI_NOT_INSTRUMENTED) \
565 : ut::make_psi_memory_key(auto_event_keys[UT_NEW_THIS_FILE_PSI_INDEX]))
566
567#endif /* __GNUG__ == 5 */
568
569#else
570
571#define UT_NEW_THIS_FILE_PSI_KEY ut::make_psi_memory_key(PSI_NOT_INSTRUMENTED)
572
573#endif /* UNIV_PFS_MEMORY */
574
575namespace ut {
576
577#ifdef HAVE_PSI_MEMORY_INTERFACE
578constexpr bool WITH_PFS_MEMORY = true;
579#else
580constexpr bool WITH_PFS_MEMORY = false;
581#endif
582
583/** Dynamically allocates storage of given size. Instruments the memory with
584 given PSI memory key in case PFS memory support is enabled.
585
586 @param[in] key PSI memory key to be used for PFS memory instrumentation.
587 @param[in] size Size of storage (in bytes) requested to be allocated.
588 @return Pointer to the allocated storage. nullptr if dynamic storage
589 allocation failed.
590
591 Example:
592 int *x = static_cast<int*>(ut::malloc_withkey(key, 10*sizeof(int)));
593 */
594inline void *malloc_withkey(PSI_memory_key_t key, std::size_t size) noexcept {
596 using malloc_impl = detail::Alloc_<impl>;
597 return malloc_impl::alloc<false>(size, key());
598}
599
600/** Dynamically allocates storage of given size.
601
602 NOTE: Given that this function will _NOT_ be instrumenting the allocation
603 through PFS, observability for particular parts of the system which want to
604 use it will be lost or in best case inaccurate. Please have a strong reason
605 to do so.
606
607 @param[in] size Size of storage (in bytes) requested to be allocated.
608 @return Pointer to the allocated storage. nullptr if dynamic storage
609 allocation failed.
610
611 Example:
612 int *x = static_cast<int*>(ut::malloc_withkey(UT_NEW_THIS_FILE_PSI_KEY,
613 10*sizeof(int)));
614 */
615inline void *malloc(std::size_t size) noexcept {
617}
618
619/** Dynamically allocates zero-initialized storage of given size. Instruments
620 the memory with given PSI memory key in case PFS memory support is enabled.
621
622 @param[in] key PSI memory key to be used for PFS memory instrumentation.
623 @param[in] size Size of storage (in bytes) requested to be allocated.
624 @return Pointer to the zero-initialized allocated storage. nullptr if
625 dynamic storage allocation failed.
626
627 Example:
628 int *x = static_cast<int*>(ut::zalloc_withkey(key, 10*sizeof(int)));
629 */
630inline void *zalloc_withkey(PSI_memory_key_t key, std::size_t size) noexcept {
632 using malloc_impl = detail::Alloc_<impl>;
633 return malloc_impl::alloc<true>(size, key());
634}
635
636/** Dynamically allocates zero-initialized storage of given size.
637
638 NOTE: Given that this function will _NOT_ be instrumenting the allocation
639 through PFS, observability for particular parts of the system which want to
640 use it will be lost or in best case inaccurate. Please have a strong reason
641 to do so.
642
643 @param[in] size Size of storage (in bytes) requested to be allocated.
644 @return Pointer to the zero-initialized allocated storage. nullptr if
645 dynamic storage allocation failed.
646
647 Example:
648 int *x = static_cast<int*>(ut::zalloc_withkey(UT_NEW_THIS_FILE_PSI_KEY,
649 10*sizeof(int)));
650 */
651inline void *zalloc(std::size_t size) noexcept {
653}
654
655/** Upsizes or downsizes already dynamically allocated storage to the new size.
656 Instruments the memory with given PSI memory key in case PFS memory support
657 is enabled.
658
659 It also supports standard realloc() semantics by:
660 * allocating size bytes of memory when passed ptr is nullptr
661 * freeing the memory pointed by ptr if passed size is 0
662
663 @param[in] key PSI memory key to be used for PFS memory instrumentation.
664 @param[in] ptr Pointer to the memory area to be reallocated.
665 @param[in] size New size of storage (in bytes) requested to be reallocated.
666 @return Pointer to the reallocated storage. nullptr if dynamic storage
667 allocation failed.
668
669 Example:
670 int *x = static_cast<int*>(ut::malloc_withkey(key, 10*sizeof(int));
671 x = static_cast<int*>(ut::realloc_withkey(key, ptr, 100*sizeof(int)));
672 */
673inline void *realloc_withkey(PSI_memory_key_t key, void *ptr,
674 std::size_t size) noexcept {
676 using malloc_impl = detail::Alloc_<impl>;
677 return malloc_impl::realloc(ptr, size, key());
678}
679
680/** Upsizes or downsizes already dynamically allocated storage to the new size.
681
682 It also supports standard realloc() semantics by:
683 * allocating size bytes of memory when passed ptr is nullptr
684 * freeing the memory pointed by ptr if passed size is 0
685
686 NOTE: Given that this function will _NOT_ be instrumenting the allocation
687 through PFS, observability for particular parts of the system which want to
688 use it will be lost or in best case inaccurate. Please have a strong reason
689 to do so.
690
691 @param[in] ptr Pointer to the memory area to be reallocated.
692 @param[in] size New size of storage (in bytes) requested to be reallocated.
693 @return Pointer to the reallocated storage. nullptr if dynamic storage
694 allocation failed.
695
696 Example:
697 int *x = static_cast<int*>(ut::malloc_withkey(UT_NEW_THIS_FILE_PSI_KEY,
698 10*sizeof(int)); x = static_cast<int*>(ut::realloc(key, ptr,
699 100*sizeof(int)));
700 */
701inline void *realloc(void *ptr, std::size_t size) noexcept {
703 size);
704}
705
706/** Releases storage which has been dynamically allocated through any of
707 the ut::malloc*(), ut::realloc* or ut::zalloc*() variants.
708
709 @param[in] ptr Pointer which has been obtained through any of the
710 ut::malloc*(), ut::realloc* or ut::zalloc*() variants.
711
712 Example:
713 ut::free(ptr);
714 */
715inline void free(void *ptr) noexcept {
717 using malloc_impl = detail::Alloc_<impl>;
719}
720
721/** Dynamically allocates storage for an object of type T. Constructs the object
722 of type T with provided Args. Instruments the memory with given PSI memory
723 key in case PFS memory support is enabled.
724
725 @param[in] key PSI memory key to be used for PFS memory instrumentation.
726 @param[in] args Arguments one wishes to pass over to T constructor(s)
727 @return Pointer to the allocated storage. Throws std::bad_alloc exception
728 if dynamic storage allocation could not be fulfilled. Re-throws whatever
729 exception that may have occured during the construction of T, in which case
730 it automatically cleans up the raw memory allocated for it.
731
732 Example 1:
733 int *ptr = ut::new_withkey<int>(key);
734
735 Example 2:
736 int *ptr = ut::new_withkey<int>(key, 10);
737 assert(*ptr == 10);
738
739 Example 3:
740 struct A {
741 A(int x, int y) : _x(x), _y(y) {}
742 int _x, _y;
743 };
744 A *ptr = ut::new_withkey<A>(key, 1, 2);
745 assert(ptr->_x == 1);
746 assert(ptr->_y == 2);
747 */
748template <typename T, typename... Args>
749inline T *new_withkey(PSI_memory_key_t key, Args &&... args) {
750 auto mem = ut::malloc_withkey(key, sizeof(T));
751 if (unlikely(!mem)) throw std::bad_alloc();
752 try {
753 new (mem) T(std::forward<Args>(args)...);
754 } catch (...) {
755 ut::free(mem);
756 throw;
757 }
758 return static_cast<T *>(mem);
759}
760
761/** Dynamically allocates storage for an object of type T. Constructs the object
762 of type T with provided Args.
763
764 NOTE: Given that this function will _NOT_ be instrumenting the allocation
765 through PFS, observability for particular parts of the system which want to
766 use it will be lost or in best case inaccurate. Please have a strong reason
767 to do so.
768
769 @param[in] args Arguments one wishes to pass over to T constructor(s)
770 @return Pointer to the allocated storage. Throws std::bad_alloc exception
771 if dynamic storage allocation could not be fulfilled. Re-throws whatever
772 exception that may have occured during the construction of T, in which case
773 it automatically cleans up the raw memory allocated for it.
774
775 Example 1:
776 int *ptr = ut::new_withkey<int>(UT_NEW_THIS_FILE_PSI_KEY);
777
778 Example 2:
779 int *ptr = ut::new_withkey<int>(UT_NEW_THIS_FILE_PSI_KEY, 10);
780 assert(*ptr == 10);
781
782 Example 3:
783 struct A {
784 A(int x, int y) : _x(x), _y(y) {}
785 int _x, _y;
786 };
787 A *ptr = ut::new_withkey<A>(UT_NEW_THIS_FILE_PSI_KEY, 1, 2);
788 assert(ptr->_x == 1);
789 assert(ptr->_y == 2);
790 */
791template <typename T, typename... Args>
792inline T *new_(Args &&... args) {
793 return ut::new_withkey<T>(make_psi_memory_key(PSI_NOT_INSTRUMENTED),
794 std::forward<Args>(args)...);
795}
796
797/** Releases storage which has been dynamically allocated through any of
798 the ut::new*() variants. Destructs the object of type T.
799
800 @param[in] ptr Pointer which has been obtained through any of the
801 ut::new*() variants
802
803 Example:
804 ut::delete_(ptr);
805 */
806template <typename T>
807inline void delete_(T *ptr) noexcept {
808 if (unlikely(!ptr)) return;
809 ptr->~T();
810 ut::free(ptr);
811}
812
813/** Dynamically allocates storage for an array of T's. Constructs objects of
814 type T with provided Args. Arguments that are to be used to construct some
815 respective instance of T shall be wrapped into a std::tuple. See examples
816 down below. Instruments the memory with given PSI memory key in case PFS
817 memory support is enabled.
818
819 To create an array of default-intialized T's, one can use this function
820 template but for convenience purposes one can achieve the same by using
821 the ut::new_arr_withkey with ut::Count overload.
822
823 @param[in] key PSI memory key to be used for PFS memory instrumentation.
824 @param[in] args Tuples of arguments one wishes to pass over to T
825 constructor(s).
826 @return Pointer to the first element of allocated storage. Throws
827 std::bad_alloc exception if dynamic storage allocation could not be
828 fulfilled. Re-throws whatever exception that may have occured during the
829 construction of any instance of T, in which case it automatically destroys
830 successfully constructed objects till that moment (if any), and finally
831 cleans up the raw memory allocated for T instances.
832
833 Example 1:
834 int *ptr = ut::new_arr_withkey<int>(key,
835 std::forward_as_tuple(1),
836 std::forward_as_tuple(2));
837 assert(ptr[0] == 1);
838 assert(ptr[1] == 2);
839
840 Example 2:
841 struct A {
842 A(int x, int y) : _x(x), _y(y) {}
843 int _x, _y;
844 };
845 A *ptr = ut::new_arr_withkey<A>(key,
846 std::forward_as_tuple(0, 1), std::forward_as_tuple(2, 3),
847 std::forward_as_tuple(4, 5), std::forward_as_tuple(6, 7),
848 std::forward_as_tuple(8, 9));
849 assert(ptr[0]->_x == 0 && ptr[0]->_y == 1);
850 assert(ptr[1]->_x == 2 && ptr[1]->_y == 3);
851 assert(ptr[2]->_x == 4 && ptr[2]->_y == 5);
852 assert(ptr[3]->_x == 6 && ptr[3]->_y == 7);
853 assert(ptr[4]->_x == 8 && ptr[4]->_y == 9);
854
855 Example 3:
856 struct A {
857 A() : _x(10), _y(100) {}
858 A(int x, int y) : _x(x), _y(y) {}
859 int _x, _y;
860 };
861 A *ptr = ut::new_arr_withkey<A>(key,
862 std::forward_as_tuple(0, 1), std::forward_as_tuple(2, 3),
863 std::forward_as_tuple(), std::forward_as_tuple(6, 7),
864 std::forward_as_tuple());
865 assert(ptr[0]->_x == 0 && ptr[0]->_y == 1);
866 assert(ptr[1]->_x == 2 && ptr[1]->_y == 3);
867 assert(ptr[2]->_x == 10 && ptr[2]->_y == 100);
868 assert(ptr[3]->_x == 6 && ptr[3]->_y == 7);
869 assert(ptr[4]->_x == 10 && ptr[4]->_y == 100);
870 */
871template <typename T, typename... Args>
872inline T *new_arr_withkey(PSI_memory_key_t key, Args &&... args) {
874 using malloc_impl = detail::Alloc_<impl>;
875 auto mem = malloc_impl::alloc<false>(sizeof(T) * sizeof...(args), key());
876 if (unlikely(!mem)) throw std::bad_alloc();
877
878 size_t idx = 0;
879 try {
880 (...,
881 detail::construct<T>(mem, sizeof(T) * idx++, std::forward<Args>(args)));
882 } catch (...) {
883 for (size_t offset = (idx - 1) * sizeof(T); offset != 0;
884 offset -= sizeof(T)) {
885 reinterpret_cast<T *>(reinterpret_cast<std::uintptr_t>(mem) + offset -
886 sizeof(T))
887 ->~T();
888 }
890 throw;
891 }
892 return static_cast<T *>(mem);
893}
894
895/** Dynamically allocates storage for an array of T's. Constructs objects of
896 type T with provided Args. Arguments that are to be used to construct some
897 respective instance of T shall be wrapped into a std::tuple. See examples
898 down below.
899
900 To create an array of default-intialized T's, one can use this function
901 template but for convenience purposes one can achieve the same by using
902 the ut::new_arr_withkey with ut::Count overload.
903
904 NOTE: Given that this function will _NOT_ be instrumenting the allocation
905 through PFS, observability for particular parts of the system which want to
906 use it will be lost or in best case inaccurate. Please have a strong reason
907 to do so.
908
909 @param[in] args Tuples of arguments one wishes to pass over to T
910 constructor(s).
911 @return Pointer to the first element of allocated storage. Throws
912 std::bad_alloc exception if dynamic storage allocation could not be
913 fulfilled. Re-throws whatever exception that may have occured during the
914 construction of any instance of T, in which case it automatically destroys
915 successfully constructed objects till that moment (if any), and finally
916 cleans up the raw memory allocated for T instances.
917
918 Example 1:
919 int *ptr = ut::new_arr_withkey<int>(UT_NEW_THIS_FILE_PSI_KEY,
920 std::forward_as_tuple(1),
921 std::forward_as_tuple(2));
922 assert(ptr[0] == 1);
923 assert(ptr[1] == 2);
924
925 Example 2:
926 struct A {
927 A(int x, int y) : _x(x), _y(y) {}
928 int _x, _y;
929 };
930 A *ptr = ut::new_arr_withkey<A>(UT_NEW_THIS_FILE_PSI_KEY,
931 std::forward_as_tuple(0, 1), std::forward_as_tuple(2, 3),
932 std::forward_as_tuple(4, 5), std::forward_as_tuple(6, 7),
933 std::forward_as_tuple(8, 9));
934 assert(ptr[0]->_x == 0 && ptr[0]->_y == 1);
935 assert(ptr[1]->_x == 2 && ptr[1]->_y == 3);
936 assert(ptr[2]->_x == 4 && ptr[2]->_y == 5);
937 assert(ptr[3]->_x == 6 && ptr[3]->_y == 7);
938 assert(ptr[4]->_x == 8 && ptr[4]->_y == 9);
939
940 Example 3:
941 struct A {
942 A() : _x(10), _y(100) {}
943 A(int x, int y) : _x(x), _y(y) {}
944 int _x, _y;
945 };
946 A *ptr = ut::new_arr_withkey<A>(UT_NEW_THIS_FILE_PSI_KEY,
947 std::forward_as_tuple(0, 1), std::forward_as_tuple(2, 3),
948 std::forward_as_tuple(), std::forward_as_tuple(6, 7),
949 std::forward_as_tuple());
950 assert(ptr[0]->_x == 0 && ptr[0]->_y == 1);
951 assert(ptr[1]->_x == 2 && ptr[1]->_y == 3);
952 assert(ptr[2]->_x == 10 && ptr[2]->_y == 100);
953 assert(ptr[3]->_x == 6 && ptr[3]->_y == 7);
954 assert(ptr[4]->_x == 10 && ptr[4]->_y == 100);
955 */
956template <typename T, typename... Args>
957inline T *new_arr(Args &&... args) {
958 return ut::new_arr_withkey<T>(make_psi_memory_key(PSI_NOT_INSTRUMENTED),
959 std::forward<Args>(args)...);
960}
961
962/** Light-weight and type-safe wrapper which serves a purpose of
963 being able to select proper ut::new_arr* overload.
964
965 Without having a separate overload with this type, creating an array of
966 default-initialized instances of T through the ut::new_arr*(Args &&... args)
967 overload would have been impossible because:
968 int *ptr = ut::new_arr_withkey<int>(UT_NEW_THIS_FILE_PSI_KEY, 5);
969 wouldn't even compile and
970 int *ptr = ut::new_arr_withkey<int>(UT_NEW_THIS_FILE_PSI_KEY,
971 std::forward_as_tuple(5)); would compile but would not have intended effect.
972 It would create an array holding 1 integer element that is initialized to 5.
973
974 Given that function templates cannot be specialized, having an overload
975 crafted specifically for given case solves the problem:
976 int *ptr = ut::new_arr_withkey<int>(UT_NEW_THIS_FILE_PSI_KEY,
977 ut::Count{5});
978*/
979struct Count {
980 explicit Count(size_t count) : m_count(count) {}
981 size_t operator()() const { return m_count; }
982 size_t m_count;
983};
984
985/** Dynamically allocates storage for an array of T's. Constructs objects of
986 type T using default constructor. If T cannot be default-initialized (e.g.
987 default constructor does not exist), then this interace cannot be used for
988 constructing such an array. ut::new_arr_withkey overload with user-provided
989 initialization must be used then. Instruments the memory with given PSI
990 memory key in case PFS memory support is enabled.
991
992 @param[in] key PSI memory key to be used for PFS memory instrumentation.
993 @param[in] count Number of T elements in an array.
994 @return Pointer to the first element of allocated storage. Throws
995 std::bad_alloc exception if dynamic storage allocation could not be
996 fulfilled. Re-throws whatever exception that may have occured during the
997 construction of any instance of T, in which case it automatically destroys
998 successfully constructed objects till that moment (if any), and finally
999 cleans up the raw memory allocated for T instances.
1000
1001 Example 1:
1002 int *ptr = ut::new_arr_withkey<int>(key, ut::Count{2});
1003
1004 Example 2:
1005 struct A {
1006 A() : _x(10), _y(100) {}
1007 int _x, _y;
1008 };
1009 A *ptr = ut::new_arr_withkey<A>(key, ut::Count{5});
1010 assert(ptr[0]->_x == 10 && ptr[0]->_y == 100);
1011 assert(ptr[1]->_x == 10 && ptr[1]->_y == 100);
1012 assert(ptr[2]->_x == 10 && ptr[2]->_y == 100);
1013 assert(ptr[3]->_x == 10 && ptr[3]->_y == 100);
1014 assert(ptr[4]->_x == 10 && ptr[4]->_y == 100);
1015
1016 Example 3:
1017 struct A {
1018 A(int x, int y) : _x(x), _y(y) {}
1019 int _x, _y;
1020 };
1021 // Following cannot compile because A is not default-constructible
1022 A *ptr = ut::new_arr_withkey<A>(key, ut::Count{5});
1023 */
1024template <typename T>
1027 using malloc_impl = detail::Alloc_<impl>;
1028 auto mem = malloc_impl::alloc<false>(sizeof(T) * count(), key());
1029 if (unlikely(!mem)) throw std::bad_alloc();
1030
1031 size_t offset = 0;
1032 try {
1033 for (; offset < sizeof(T) * count(); offset += sizeof(T)) {
1034 new (reinterpret_cast<uint8_t *>(mem) + offset) T{};
1035 }
1036 } catch (...) {
1037 for (; offset != 0; offset -= sizeof(T)) {
1038 reinterpret_cast<T *>(reinterpret_cast<std::uintptr_t>(mem) + offset -
1039 sizeof(T))
1040 ->~T();
1041 }
1043 throw;
1044 }
1045 return static_cast<T *>(mem);
1046}
1047
1048/** Dynamically allocates storage for an array of T's. Constructs objects of
1049 type T using default constructor. If T cannot be default-initialized (e.g.
1050 default constructor does not exist), then this interace cannot be used for
1051 constructing such an array. ut::new_arr overload with user-provided
1052 initialization must be used then.
1053
1054 NOTE: Given that this function will _NOT_ be instrumenting the allocation
1055 through PFS, observability for particular parts of the system which want to
1056 use it will be lost or in best case inaccurate. Please have a strong reason
1057 to do so.
1058
1059 @param[in] count Number of T elements in an array.
1060 @return Pointer to the first element of allocated storage. Throws
1061 std::bad_alloc exception if dynamic storage allocation could not be
1062 fulfilled. Re-throws whatever exception that may have occured during the
1063 construction of any instance of T, in which case it automatically destroys
1064 successfully constructed objects till that moment (if any), and finally
1065 cleans up the raw memory allocated for T instances.
1066
1067 Example 1:
1068 int *ptr = ut::new_arr_withkey<int>(UT_NEW_THIS_FILE_PSI_KEY,
1069 ut::Count{2});
1070
1071 Example 2:
1072 struct A {
1073 A() : _x(10), _y(100) {}
1074 int _x, _y;
1075 };
1076 A *ptr = ut::new_arr_withkey<A>(UT_NEW_THIS_FILE_PSI_KEY, ut::Count{5});
1077 assert(ptr[0]->_x == 10 && ptr[0]->_y == 100);
1078 assert(ptr[1]->_x == 10 && ptr[1]->_y == 100);
1079 assert(ptr[2]->_x == 10 && ptr[2]->_y == 100);
1080 assert(ptr[3]->_x == 10 && ptr[3]->_y == 100);
1081 assert(ptr[4]->_x == 10 && ptr[4]->_y == 100);
1082
1083 Example 3:
1084 struct A {
1085 A(int x, int y) : _x(x), _y(y) {}
1086 int _x, _y;
1087 };
1088 // Following cannot compile because A is not default-constructible
1089 A *ptr = ut::new_arr_withkey<A>(UT_NEW_THIS_FILE_PSI_KEY, ut::Count{5});
1090 */
1091template <typename T>
1092inline T *new_arr(Count count) {
1093 return ut::new_arr_withkey<T>(make_psi_memory_key(PSI_NOT_INSTRUMENTED),
1094 count);
1095}
1096
1097/** Releases storage which has been dynamically allocated through any of
1098 the ut::new_arr*() variants. Destructs all objects of type T.
1099
1100 @param[in] ptr Pointer which has been obtained through any of the
1101 ut::new_arr*() variants
1102
1103 Example:
1104 ut::delete_arr(ptr);
1105 */
1106template <typename T>
1107inline void delete_arr(T *ptr) noexcept {
1108 if (unlikely(!ptr)) return;
1110 using malloc_impl = detail::Alloc_<impl>;
1111 const auto data_len = malloc_impl::datalen(ptr);
1112 for (size_t offset = 0; offset < data_len; offset += sizeof(T)) {
1113 reinterpret_cast<T *>(reinterpret_cast<std::uintptr_t>(ptr) + offset)->~T();
1114 }
1115 malloc_impl::free(ptr);
1116}
1117
1118/** Returns number of bytes that ut::malloc_*, ut::zalloc_*, ut::realloc_* and
1119 ut::new_* variants will be using to store the necessary metadata for PFS.
1120
1121 @return Size of the PFS metadata.
1122*/
1123inline size_t pfs_overhead() noexcept {
1125 using malloc_impl = detail::Alloc_<impl>;
1127}
1128
1129/** Dynamically allocates system page-aligned storage of given size. Instruments
1130 the memory with given PSI memory key in case PFS memory support is enabled.
1131
1132 Actual page-alignment, and thus page-size, will depend on CPU architecture
1133 but in general page is traditionally mostly 4K large. In contrast to Unices,
1134 Windows do make an exception here and implement 64K granularity on top of
1135 regular page-size for some legacy reasons. For more details see:
1136 https://devblogs.microsoft.com/oldnewthing/20031008-00/?p=42223
1137
1138 @param[in] key PSI memory key to be used for PFS memory instrumentation.
1139 @param[in] size Size of storage (in bytes) requested to be allocated.
1140 @return Pointer to the page-aligned storage. nullptr if dynamic storage
1141 allocation failed.
1142
1143 Example:
1144 int *x = static_cast<int*>(ut::malloc_page_withkey(key, 10*sizeof(int)));
1145 */
1147 std::size_t size) noexcept {
1149 using page_alloc_impl = detail::Page_alloc_<impl>;
1150 return page_alloc_impl::alloc(size, key());
1151}
1152
1153/** Dynamically allocates system page-aligned storage of given size.
1154
1155 Actual page-alignment, and thus page-size, will depend on CPU architecture
1156 but in general page is traditionally mostly 4K large. In contrast to Unices,
1157 Windows do make an exception here and implement 64K granularity on top of
1158 regular page-size for some legacy reasons. For more details see:
1159 https://devblogs.microsoft.com/oldnewthing/20031008-00/?p=42223
1160
1161 NOTE: Given that this function will _NOT_ be instrumenting the allocation
1162 through PFS, observability for particular parts of the system which want to
1163 use it will be lost or in best case inaccurate. Please have a strong reason
1164 to do so.
1165
1166 @param[in] size Size of storage (in bytes) requested to be allocated.
1167 @return Pointer to the page-aligned storage. nullptr if dynamic storage
1168 allocation failed.
1169
1170 Example:
1171 int *x = static_cast<int*>(ut::malloc_page(10*sizeof(int)));
1172 */
1173inline void *malloc_page(std::size_t size) noexcept {
1175 size);
1176}
1177
1178/** Retrieves the total amount of bytes that are available for application code
1179 to use.
1180
1181 Amount of bytes returned does _not_ have to match bytes requested
1182 through ut::malloc_page*(). This is so because bytes requested will always
1183 be implicitly rounded up to the next regular page size (e.g. 4K).
1184
1185 @param[in] ptr Pointer which has been obtained through any of the
1186 ut::malloc_page*() variants.
1187 @return Number of bytes available.
1188
1189 Example:
1190 int *x = static_cast<int*>(ut::malloc_page(10*sizeof(int)));
1191 assert(page_allocation_size(x) == CPU_PAGE_SIZE);
1192 */
1193inline size_t page_allocation_size(void *ptr) noexcept {
1195 using page_alloc_impl = detail::Page_alloc_<impl>;
1196 return page_alloc_impl::datalen(ptr);
1197}
1198
1199/** Retrieves the pointer and size of the allocation provided by the OS. It is a
1200 low level information, and is needed only to call low level memory-related
1201 OS functions.
1202
1203 @param[in] ptr Pointer which has been obtained through any of the
1204 ut::malloc_page*() variants.
1205 @return Low level OS allocation info.
1206 */
1209 using page_alloc_impl = detail::Page_alloc_<impl>;
1210 return page_alloc_impl::low_level_info(ptr);
1211}
1212
1213/** Releases storage which has been dynamically allocated through any of
1214 the ut::malloc_page*() variants.
1215
1216 @param[in] ptr Pointer which has been obtained through any of the
1217 ut::malloc_page*() variants.
1218 @return True if releasing the page-aligned memory was successful.
1219
1220 Example:
1221 ut::free_page(ptr);
1222 */
1223inline bool free_page(void *ptr) noexcept {
1225 using page_alloc_impl = detail::Page_alloc_<impl>;
1226 return page_alloc_impl::free(ptr);
1227}
1228
1229/** Dynamically allocates memory backed up by large (huge) pages. Instruments
1230 the memory with given PSI memory key in case PFS memory support is enabled.
1231
1232 For large (huge) pages to be functional, usually some steps in system admin
1233 preparation is required. Exact steps vary from system to system.
1234
1235 @param[in] key PSI memory key to be used for PFS memory instrumentation.
1236 @param[in] size Size of storage (in bytes) requested to be allocated.
1237 @return Pointer to the page-aligned storage. nullptr if dynamic storage
1238 allocation failed.
1239
1240 Example:
1241 int *x = static_cast<int*>(
1242 ut::malloc_large_page_withkey(key, 10*sizeof(int))
1243 );
1244 */
1246 std::size_t size) noexcept {
1248 using large_page_alloc_impl = detail::Large_alloc_<impl>;
1249 return large_page_alloc_impl::alloc(size, key());
1250}
1251
1252/** Dynamically allocates memory backed up by large (huge) pages.
1253
1254 For large (huge) pages to be functional, usually some steps in system admin
1255 preparation is required. Exact steps vary from system to system.
1256
1257 NOTE: Given that this function will _NOT_ be instrumenting the allocation
1258 through PFS, observability for particular parts of the system which want to
1259 use it will be lost or in best case inaccurate. Please have a strong reason
1260 to do so.
1261
1262 @param[in] size Size of storage (in bytes) requested to be allocated.
1263 @return Pointer to the page-aligned storage. nullptr if dynamic storage
1264 allocation failed.
1265
1266 Example:
1267 int *x = static_cast<int*>(ut::malloc_large_page(10*sizeof(int)));
1268 */
1269inline void *malloc_large_page(std::size_t size) noexcept {
1272}
1273
1274/** Retrieves the total amount of bytes that are available for application code
1275 to use.
1276
1277 Amount of bytes returned does _not_ have to match bytes requested
1278 through ut::malloc_large_page*(). This is so because bytes requested will
1279 always be implicitly rounded up to the next multiple of huge-page size (e.g.
1280 2MiB). Exact huge-page size value that is going to be used will be stored
1281 in large_page_default_size.
1282
1283 @param[in] ptr Pointer which has been obtained through any of the
1284 ut::malloc_large_page*() variants.
1285 @return Number of bytes available.
1286
1287 Example:
1288 int *x = static_cast<int*>(ut::malloc_large_page(10*sizeof(int)));
1289 assert(large_page_allocation_size(x) == HUGE_PAGE_SIZE);
1290 */
1291inline size_t large_page_allocation_size(void *ptr) noexcept {
1293 using large_page_alloc_impl = detail::Large_alloc_<impl>;
1294 return large_page_alloc_impl::datalen(ptr);
1295}
1296
1297/** Retrieves the pointer and size of the allocation provided by the OS. It is a
1298 low level information, and is needed only to call low level memory-related
1299 OS functions.
1300
1301 @param[in] ptr Pointer which has been obtained through any of the
1302 ut::malloc_large_page*() variants.
1303 @return Low level OS allocation info.
1304 */
1307 using large_page_alloc_impl = detail::Large_alloc_<impl>;
1308 return large_page_alloc_impl::low_level_info(ptr);
1309}
1310
1311/** Releases storage which has been dynamically allocated through any of
1312 the ut::malloc_large_page*() variants.
1313
1314 @param[in] ptr Pointer which has been obtained through any of the
1315 ut::malloc_large_page*() variants.
1316 @return True if releasing the large (huge) page-aligned memory was
1317 successful.
1318
1319 Example:
1320 ut::free_large_page(ptr);
1321 */
1322inline bool free_large_page(void *ptr) noexcept {
1324 using large_page_alloc_impl = detail::Large_alloc_<impl>;
1325 return large_page_alloc_impl::free(ptr);
1326}
1327
1328/* Helper type for tag-dispatch */
1330
1331/** Dynamically allocates memory backed up by large (huge) pages. In the event
1332 that large (huge) pages are unavailable or disabled explicitly through
1333 os_use_large_pages, it will fallback to dynamic allocation backed by
1334 page-aligned memory. Instruments the memory with given PSI memory key in
1335 case PFS memory support is enabled.
1336
1337 @param[in] key PSI memory key to be used for PFS memory instrumentation.
1338 @param[in] size Size of storage (in bytes) requested to be allocated.
1339 @param[in] large_pages_enabled If true, the large pages will be tried to be
1340 used.
1341 @return Pointer to the page-aligned storage. nullptr if dynamic storage
1342 allocation failed.
1343
1344 Example:
1345 int *x = static_cast<int*>(
1346 ut::malloc_large_page_withkey(
1347 key,
1348 10*sizeof(int),
1349 fallback_to_normal_page_t{}
1350 )
1351 );
1352 */
1355 bool large_pages_enabled = os_use_large_pages) noexcept {
1356 void *large_page_mem = nullptr;
1357 if (large_pages_enabled) {
1358 large_page_mem = malloc_large_page_withkey(key, size);
1359 }
1360 return large_page_mem ? large_page_mem : malloc_page_withkey(key, size);
1361}
1362
1363/** Dynamically allocates memory backed up by large (huge) pages. In the event
1364 that large (huge) pages are unavailable or disabled explicitly through
1365 os_use_large_pages, it will fallback to dynamic allocation backed by
1366 page-aligned memory.
1367
1368 NOTE: Given that this function will _NOT_ be instrumenting the allocation
1369 through PFS, observability for particular parts of the system which want to
1370 use it will be lost or in best case inaccurate. Please have a strong reason
1371 to do so.
1372
1373 @param[in] size Size of storage (in bytes) requested to be allocated.
1374 @param[in] large_pages_enabled If true, the large pages will be tried to be
1375 used.
1376 @return Pointer to the page-aligned storage. nullptr if dynamic storage
1377 allocation failed.
1378
1379 Example:
1380 int *x = static_cast<int*>(
1381 ut::malloc_large_page(
1382 10*sizeof(int),
1383 fallback_to_normal_page_t{}
1384 )
1385 );
1386 */
1388 std::size_t size, fallback_to_normal_page_t,
1389 bool large_pages_enabled = os_use_large_pages) noexcept {
1392 fallback_to_normal_page_t{}, large_pages_enabled);
1393}
1394
1395/** Retrieves the total amount of bytes that are available for application code
1396 to use.
1397
1398 Amount of bytes returned does _not_ have to match bytes requested
1399 through ut::malloc_large_page*(fallback_to_normal_page_t). This is so
1400 because bytes requested will always be implicitly rounded up to the next
1401 multiple of either huge-page size (e.g. 2MiB) or regular page size (e.g.
1402 4K).
1403
1404 @param[in] ptr Pointer which has been obtained through any of the
1405 ut::malloc_large_page*(fallback_to_normal_page_t) variants.
1406 @return Number of bytes available for use.
1407 */
1408inline size_t large_page_allocation_size(void *ptr,
1409 fallback_to_normal_page_t) noexcept {
1410 assert(ptr);
1412 using large_page_alloc_impl = detail::Large_alloc_<impl>;
1413 if (large_page_alloc_impl::page_type(ptr) == detail::Page_type::system_page)
1414 return ut::page_allocation_size(ptr);
1415 ut_a(large_page_alloc_impl::page_type(ptr) == detail::Page_type::large_page);
1417}
1418
1419/** Retrieves the pointer and size of the allocation provided by the OS. It is a
1420 low level information, and is needed only to call low level memory-related
1421 OS functions.
1422
1423 @param[in] ptr Pointer which has been obtained through any of the
1424 ut::malloc_large_page*(fallback_to_normal_page_t) variants.
1425 @return Low level OS allocation info.
1426 */
1428 void *ptr, fallback_to_normal_page_t) noexcept {
1429 assert(ptr);
1431 using large_page_alloc_impl = detail::Large_alloc_<impl>;
1432 if (large_page_alloc_impl::page_type(ptr) == detail::Page_type::system_page)
1433 return ut::page_low_level_info(ptr);
1434 ut_a(large_page_alloc_impl::page_type(ptr) == detail::Page_type::large_page);
1436}
1437
1438/** Releases storage which has been dynamically allocated through any of
1439 the ut::malloc_large_page*(fallback_to_normal_page_t) variants.
1440
1441 Whether the pointer is representing area backed up by regular or huge-pages,
1442 this function will know the difference and therefore act accordingly.
1443
1444 @param[in] ptr Pointer which has been obtained through any of the
1445 ut::malloc_large_page*(fallback_to_normal_page_t) variants.
1446 @return True if releasing the memory was successful.
1447
1448 Example:
1449 ut::free_large_page(ptr);
1450 */
1451inline bool free_large_page(void *ptr, fallback_to_normal_page_t) noexcept {
1453 using large_page_alloc_impl = detail::Large_alloc_<impl>;
1454
1455 if (!ptr) return false;
1456
1457 bool success;
1458 if (large_page_alloc_impl::page_type(ptr) == detail::Page_type::system_page) {
1459 success = free_page(ptr);
1460 } else {
1461 ut_a(large_page_alloc_impl::page_type(ptr) ==
1463 success = free_large_page(ptr);
1464 }
1465 assert(success);
1466 return success;
1467}
1468
1469/** Dynamically allocates storage of given size and at the address aligned to
1470 the requested alignment. Instruments the memory with given PSI memory key
1471 in case PFS memory support is enabled.
1472
1473 @param[in] key PSI memory key to be used for PFS memory instrumentation.
1474 @param[in] size Size of storage (in bytes) requested to be allocated.
1475 @param[in] alignment Alignment requirement for storage to be allocated.
1476 @return Pointer to the allocated storage. nullptr if dynamic storage
1477 allocation failed.
1478
1479 Example:
1480 int* x = static_cast<int*>(aligned_alloc_withkey(key, 10*sizeof(int), 64));
1481 */
1482inline void *aligned_alloc_withkey(PSI_memory_key_t key, std::size_t size,
1483 std::size_t alignment) noexcept {
1485 using aligned_alloc_impl = detail::Aligned_alloc_<impl>;
1486 return aligned_alloc_impl::alloc<false>(size, alignment, key());
1487}
1488
1489/** Dynamically allocates storage of given size and at the address aligned to
1490 the requested alignment.
1491
1492 NOTE: Given that this function will _NOT_ be instrumenting the allocation
1493 through PFS, observability for particular parts of the system which want to
1494 use it will be lost or in best case inaccurate. Please have a strong reason
1495 to do so.
1496
1497 @param[in] size Size of storage (in bytes) requested to be allocated.
1498 @param[in] alignment Alignment requirement for storage to be allocated.
1499 @return Pointer to the allocated storage. nullptr if dynamic storage
1500 allocation failed.
1501
1502 Example:
1503 int* x = static_cast<int*>(aligned_alloc(10*sizeof(int), 64));
1504 */
1505inline void *aligned_alloc(std::size_t size, std::size_t alignment) noexcept {
1507 alignment);
1508}
1509
1510/** Dynamically allocates zero-initialized storage of given size and at the
1511 address aligned to the requested alignment. Instruments the memory with
1512 given PSI memory key in case PFS memory support is enabled.
1513
1514 @param[in] key PSI memory key to be used for PFS memory instrumentation.
1515 @param[in] size Size of storage (in bytes) requested to be allocated.
1516 @param[in] alignment Alignment requirement for storage to be allocated.
1517 @return Pointer to the zero-initialized allocated storage. nullptr if
1518 dynamic storage allocation failed.
1519
1520 Example:
1521 int* x =
1522 static_cast<int*>(aligned_zalloc_withkey(key, 10*sizeof(int), 64));
1523 */
1524inline void *aligned_zalloc_withkey(PSI_memory_key_t key, std::size_t size,
1525 std::size_t alignment) noexcept {
1527 using aligned_alloc_impl = detail::Aligned_alloc_<impl>;
1528 return aligned_alloc_impl::alloc<true>(size, alignment, key());
1529}
1530
1531/** Dynamically allocates zero-initialized storage of given size and at the
1532 address aligned to the requested alignment.
1533
1534 NOTE: Given that this function will _NOT_ be instrumenting the allocation
1535 through PFS, observability for particular parts of the system which want to
1536 use it will be lost or in best case inaccurate. Please have a strong reason
1537 to do so.
1538
1539 @param[in] size Size of storage (in bytes) requested to be allocated.
1540 @param[in] alignment Alignment requirement for storage to be allocated.
1541 @return Pointer to the zero-initialized allocated storage. nullptr if
1542 dynamic storage allocation failed.
1543
1544 Example:
1545 int* x = static_cast<int*>(aligned_zalloc(10*sizeof(int), 64));
1546 */
1547inline void *aligned_zalloc(std::size_t size, std::size_t alignment) noexcept {
1549 alignment);
1550}
1551
1552/** Releases storage which has been dynamically allocated through any of
1553 the aligned_alloc_*() or aligned_zalloc_*() variants.
1554
1555 @param[in] ptr Pointer which has been obtained through any of the
1556 aligned_alloc_*() or aligned_zalloc_*() variants.
1557
1558 Example:
1559 aligned_free(ptr);
1560 */
1561inline void aligned_free(void *ptr) noexcept {
1563 using aligned_alloc_impl = detail::Aligned_alloc_<impl>;
1565}
1566
1567/** Dynamically allocates storage for an object of type T at address aligned
1568 to the requested alignment. Constructs the object of type T with provided
1569 Args. Instruments the memory with given PSI memory key in case PFS memory
1570 support is enabled.
1571
1572 @param[in] key PSI memory key to be used for PFS memory instrumentation.
1573 @param[in] alignment Alignment requirement for storage to be allocated.
1574 @param[in] args Arguments one wishes to pass over to T constructor(s)
1575 @return Pointer to the allocated storage. Throws std::bad_alloc exception
1576 if dynamic storage allocation could not be fulfilled.
1577
1578 Example 1:
1579 int *ptr = aligned_new_withkey<int>(key, 2);
1580
1581 Example 2:
1582 int *ptr = aligned_new_withkey<int>(key, 2, 10);
1583 assert(*ptr == 10);
1584
1585 Example 3:
1586 struct A { A(int x, int y) : _x(x), _y(y) {} int x, y; }
1587 A *ptr = aligned_new_withkey<A>(key, 2, 1, 2);
1588 assert(ptr->x == 1);
1589 assert(ptr->y == 2);
1590 */
1591template <typename T, typename... Args>
1592inline T *aligned_new_withkey(PSI_memory_key_t key, std::size_t alignment,
1593 Args &&... args) {
1594 auto mem = aligned_alloc_withkey(key, sizeof(T), alignment);
1595 if (unlikely(!mem)) throw std::bad_alloc();
1596 try {
1597 new (mem) T(std::forward<Args>(args)...);
1598 } catch (...) {
1600 throw;
1601 }
1602 return static_cast<T *>(mem);
1603}
1604
1605/** Dynamically allocates storage for an object of type T at address aligned
1606 to the requested alignment. Constructs the object of type T with provided
1607 Args.
1608
1609 NOTE: Given that this function will _NOT_ be instrumenting the allocation
1610 through PFS, observability for particular parts of the system which want to
1611 use it will be lost or in best case inaccurate. Please have a strong reason
1612 to do so.
1613
1614 @param[in] alignment Alignment requirement for storage to be allocated.
1615 @param[in] args Arguments one wishes to pass over to T constructor(s)
1616 @return Pointer to the allocated storage. Throws std::bad_alloc exception
1617 if dynamic storage allocation could not be fulfilled.
1618
1619 Example 1:
1620 int *ptr = aligned_new<int>(2);
1621
1622 Example 2:
1623 int *ptr = aligned_new<int>(2, 10);
1624 assert(*ptr == 10);
1625
1626 Example 3:
1627 struct A { A(int x, int y) : _x(x), _y(y) {} int x, y; }
1628 A *ptr = aligned_new<A>(2, 1, 2);
1629 assert(ptr->x == 1);
1630 assert(ptr->y == 2);
1631 */
1632template <typename T, typename... Args>
1633inline T *aligned_new(std::size_t alignment, Args &&... args) {
1634 return aligned_new_withkey<T>(make_psi_memory_key(PSI_NOT_INSTRUMENTED),
1635 alignment, std::forward<Args>(args)...);
1636}
1637
1638/** Releases storage which has been dynamically allocated through any of
1639 the aligned_new_*() variants. Destructs the object of type T.
1640
1641 @param[in] ptr Pointer which has been obtained through any of the
1642 aligned_new_*() variants
1643
1644 Example:
1645 aligned_delete(ptr);
1646 */
1647template <typename T>
1648inline void aligned_delete(T *ptr) noexcept {
1649 ptr->~T();
1650 aligned_free(ptr);
1651}
1652
1653/** Dynamically allocates storage for an array of T's at address aligned to the
1654 requested alignment. Constructs objects of type T with provided Args.
1655 Arguments that are to be used to construct some respective instance of T
1656 shall be wrapped into a std::tuple. See examples down below. Instruments the
1657 memory with given PSI memory key in case PFS memory support is enabled.
1658
1659 To create an array of default-intialized T's, one can use this function
1660 template but for convenience purposes one can achieve the same by using
1661 the ut::aligned_new_arr_withkey with ut::Count overload.
1662
1663 @param[in] key PSI memory key to be used for PFS memory instrumentation.
1664 @param[in] alignment Alignment requirement for storage to be allocated.
1665 @param[in] args Tuples of arguments one wishes to pass over to T
1666 constructor(s).
1667 @return Pointer to the first element of allocated storage. Throws
1668 std::bad_alloc exception if dynamic storage allocation could not be
1669 fulfilled. Re-throws whatever exception that may have occured during the
1670 construction of any instance of T, in which case it automatically destroys
1671 successfully constructed objects till that moment (if any), and finally
1672 cleans up the raw memory allocated for T instances.
1673
1674 Example 1:
1675 int *ptr = ut::aligned_new_arr_withkey<int>(key, 32,
1676 std::forward_as_tuple(1),
1677 std::forward_as_tuple(2));
1678 assert(ptr[0] == 1);
1679 assert(ptr[1] == 2);
1680
1681 Example 2:
1682 struct A {
1683 A(int x, int y) : _x(x), _y(y) {}
1684 int _x, _y;
1685 };
1686 A *ptr = ut::aligned_new_arr_withkey<A>(key, 32,
1687 std::forward_as_tuple(0, 1), std::forward_as_tuple(2, 3),
1688 std::forward_as_tuple(4, 5), std::forward_as_tuple(6, 7),
1689 std::forward_as_tuple(8, 9));
1690 assert(ptr[0]->_x == 0 && ptr[0]->_y == 1);
1691 assert(ptr[1]->_x == 2 && ptr[1]->_y == 3);
1692 assert(ptr[2]->_x == 4 && ptr[2]->_y == 5);
1693 assert(ptr[3]->_x == 6 && ptr[3]->_y == 7);
1694 assert(ptr[4]->_x == 8 && ptr[4]->_y == 9);
1695
1696 Example 3:
1697 struct A {
1698 A() : _x(10), _y(100) {}
1699 A(int x, int y) : _x(x), _y(y) {}
1700 int _x, _y;
1701 };
1702 A *ptr = ut::aligned_new_arr_withkey<A>(key, 32,
1703 std::forward_as_tuple(0, 1), std::forward_as_tuple(2, 3),
1704 std::forward_as_tuple(), std::forward_as_tuple(6, 7),
1705 std::forward_as_tuple());
1706 assert(ptr[0]->_x == 0 && ptr[0]->_y == 1);
1707 assert(ptr[1]->_x == 2 && ptr[1]->_y == 3);
1708 assert(ptr[2]->_x == 10 && ptr[2]->_y == 100);
1709 assert(ptr[3]->_x == 6 && ptr[3]->_y == 7);
1710 assert(ptr[4]->_x == 10 && ptr[4]->_y == 100);
1711 */
1712template <typename T, typename... Args>
1713inline T *aligned_new_arr_withkey(PSI_memory_key_t key, std::size_t alignment,
1714 Args &&... args) {
1715 auto mem = aligned_alloc_withkey(key, sizeof(T) * sizeof...(args), alignment);
1716 if (unlikely(!mem)) throw std::bad_alloc();
1717
1718 size_t idx = 0;
1719 try {
1720 (...,
1721 detail::construct<T>(mem, sizeof(T) * idx++, std::forward<Args>(args)));
1722 } catch (...) {
1723 for (size_t offset = (idx - 1) * sizeof(T); offset != 0;
1724 offset -= sizeof(T)) {
1725 reinterpret_cast<T *>(reinterpret_cast<std::uintptr_t>(mem) + offset -
1726 sizeof(T))
1727 ->~T();
1728 }
1730 throw;
1731 }
1732 return static_cast<T *>(mem);
1733}
1734
1735/** Dynamically allocates storage for an array of T's at address aligned to the
1736 requested alignment. Constructs objects of type T using default constructor.
1737 If T cannot be default-initialized (e.g. default constructor does not
1738 exist), then this interace cannot be used for constructing such an array.
1739 ut::new_arr_withkey overload with user-provided initialization must be used
1740 then. Instruments the memory with given PSI memory key in case PFS memory
1741 support is enabled.
1742
1743 @param[in] key PSI memory key to be used for PFS memory instrumentation.
1744 @param[in] alignment Alignment requirement for storage to be allocated.
1745 @param[in] count Number of T elements in an array.
1746 @return Pointer to the first element of allocated storage. Throws
1747 std::bad_alloc exception if dynamic storage allocation could not be
1748 fulfilled. Re-throws whatever exception that may have occured during the
1749 construction of any instance of T, in which case it automatically destroys
1750 successfully constructed objects till that moment (if any), and finally
1751 cleans up the raw memory allocated for T instances.
1752
1753 Example 1:
1754 int *ptr = ut::aligned_new_arr_withkey<int>(key, 32, ut::Count{2});
1755
1756 Example 2:
1757 struct A {
1758 A() : _x(10), _y(100) {}
1759 int _x, _y;
1760 };
1761 A *ptr = ut::aligned_new_arr_withkey<A>(key, 32, ut::Count{5});
1762 assert(ptr[0]->_x == 10 && ptr[0]->_y == 100);
1763 assert(ptr[1]->_x == 10 && ptr[1]->_y == 100);
1764 assert(ptr[2]->_x == 10 && ptr[2]->_y == 100);
1765 assert(ptr[3]->_x == 10 && ptr[3]->_y == 100);
1766 assert(ptr[4]->_x == 10 && ptr[4]->_y == 100);
1767
1768 Example 3:
1769 struct A {
1770 A(int x, int y) : _x(x), _y(y) {}
1771 int _x, _y;
1772 };
1773 // Following cannot compile because A is not default-constructible
1774 A *ptr = ut::aligned_new_arr_withkey<A>(key, 32, ut::Count{5});
1775 */
1776template <typename T>
1777inline T *aligned_new_arr_withkey(PSI_memory_key_t key, std::size_t alignment,
1778 Count count) {
1779 auto mem = aligned_alloc_withkey(key, sizeof(T) * count(), alignment);
1780 if (unlikely(!mem)) throw std::bad_alloc();
1781
1782 size_t offset = 0;
1783 try {
1784 for (; offset < sizeof(T) * count(); offset += sizeof(T)) {
1785 new (reinterpret_cast<uint8_t *>(mem) + offset) T{};
1786 }
1787 } catch (...) {
1788 for (; offset != 0; offset -= sizeof(T)) {
1789 reinterpret_cast<T *>(reinterpret_cast<std::uintptr_t>(mem) + offset -
1790 sizeof(T))
1791 ->~T();
1792 }
1794 throw;
1795 }
1796 return static_cast<T *>(mem);
1797}
1798
1799/** Dynamically allocates storage for an array of T's at address aligned to
1800 the requested alignment. Constructs objects of type T with provided Args.
1801
1802 NOTE: Given that this function will _NOT_ be instrumenting the allocation
1803 through PFS, observability for particular parts of the system which want to
1804 use it will be lost or in best case inaccurate. Please have a strong reason
1805 to do so.
1806
1807 @param[in] alignment Alignment requirement for storage to be allocated.
1808 @param[in] args Arguments one wishes to pass over to T constructor(s)
1809 @return Pointer to the first element of allocated storage. Throws
1810 std::bad_alloc exception if dynamic storage allocation could not be
1811 fulfilled.
1812
1813 Example 1:
1814 int *ptr = aligned_new_arr<int, 5>(2);
1815 ptr[0] ... ptr[4]
1816
1817 Example 2:
1818 int *ptr = aligned_new_arr<int, 5>(2, 1, 2, 3, 4, 5);
1819 assert(*ptr[0] == 1);
1820 assert(*ptr[1] == 2);
1821 ...
1822 assert(*ptr[4] == 5);
1823
1824 Example 3:
1825 struct A { A(int x, int y) : _x(x), _y(y) {} int x, y; }
1826 A *ptr = aligned_new_arr<A, 5>(2, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10);
1827 assert(ptr[0]->x == 1);
1828 assert(ptr[0]->y == 2);
1829 assert(ptr[1]->x == 3);
1830 assert(ptr[1]->y == 4);
1831 ...
1832 assert(ptr[4]->x == 9);
1833 assert(ptr[4]->y == 10);
1834 */
1835template <typename T, typename... Args>
1836inline T *aligned_new_arr(std::size_t alignment, Args &&... args) {
1837 return aligned_new_arr_withkey<T>(make_psi_memory_key(PSI_NOT_INSTRUMENTED),
1838 alignment, std::forward<Args>(args)...);
1839}
1840
1841/** Dynamically allocates storage for an array of T's at address aligned to
1842 the requested alignment. Constructs objects of type T using default
1843 constructor.
1844
1845 NOTE: Given that this function will _NOT_ be instrumenting the allocation
1846 through PFS, observability for particular parts of the system which want to
1847 use it will be lost or in best case inaccurate. Please have a strong reason
1848 to do so.
1849
1850 @param[in] alignment Alignment requirement for storage to be allocated.
1851 @param[in] count Number of T elements in an array.
1852 @return Pointer to the first element of allocated storage. Throws
1853 std::bad_alloc exception if dynamic storage allocation could not be
1854 fulfilled.
1855
1856 Example 1:
1857 int *ptr = aligned_new_arr<int>(2, 5);
1858 assert(*ptr[0] == 0);
1859 assert(*ptr[1] == 0);
1860 ...
1861 assert(*ptr[4] == 0);
1862
1863 Example 2:
1864 struct A { A) : x(1), y(2) {} int x, y; }
1865 A *ptr = aligned_new_arr<A>(2, 5);
1866 assert(ptr[0].x == 1);
1867 assert(ptr[0].y == 2);
1868 ...
1869 assert(ptr[4].x == 1);
1870 assert(ptr[4].y == 2);
1871
1872 Example 3:
1873 struct A { A(int x, int y) : _x(x), _y(y) {} int x, y; }
1874 A *ptr = aligned_new_arr<A>(2, 5);
1875 // will not compile, no default constructor
1876 */
1877template <typename T>
1878inline T *aligned_new_arr(std::size_t alignment, Count count) {
1879 return aligned_new_arr_withkey<T>(make_psi_memory_key(PSI_NOT_INSTRUMENTED),
1880 alignment, count);
1881}
1882
1883/** Releases storage which has been dynamically allocated through any of the
1884 aligned_new_arr_*() variants. Destructs all objects of type T.
1885
1886 @param[in] ptr Pointer which has been obtained through any of the
1887 aligned_new_arr_*() variants.
1888
1889 Example:
1890 aligned_delete_arr(ptr);
1891 */
1892template <typename T>
1893inline void aligned_delete_arr(T *ptr) noexcept {
1895 using aligned_alloc_impl = detail::Aligned_alloc_<impl>;
1896 const auto data_len = aligned_alloc_impl::datalen(ptr);
1897 for (size_t offset = 0; offset < data_len; offset += sizeof(T)) {
1898 reinterpret_cast<T *>(reinterpret_cast<std::uintptr_t>(ptr) + offset)->~T();
1899 }
1900 aligned_free(ptr);
1901}
1902
1903/** Lightweight convenience wrapper which manages dynamically allocated
1904 over-aligned type. Wrapper makes use of RAII to do the resource cleanup.
1905
1906 Example usage:
1907 struct My_fancy_type {
1908 My_fancy_type(int x, int y) : _x(x), _y(y) {}
1909 int _x, _y;
1910 };
1911
1912 aligned_pointer<My_fancy_type, 32> ptr;
1913 ptr.alloc(10, 5);
1914 My_fancy_type *p = ptr;
1915 assert(p->_x == 10 && p->_y == 5);
1916
1917 @tparam T Type to be managed.
1918 @tparam Alignment Number of bytes to align the type T to.
1919 */
1920template <typename T, size_t Alignment>
1922 T *ptr = nullptr;
1923
1924 public:
1925 /** Destructor. Invokes destructor of the underlying instance of
1926 type T. Releases dynamically allocated resources, if there had been
1927 left any.
1928 */
1930 if (ptr) dealloc();
1931 }
1932
1933 /** Allocates sufficiently large memory of dynamic storage duration to fit
1934 the instance of type T at the address which is aligned to Alignment bytes.
1935 Constructs the instance of type T with given Args.
1936
1937 Underlying instance of type T is accessed through the conversion operator.
1938
1939 @param[in] args Any number and type of arguments that type T can be
1940 constructed with.
1941 */
1942 template <typename... Args>
1943 void alloc(Args &&... args) {
1944 ut_ad(ptr == nullptr);
1945 ptr = ut::aligned_new<T>(Alignment, args...);
1946 }
1947
1948 /** Allocates sufficiently large memory of dynamic storage duration to fit
1949 the instance of type T at the address which is aligned to Alignment bytes.
1950 Constructs the instance of type T with given Args. Instruments the memory
1951 with given PSI memory key in case PFS memory support is enabled.
1952
1953 Underlying instance of type T is accessed through the conversion operator.
1954
1955 @param[in] key PSI memory key to be used for PFS memory instrumentation.
1956 @param[in] args Any number and type of arguments that type T can be
1957 constructed with.
1958 */
1959 template <typename... Args>
1960 void alloc_withkey(PSI_memory_key_t key, Args &&... args) {
1961 ut_ad(ptr == nullptr);
1962 ptr =
1963 ut::aligned_new_withkey<T>(key, Alignment, std::forward<Args>(args)...);
1964 }
1965
1966 /** Invokes the destructor of instance of type T, if applicable.
1967 Releases the resources previously allocated with alloc().
1968 */
1969 void dealloc() {
1970 ut_ad(ptr != nullptr);
1972 ptr = nullptr;
1973 }
1974
1975 /** Conversion operator. Used for accessing the underlying instance of
1976 type T.
1977 */
1978 operator T *() const {
1979 ut_ad(ptr != nullptr);
1980 return ptr;
1981 }
1982};
1983
1984/** Lightweight convenience wrapper which manages a dynamically
1985 allocated array of over-aligned types. Only the first element of an array is
1986 guaranteed to be aligned to the requested Alignment. Wrapper makes use of
1987 RAII to do the resource cleanup.
1988
1989 Example usage 1:
1990 struct My_fancy_type {
1991 My_fancy_type() : _x(0), _y(0) {}
1992 My_fancy_type(int x, int y) : _x(x), _y(y) {}
1993 int _x, _y;
1994 };
1995
1996 aligned_array_pointer<My_fancy_type, 32> ptr;
1997 ptr.alloc(3);
1998 My_fancy_type *p = ptr;
1999 assert(p[0]._x == 0 && p[0]._y == 0);
2000 assert(p[1]._x == 0 && p[1]._y == 0);
2001 assert(p[2]._x == 0 && p[2]._y == 0);
2002
2003 Example usage 2:
2004 aligned_array_pointer<My_fancy_type, 32> ptr;
2005 ptr.alloc<3>(1, 2, 3, 4, 5, 6);
2006 My_fancy_type *p = ptr;
2007 assert(p[0]._x == 1 && p[0]._y == 2);
2008 assert(p[1]._x == 3 && p[1]._y == 4);
2009 assert(p[2]._x == 5 && p[2]._y == 6);
2010
2011 @tparam T Type to be managed.
2012 @tparam Alignment Number of bytes to align the first element of array to.
2013 */
2014template <typename T, size_t Alignment>
2016 T *ptr = nullptr;
2017
2018 public:
2019 /** Destructor. Invokes destructors of the underlying instances of
2020 type T. Releases dynamically allocated resources, if there had been
2021 left any.
2022 */
2024 if (ptr) dealloc();
2025 }
2026
2027 /** Allocates sufficiently large memory of dynamic storage duration to fit
2028 the array of size number of elements of type T at the address which is
2029 aligned to Alignment bytes. Constructs the size number of instances of
2030 type T, each being initialized through the means of default constructor.
2031
2032 Underlying instances of type T are accessed through the conversion
2033 operator.
2034
2035 @param[in] count Number of T elements in an array.
2036 */
2038 ut_ad(ptr == nullptr);
2039 ptr = ut::aligned_new_arr<T>(Alignment, count);
2040 }
2041
2042 /** Allocates sufficiently large memory of dynamic storage duration to fit
2043 the array of size number of elements of type T at the address which is
2044 aligned to Alignment bytes. Constructs the size number of instances of
2045 type T, each being initialized through the means of provided Args and
2046 corresponding constructors.
2047
2048 Underlying instances of type T are accessed through the conversion
2049 operator.
2050
2051 @param[in] args Any number and type of arguments that type T can be
2052 constructed with.
2053 */
2054 template <typename... Args>
2055 void alloc(Args &&... args) {
2056 ut_ad(ptr == nullptr);
2057 ptr = ut::aligned_new_arr<T>(Alignment, std::forward<Args>(args)...);
2058 }
2059
2060 /** Allocates sufficiently large memory of dynamic storage duration to fit
2061 the array of size number of elements of type T at the address which is
2062 aligned to Alignment bytes. Constructs the size number of instances of
2063 type T, each being initialized through the means of default constructor.
2064 Instruments the memory with given PSI memory key in case PFS memory
2065 support is enabled.
2066
2067 Underlying instances of type T are accessed through the conversion
2068 operator.
2069
2070 @param[in] key PSI memory key to be used for PFS memory instrumentation.
2071 @param[in] count Number of T elements in an array.
2072 */
2074 ut_ad(ptr == nullptr);
2075 ptr = ut::aligned_new_arr_withkey<T>(key, Alignment, count);
2076 }
2077
2078 /** Allocates sufficiently large memory of dynamic storage duration to fit
2079 the array of size number of elements of type T at the address which is
2080 aligned to Alignment bytes. Constructs the size number of instances of
2081 type T, each being initialized through the means of provided Args and
2082 corresponding constructors. Instruments the memory with given PSI memory
2083 key in case PFS memory support is enabled.
2084
2085 Underlying instances of type T are accessed through the conversion
2086 operator.
2087
2088 @param[in] key PSI memory key to be used for PFS memory instrumentation.
2089 @param[in] args Any number and type of arguments that type T can be
2090 constructed with.
2091 */
2092 template <typename... Args>
2093 void alloc_withkey(PSI_memory_key_t key, Args &&... args) {
2094 ut_ad(ptr == nullptr);
2095 ptr = ut::aligned_new_arr_withkey<T>(key, Alignment,
2096 std::forward<Args>(args)...);
2097 }
2098
2099 /** Invokes destructors of instances of type T, if applicable.
2100 Releases the resources previously allocated with any variant of
2101 alloc().
2102 */
2103 void dealloc() {
2105 ptr = nullptr;
2106 }
2107
2108 /** Conversion operator. Used for accessing the underlying instances of
2109 type T.
2110 */
2111 operator T *() const {
2112 ut_ad(ptr != nullptr);
2113 return ptr;
2114 }
2115};
2116
2117namespace detail {
2118template <typename T>
2120 explicit allocator_base(PSI_memory_key /*key*/) {}
2121
2122 template <typename U>
2124
2125 void *allocate_impl(size_t n_bytes) { return ut::malloc(n_bytes); }
2126};
2127
2128template <typename T>
2131
2132 template <typename U>
2134 : allocator_base_pfs(other.get_mem_key()) {}
2135
2137
2138 void *allocate_impl(size_t n_bytes) {
2140 }
2141
2142 private:
2144};
2145} // namespace detail
2146
2147/** Allocator that allows std::* containers to manage their memory through
2148 ut::malloc* and ut::free library functions.
2149
2150 Main purpose of this custom allocator is to instrument all of the memory
2151 allocations and deallocations that are being done by std::* containers under
2152 the hood, and have them recorded through the PFS (memory) engine.
2153
2154 Other than std::* containers, this allocator is of course also suitable for
2155 use in any other allocator-aware containers and/or code.
2156
2157 Given that ut::malloc* and ut::free library functions already handle all
2158 the PFS and non-PFS implementation bits and pieces, this allocator is a mere
2159 wrapper around them.
2160
2161 Example which uses default PFS key (mem_key_std) to trace all std::vector
2162 allocations and deallocations:
2163 std::vector<int, ut::allocator<int>> vec;
2164 vec.push_back(...);
2165 ...
2166 vec.push_back(...);
2167
2168 Example which uses user-provided PFS key to trace std::vector allocations
2169 and deallocations:
2170 ut::allocator<int> allocator(some_other_psi_key);
2171 std::vector<int, ut::allocator<int>> vec(allocator);
2172 vec.push_back(...);
2173 ...
2174 vec.push_back(...);
2175 */
2176template <typename T, typename Allocator_base = std::conditional_t<
2179class allocator : public Allocator_base {
2180 public:
2181 using pointer = T *;
2182 using const_pointer = const T *;
2183 using reference = T &;
2184 using const_reference = const T &;
2185 using value_type = T;
2186 using size_type = size_t;
2187 using difference_type = ptrdiff_t;
2188
2189 static_assert(alignof(T) <= alignof(std::max_align_t),
2190 "ut::allocator does not support over-aligned types. Use "
2191 "ut::aligned_* API to handle such types.");
2192
2193 /** Default constructor.
2194 @param[in] key performance schema key.
2195 */
2196 explicit allocator(PSI_memory_key key = mem_key_std) : Allocator_base(key) {}
2197
2198 /* Rule-of-five */
2201 const allocator<T, Allocator_base> &) = default;
2204 default;
2205 ~allocator() = default;
2206
2207 /** Copy-construct a new instance of allocator with type T by using existing
2208 instance of allocator constructed with a different type U.
2209 @param[in] other the allocator to copy from.
2210 */
2211 template <typename U>
2213 : Allocator_base(other) {}
2214
2215 /* NOTE: rebind is deprecated in C++17 and to be removed in C++20 but one of
2216 our toolchains, when used in 32-bit setting, still does not support custom
2217 allocators that do not provide rebind support explicitly. In future, this
2218 part will become redundant and can be removed.
2219 */
2220 template <typename U>
2221 struct rebind {
2223 };
2224
2225 /** Equality of allocators instantiated with same types T. */
2227 return true;
2228 }
2229 /** Non-equality of allocators instantiated with same types T. */
2230 inline bool operator!=(const ut::allocator<T, Allocator_base> &other) const {
2231 return !(*this == other);
2232 }
2233
2234 /** Return the maximum number of objects that can be allocated by
2235 this allocator. This number is somewhat lower for PFS-enabled
2236 builds because of extra few bytes needed for PFS.
2237 */
2239 const size_type s_max = std::numeric_limits<size_type>::max();
2240 return (s_max - ut::pfs_overhead()) / sizeof(T);
2241 }
2242
2243 /** Allocates chunk of memory that can hold n_elements objects of
2244 type T. Returned pointer is always valid. In case underlying
2245 allocation function was not able to fulfill the allocation request,
2246 this function will throw std::bad_alloc exception. After successful
2247 allocation, returned pointer must be passed back to
2248 ut::allocator<T>::deallocate() when no longer needed.
2249
2250 @param[in] n_elements number of elements
2251 @param[in] hint pointer to a nearby memory location,
2252 not used by this implementation
2253 @return pointer to the allocated memory
2254 */
2256 const_pointer hint [[maybe_unused]] = nullptr) {
2257 if (unlikely(n_elements > max_size())) {
2258 throw std::bad_array_new_length();
2259 }
2260
2261 auto ptr = Allocator_base::allocate_impl(n_elements * sizeof(T));
2262
2263 if (unlikely(!ptr)) {
2264 throw std::bad_alloc();
2265 }
2266
2267 return static_cast<pointer>(ptr);
2268 }
2269
2270 /** Releases the memory allocated through ut::allocator<T>::allocate().
2271
2272 @param[in,out] ptr pointer to memory to free
2273 @param[in] n_elements number of elements allocated (unused)
2274 */
2275 void deallocate(pointer ptr, size_type n_elements [[maybe_unused]] = 0) {
2276 ut::free(ptr);
2277 }
2278};
2279
2280namespace detail {
2281template <typename>
2282constexpr bool is_unbounded_array_v = false;
2283template <typename T>
2284constexpr bool is_unbounded_array_v<T[]> = true;
2285
2286template <typename>
2287constexpr bool is_bounded_array_v = false;
2288template <typename T, std::size_t N>
2289constexpr bool is_bounded_array_v<T[N]> = true;
2290
2291template <typename>
2292constexpr size_t bounded_array_size_v = 0;
2293template <typename T, std::size_t N>
2295
2296template <typename T>
2297struct Deleter {
2298 void operator()(T *ptr) { ut::delete_(ptr); }
2299};
2300
2301template <typename T>
2303 void operator()(T *ptr) { ut::delete_arr(ptr); }
2304};
2305
2306template <typename T>
2308 void operator()(T *ptr) { ut::aligned_delete(ptr); }
2309};
2310
2311template <typename T>
2313 void operator()(T *ptr) { ut::aligned_delete_arr(ptr); }
2314};
2315
2316} // namespace detail
2317
2318/** Dynamically allocates storage for an object of type T. Constructs the object
2319 of type T with provided Args. Wraps the pointer to T instance into the
2320 std::unique_ptr.
2321
2322 This overload participates in overload resolution only if T
2323 is not an array type.
2324
2325 NOTE: Given that this function will _NOT_ be instrumenting the allocation
2326 through PFS, observability for particular parts of the system which want to
2327 use it will be lost or in best case inaccurate. Please have a strong reason
2328 to do so.
2329
2330 @param[in] args Arguments one wishes to pass over to T constructor(s) .
2331 @return std::unique_ptr holding a pointer to instance of T.
2332 */
2333template <typename T, typename Deleter = detail::Deleter<T>, typename... Args>
2334std::enable_if_t<!std::is_array<T>::value, std::unique_ptr<T, Deleter>>
2335make_unique(Args &&... args) {
2336 return std::unique_ptr<T, Deleter>(ut::new_<T>(std::forward<Args>(args)...));
2337}
2338
2339/** Dynamically allocates storage for an object of type T. Constructs the object
2340 of type T with provided Args. Wraps the pointer to T instance into the
2341 std::unique_ptr with custom deleter which knows how to handle PFS-enabled
2342 dynamic memory allocations. Instruments the memory with given PSI memory key
2343 in case PFS memory support is enabled.
2344
2345 This overload participates in overload resolution only if T
2346 is not an array type.
2347
2348 @param[in] key PSI memory key to be used for PFS memory instrumentation.
2349 @param[in] args Arguments one wishes to pass over to T constructor(s) .
2350 @return std::unique_ptr holding a pointer to instance of T.
2351 */
2352template <typename T, typename Deleter = detail::Deleter<T>, typename... Args>
2353std::enable_if_t<!std::is_array<T>::value, std::unique_ptr<T, Deleter>>
2355 return std::unique_ptr<T, Deleter>(
2356 ut::new_withkey<T>(key, std::forward<Args>(args)...));
2357}
2358
2359/** Dynamically allocates storage for an object of type T. Constructs the object
2360 of type T with provided Args. Wraps the pointer to an array of T instance
2361 into the std::unique_ptr.
2362
2363 This overload participates in overload resolution only if T
2364 is an array type with unknown compile-time bound.
2365
2366 NOTE: Given that this function will _NOT_ be instrumenting the allocation
2367 through PFS, observability for particular parts of the system which want to
2368 use it will be lost or in best case inaccurate. Please have a strong reason
2369 to do so.
2370
2371 @return std::unique_ptr holding a pointer to an array of size instances of
2372 T.
2373 */
2374template <typename T,
2375 typename Deleter = detail::Array_deleter<std::remove_extent_t<T>>>
2376std::enable_if_t<detail::is_unbounded_array_v<T>, std::unique_ptr<T, Deleter>>
2377make_unique(size_t size) {
2378 return std::unique_ptr<T, Deleter>(
2379 ut::new_arr<std::remove_extent_t<T>>(ut::Count{size}));
2380}
2381
2382/** Dynamically allocates storage for an object of type T. Constructs the object
2383 of type T with provided Args. Wraps the pointer to an array of T instances
2384 into the std::unique_ptr with custom deleter which knows how to handle
2385 PFS-enabled dyanmic memory allocations. Instruments the memory with given
2386 PSI memory key in case PFS memory support is enabled.
2387
2388 This overload participates in overload resolution only if T
2389 is an array type with unknown compile-time bound.
2390
2391 @return std::unique_ptr holding a pointer to an array of size instances of
2392 T.
2393 */
2394template <typename T,
2395 typename Deleter = detail::Array_deleter<std::remove_extent_t<T>>>
2396std::enable_if_t<detail::is_unbounded_array_v<T>, std::unique_ptr<T, Deleter>>
2398 return std::unique_ptr<T, Deleter>(
2399 ut::new_arr_withkey<std::remove_extent_t<T>>(key, ut::Count{size}));
2400}
2401
2402/** std::unique_ptr for arrays of known compile-time bound are disallowed.
2403
2404 For more details see 4.3 paragraph from
2405 http://www.open-std.org/jtc1/sc22/wg21/docs/papers/2013/n3588.txt
2406 */
2407template <typename T, typename... Args>
2408std::enable_if_t<detail::is_bounded_array_v<T>> make_unique(Args &&...) =
2409 delete;
2410
2411/** std::unique_ptr in PFS-enabled builds for arrays of known compile-time bound
2412 are disallowed.
2413
2414 For more details see 4.3 paragraph from
2415 http://www.open-std.org/jtc1/sc22/wg21/docs/papers/2013/n3588.txt
2416 */
2417template <typename T, typename... Args>
2418std::enable_if_t<detail::is_bounded_array_v<T>> make_unique(
2419 PSI_memory_key_t key, Args &&...) = delete;
2420
2421/** The following is a common type that is returned by all the ut::make_unique
2422 (non-aligned) specializations listed above. This is effectively a if-ladder
2423 for the following list of conditions on the input type:
2424 !std::is_array<T>::value -> std::unique_ptr<T, detail::Deleter<T>>
2425 detail::is_unbounded_array_v<T> ->
2426 std::unique_ptr<T,detail::Array_deleter<std::remove_extent_t<T>>> else (or
2427 else if detail::is_bounded_array_v<T>) -> void (we do not support bounded
2428 array ut::make_unique)
2429 */
2430template <typename T>
2431using unique_ptr = std::conditional_t<
2432 !std::is_array<T>::value, std::unique_ptr<T, detail::Deleter<T>>,
2433 std::conditional_t<
2434 detail::is_unbounded_array_v<T>,
2435 std::unique_ptr<T, detail::Array_deleter<std::remove_extent_t<T>>>,
2436 void>>;
2437
2438/** Dynamically allocates storage for an object of type T at address aligned to
2439 the requested alignment. Constructs the object of type T with provided Args.
2440 Wraps the pointer to T instance into the std::unique_ptr.
2441
2442 This overload participates in overload resolution only if T
2443 is not an array type.
2444
2445 NOTE: Given that this function will _NOT_ be instrumenting the allocation
2446 through PFS, observability for particular parts of the system which want to
2447 use it will be lost or in best case inaccurate. Please have a strong reason
2448 to do so.
2449
2450 @param[in] alignment Alignment requirement for storage to be allocated.
2451 @param[in] args Arguments one wishes to pass over to T constructor(s) .
2452 @return std::unique_ptr holding a pointer to instance of T.
2453 */
2454template <typename T, typename Deleter = detail::Aligned_deleter<T>,
2455 typename... Args>
2456std::enable_if_t<!std::is_array<T>::value, std::unique_ptr<T, Deleter>>
2457make_unique_aligned(size_t alignment, Args &&... args) {
2458 return std::unique_ptr<T, Deleter>(
2459 ut::aligned_new<T>(alignment, std::forward<Args>(args)...));
2460}
2461
2462/** Dynamically allocates storage for an array of objects of type T at address
2463 aligned to the requested alignment. Constructs the object of type T with
2464 provided Args. Wraps the pointer to T instance into the std::unique_ptr with
2465 custom deleter which knows how to handle PFS-enabled dynamic memory
2466 allocations. Instruments the memory with given PSI memory key in case PFS
2467 memory support is enabled.
2468
2469 This overload participates in overload resolution only if T is not an array
2470 type.
2471
2472 @param[in] key PSI memory key to be used for PFS memory instrumentation.
2473 @param[in] alignment Alignment requirement for storage to be allocated.
2474 @param[in] args Arguments one wishes to pass over to T constructor(s) .
2475 @return std::unique_ptr holding a pointer to instance of T.
2476 */
2477template <typename T, typename Deleter = detail::Aligned_deleter<T>,
2478 typename... Args>
2479std::enable_if_t<!std::is_array<T>::value, std::unique_ptr<T, Deleter>>
2480make_unique_aligned(PSI_memory_key_t key, size_t alignment, Args &&... args) {
2481 return std::unique_ptr<T, Deleter>(
2482 ut::aligned_new_withkey<T>(key, alignment, std::forward<Args>(args)...));
2483}
2484
2485/** Dynamically allocates storage for an array of requested size of objects of
2486 type T at address aligned to the requested alignment. Constructs the object
2487 of type T with provided Args. Wraps the pointer to an array of T instance
2488 into the std::unique_ptr.
2489
2490 This overload participates in overload resolution only if T
2491 is an array type with unknown compile-time bound.
2492
2493 NOTE: Given that this function will _NOT_ be instrumenting the allocation
2494 through PFS, observability for particular parts of the system which want to
2495 use it will be lost or in best case inaccurate. Please have a strong reason
2496 to do so.
2497
2498 @param[in] alignment Alignment requirement for storage to be allocated.
2499 @param[in] size Size of the array of objects T to allocate.
2500 @return std::unique_ptr holding a pointer to an array of size instances of
2501 T.
2502 */
2503template <typename T, typename Deleter = detail::Aligned_array_deleter<
2504 std::remove_extent_t<T>>>
2505std::enable_if_t<detail::is_unbounded_array_v<T>, std::unique_ptr<T, Deleter>>
2506make_unique_aligned(size_t alignment, size_t size) {
2507 return std::unique_ptr<T, Deleter>(
2508 ut::aligned_new_arr<std::remove_extent_t<T>>(alignment, ut::Count{size}));
2509}
2510
2511/** Dynamically allocates storage for an array of requested size of objects of
2512 type T at address aligned to the requested alignment. Constructs the object
2513 of type T with provided Args. Wraps the pointer to an array of T instances
2514 into the std::unique_ptr with custom deleter which knows how to handle
2515 PFS-enabled dynamic memory allocations. Instruments the memory with given
2516 PSI memory key in case PFS memory support is enabled.
2517
2518 This overload participates in overload resolution only if T
2519 is an array type with unknown compile-time bound.
2520
2521 @param[in] key PSI memory key to be used for PFS memory instrumentation.
2522 @param[in] alignment Alignment requirement for storage to be allocated.
2523 @param[in] size Size of the array of objects T to allocate.
2524 @return std::unique_ptr holding a pointer to an array of size instances of
2525 T.
2526 */
2527template <typename T, typename Deleter = detail::Aligned_array_deleter<
2528 std::remove_extent_t<T>>>
2529std::enable_if_t<detail::is_unbounded_array_v<T>, std::unique_ptr<T, Deleter>>
2530make_unique_aligned(PSI_memory_key_t key, size_t alignment, size_t size) {
2531 return std::unique_ptr<T, Deleter>(
2532 ut::aligned_new_arr_withkey<std::remove_extent_t<T>>(key, alignment,
2533 ut::Count{size}));
2534}
2535
2536/** std::unique_ptr for arrays of known compile-time bound are disallowed.
2537
2538 For more details see 4.3 paragraph from
2539 http://www.open-std.org/jtc1/sc22/wg21/docs/papers/2013/n3588.txt
2540 */
2541template <typename T, typename... Args>
2542std::enable_if_t<detail::is_bounded_array_v<T>> make_unique_aligned(
2543 Args &&...) = delete;
2544
2545/** std::unique_ptr in PFS-enabled builds for arrays of known compile-time bound
2546 are disallowed.
2547
2548 For more details see 4.3 paragraph from
2549 http://www.open-std.org/jtc1/sc22/wg21/docs/papers/2013/n3588.txt
2550 */
2551template <typename T, typename... Args>
2552std::enable_if_t<detail::is_bounded_array_v<T>> make_unique_aligned(
2553 PSI_memory_key_t key, Args &&...) = delete;
2554
2555/** Dynamically allocates storage for an object of type T. Constructs the object
2556 of type T with provided Args. Wraps the pointer to T instance into the
2557 std::shared_ptr.
2558
2559 This overload participates in overload resolution only if T
2560 is not an array type.
2561
2562 NOTE: Given that this function will _NOT_ be instrumenting the allocation
2563 through PFS, observability for particular parts of the system which want to
2564 use it will be lost or in best case inaccurate. Please have a strong reason
2565 to do so.
2566
2567 @param[in] args Arguments one wishes to pass over to T constructor(s) .
2568 @return std::shared_ptr holding a pointer to instance of T.
2569 */
2570template <typename T, typename Deleter = detail::Deleter<T>, typename... Args>
2571std::enable_if_t<!std::is_array<T>::value, std::shared_ptr<T>> make_shared(
2572 Args &&... args) {
2573 return std::shared_ptr<T>(ut::new_<T>(std::forward<Args>(args)...),
2574 Deleter{});
2575}
2576
2577/** Dynamically allocates storage for an object of type T. Constructs the object
2578 of type T with provided Args. Wraps the pointer to T instance into the
2579 std::shared_ptr with custom deleter which knows how to handle PFS-enabled
2580 dynamic memory allocations. Instruments the memory with given PSI memory key
2581 in case PFS memory support is enabled.
2582
2583 This overload participates in overload resolution only if T
2584 is not an array type.
2585
2586 @param[in] key PSI memory key to be used for PFS memory instrumentation.
2587 @param[in] args Arguments one wishes to pass over to T constructor(s) .
2588 @return std::shared_ptr holding a pointer to instance of T.
2589 */
2590template <typename T, typename Deleter = detail::Deleter<T>, typename... Args>
2591std::enable_if_t<!std::is_array<T>::value, std::shared_ptr<T>> make_shared(
2592 PSI_memory_key_t key, Args &&... args) {
2593 return std::shared_ptr<T>(
2594 ut::new_withkey<T>(key, std::forward<Args>(args)...), Deleter{});
2595}
2596
2597/** Dynamically allocates storage for an array of requested size of objects of
2598 type T. Constructs the object of type T with provided Args. Wraps the
2599 pointer to an array of T instance into the std::shared_ptr.
2600
2601 This overload participates in overload resolution only if T
2602 is an array type with unknown compile-time bound.
2603
2604 NOTE: Given that this function will _NOT_ be instrumenting the allocation
2605 through PFS, observability for particular parts of the system which want to
2606 use it will be lost or in best case inaccurate. Please have a strong reason
2607 to do so.
2608
2609 @param[in] size Size of the array of objects T to allocate.
2610 @return std::shared_ptr holding a pointer to an array of size instances of
2611 T.
2612 */
2613template <typename T,
2614 typename Deleter = detail::Array_deleter<std::remove_extent_t<T>>>
2615std::enable_if_t<detail::is_unbounded_array_v<T>, std::shared_ptr<T>>
2616make_shared(size_t size) {
2617 return std::shared_ptr<T>(
2618 ut::new_arr<std::remove_extent_t<T>>(ut::Count{size}), Deleter{});
2619}
2620
2621/** Dynamically allocates storage for an array of requested size of objects of
2622 type T. Constructs the object of type T with provided Args. Wraps the
2623 pointer to an array of T instances into the std::shared_ptr with custom
2624 deleter which knows how to handle PFS-enabled dynamic memory allocations.
2625 Instruments the memory with given PSI memory key in case PFS memory support
2626 is enabled.
2627
2628 This overload participates in overload resolution only if T
2629 is an array type with unknown compile-time bound.
2630
2631 @param[in] key PSI memory key to be used for PFS memory instrumentation.
2632 @param[in] size Size of the array of objects T to allocate.
2633 @return std::shared_ptr holding a pointer to an array of size instances of
2634 T.
2635 */
2636template <typename T,
2637 typename Deleter = detail::Array_deleter<std::remove_extent_t<T>>>
2638std::enable_if_t<detail::is_unbounded_array_v<T>, std::shared_ptr<T>>
2640 return std::shared_ptr<T>(
2641 ut::new_arr_withkey<std::remove_extent_t<T>>(key, ut::Count{size}),
2642 Deleter{});
2643}
2644
2645/** Dynamically allocates storage for an array of objects of type T. Constructs
2646 the object of type T with provided Args. Wraps the pointer to an array of T
2647 instance into the std::shared_ptr.
2648
2649 This overload participates in overload resolution only if T
2650 is an array type with known compile-time bound.
2651
2652 NOTE: Given that this function will _NOT_ be instrumenting the allocation
2653 through PFS, observability for particular parts of the system which want to
2654 use it will be lost or in best case inaccurate. Please have a strong reason
2655 to do so.
2656
2657 @return std::shared_ptr holding a pointer to an array of size instances of
2658 T.
2659 */
2660template <typename T,
2661 typename Deleter = detail::Array_deleter<std::remove_extent_t<T>>>
2662std::enable_if_t<detail::is_bounded_array_v<T>, std::shared_ptr<T>>
2664 return std::shared_ptr<T>(ut::new_arr<std::remove_extent_t<T>>(
2665 ut::Count{detail::bounded_array_size_v<T>}),
2666 Deleter{});
2667}
2668
2669/** Dynamically allocates storage for an array of objects of type T. Constructs
2670 the object of type T with provided Args. Wraps the pointer to an array of T
2671 instances into the std::shared_ptr with custom deleter which knows how to
2672 handle PFS-enabled dynamic memory allocations. Instruments the memory with
2673 given PSI memory key in case PFS memory support is enabled.
2674
2675 This overload participates in overload resolution only if T
2676 is an array type with known compile-time bound.
2677
2678 @param[in] key PSI memory key to be used for PFS memory instrumentation.
2679 @return std::shared_ptr holding a pointer to an array of size instances of
2680 T.
2681 */
2682template <typename T,
2683 typename Deleter = detail::Array_deleter<std::remove_extent_t<T>>>
2684std::enable_if_t<detail::is_bounded_array_v<T>, std::shared_ptr<T>> make_shared(
2686 return std::shared_ptr<T>(
2687 ut::new_arr_withkey<std::remove_extent_t<T>>(
2688 key, ut::Count{detail::bounded_array_size_v<T>}),
2689 Deleter{});
2690}
2691
2692/** Dynamically allocates storage for an object of type T at address aligned to
2693 the requested alignment. Constructs the object of type T with provided Args.
2694 Wraps the pointer to T instance into the std::shared_ptr.
2695
2696 This overload participates in overload resolution only if T
2697 is not an array type.
2698
2699 NOTE: Given that this function will _NOT_ be instrumenting the allocation
2700 through PFS, observability for particular parts of the system which want to
2701 use it will be lost or in best case inaccurate. Please have a strong reason
2702 to do so.
2703
2704 @param[in] alignment Alignment requirement for storage to be allocated.
2705 @param[in] args Arguments one wishes to pass over to T constructor(s) .
2706 @return std::shared_ptr holding a pointer to instance of T.
2707 */
2708template <typename T, typename Deleter = detail::Aligned_deleter<T>,
2709 typename... Args>
2710std::enable_if_t<!std::is_array<T>::value, std::shared_ptr<T>>
2711make_shared_aligned(size_t alignment, Args &&... args) {
2712 return std::shared_ptr<T>(
2713 ut::aligned_new<T>(alignment, std::forward<Args>(args)...), Deleter{});
2714}
2715
2716/** Dynamically allocates storage for an object of type T at address aligned to
2717 the requested alignment. Constructs the object of type T with provided Args.
2718 Wraps the pointer to T instance into the std::shared_ptr with custom deleter
2719 which knows how to handle PFS-enabled dynamic memory allocations.
2720 Instruments the memory with given PSI memory key in case PFS memory support
2721 is enabled.
2722
2723 This overload participates in overload resolution only if T
2724 is not an array type.
2725
2726 @param[in] key PSI memory key to be used for PFS memory instrumentation.
2727 @param[in] alignment Alignment requirement for storage to be allocated.
2728 @param[in] args Arguments one wishes to pass over to T constructor(s) .
2729 @return std::shared_ptr holding a pointer to instance of T.
2730 */
2731template <typename T, typename Deleter = detail::Aligned_deleter<T>,
2732 typename... Args>
2733std::enable_if_t<!std::is_array<T>::value, std::shared_ptr<T>>
2734make_shared_aligned(PSI_memory_key_t key, size_t alignment, Args &&... args) {
2735 return std::shared_ptr<T>(
2736 ut::aligned_new_withkey<T>(key, alignment, std::forward<Args>(args)...),
2737 Deleter{});
2738}
2739
2740/** Dynamically allocates storage for an array of requested size of objects of
2741 type T at address aligned to the requested alignment. Constructs the object
2742 of type T with provided Args. Wraps the pointer to an array of T instance
2743 into the std::shared_ptr.
2744
2745 This overload participates in overload resolution only if T
2746 is an array type with unknown compile-time bound.
2747
2748 NOTE: Given that this function will _NOT_ be instrumenting the allocation
2749 through PFS, observability for particular parts of the system which want to
2750 use it will be lost or in best case inaccurate. Please have a strong reason
2751 to do so.
2752
2753 @param[in] alignment Alignment requirement for storage to be allocated.
2754 @param[in] size Size of the array of objects T to allocate.
2755 @return std::shared_ptr holding a pointer to an array of size instances of
2756 T.
2757 */
2758template <typename T, typename Deleter = detail::Aligned_array_deleter<
2759 std::remove_extent_t<T>>>
2760std::enable_if_t<detail::is_unbounded_array_v<T>, std::shared_ptr<T>>
2761make_shared_aligned(size_t alignment, size_t size) {
2762 return std::shared_ptr<T>(
2763 ut::aligned_new_arr<std::remove_extent_t<T>>(alignment, ut::Count{size}),
2764 Deleter{});
2765}
2766
2767/** Dynamically allocates storage for an array of requested size of objects of
2768 type T at address aligned to the requested alignment. Constructs the object
2769 of type T with provided Args. Wraps the pointer to an array of T instances
2770 into the std::shared_ptr with custom deleter which knows how to handle
2771 PFS-enabled dynamic memory allocations. Instruments the memory with given
2772 PSI memory key in case PFS memory support is enabled.
2773
2774 This overload participates in overload resolution only if T
2775 is an array type with unknown compile-time bound.
2776
2777 @param[in] key PSI memory key to be used for PFS memory instrumentation.
2778 @param[in] alignment Alignment requirement for storage to be allocated.
2779 @param[in] size Size of the array of objects T to allocate.
2780 @return std::shared_ptr holding a pointer to an array of size instances of
2781 T.
2782 */
2783template <typename T, typename Deleter = detail::Aligned_array_deleter<
2784 std::remove_extent_t<T>>>
2785std::enable_if_t<detail::is_unbounded_array_v<T>, std::shared_ptr<T>>
2786make_shared_aligned(PSI_memory_key_t key, size_t alignment, size_t size) {
2787 return std::shared_ptr<T>(
2788 ut::aligned_new_arr_withkey<std::remove_extent_t<T>>(key, alignment,
2789 ut::Count{size}),
2790 Deleter{});
2791}
2792
2793/** Dynamically allocates storage for an array of objects of type T at address
2794 aligned to the requested alignment. Constructs the object of type T with
2795 provided Args. Wraps the pointer to an array of T instance into the
2796 std::shared_ptr.
2797
2798 This overload participates in overload resolution only if T
2799 is an array type with known compile-time bound.
2800
2801 NOTE: Given that this function will _NOT_ be instrumenting the allocation
2802 through PFS, observability for particular parts of the system which want to
2803 use it will be lost or in best case inaccurate. Please have a strong reason
2804 to do so.
2805
2806 @param[in] alignment Alignment requirement for storage to be allocated.
2807 @return std::shared_ptr holding a pointer to an array of size instances of
2808 T.
2809 */
2810template <typename T, typename Deleter = detail::Aligned_array_deleter<
2811 std::remove_extent_t<T>>>
2812std::enable_if_t<detail::is_bounded_array_v<T>, std::shared_ptr<T>>
2813make_shared_aligned(size_t alignment) {
2814 return std::shared_ptr<T>(
2815 ut::aligned_new_arr<std::remove_extent_t<T>>(
2816 alignment, ut::Count{detail::bounded_array_size_v<T>}),
2817 Deleter{});
2818}
2819
2820/** Dynamically allocates storage for an array of objects of type T at address
2821 aligned to the requested alignment. Constructs the object of type T with
2822 provided Args. Wraps the pointer to an array of T instances into the
2823 std::shared_ptr with custom deleter which knows how to handle PFS-enabled
2824 dynamic memory allocations. Instruments the memory with given PSI memory key
2825 in case PFS memory support is enabled.
2826
2827 This overload participates in overload resolution only if T
2828 is an array type with known compile-time bound.
2829
2830 @param[in] key PSI memory key to be used for PFS memory instrumentation.
2831 @param[in] alignment Alignment requirement for storage to be allocated.
2832 @return std::shared_ptr holding a pointer to an array of size instances of
2833 T.
2834 */
2835template <typename T, typename Deleter = detail::Aligned_array_deleter<
2836 std::remove_extent_t<T>>>
2837std::enable_if_t<detail::is_bounded_array_v<T>, std::shared_ptr<T>>
2839 return std::shared_ptr<T>(
2840 ut::aligned_new_arr_withkey<std::remove_extent_t<T>>(
2841 key, alignment, ut::Count{detail::bounded_array_size_v<T>}),
2842 Deleter{});
2843}
2844
2845/** Specialization of basic_ostringstream which uses ut::allocator. Please note
2846 that it's .str() method returns std::basic_string which is not std::string,
2847 so it has similar API (in particular .c_str()), but you can't assign it to
2848 regular, std::string.
2849 */
2851 std::basic_ostringstream<char, std::char_traits<char>, ut::allocator<char>>;
2852
2853/** Specialization of vector which uses allocator. */
2854template <typename T>
2855using vector = std::vector<T, ut::allocator<T>>;
2856
2857/** Specialization of list which uses ut_allocator. */
2858template <typename T>
2859using list = std::list<T, ut::allocator<T>>;
2860
2861/** Specialization of set which uses ut_allocator. */
2862template <typename Key, typename Compare = std::less<Key>>
2863using set = std::set<Key, Compare, ut::allocator<Key>>;
2864
2865template <typename Key>
2867 std::unordered_set<Key, std::hash<Key>, std::equal_to<Key>,
2869
2870/** Specialization of map which uses ut_allocator. */
2871template <typename Key, typename Value, typename Compare = std::less<Key>>
2872using map =
2873 std::map<Key, Value, Compare, ut::allocator<std::pair<const Key, Value>>>;
2874
2875} // namespace ut
2876
2877#endif /* ut0new_h */
Definition: tls_cipher.cc:36
Lightweight convenience wrapper which manages a dynamically allocated array of over-aligned types.
Definition: ut0new.h:2015
void dealloc()
Invokes destructors of instances of type T, if applicable.
Definition: ut0new.h:2103
void alloc(Count count)
Allocates sufficiently large memory of dynamic storage duration to fit the array of size number of el...
Definition: ut0new.h:2037
T * ptr
Definition: ut0new.h:2016
void alloc_withkey(PSI_memory_key_t key, Count count)
Allocates sufficiently large memory of dynamic storage duration to fit the array of size number of el...
Definition: ut0new.h:2073
void alloc_withkey(PSI_memory_key_t key, Args &&... args)
Allocates sufficiently large memory of dynamic storage duration to fit the array of size number of el...
Definition: ut0new.h:2093
~aligned_array_pointer()
Destructor.
Definition: ut0new.h:2023
void alloc(Args &&... args)
Allocates sufficiently large memory of dynamic storage duration to fit the array of size number of el...
Definition: ut0new.h:2055
Lightweight convenience wrapper which manages dynamically allocated over-aligned type.
Definition: ut0new.h:1921
~aligned_pointer()
Destructor.
Definition: ut0new.h:1929
void alloc(Args &&... args)
Allocates sufficiently large memory of dynamic storage duration to fit the instance of type T at the ...
Definition: ut0new.h:1943
void alloc_withkey(PSI_memory_key_t key, Args &&... args)
Allocates sufficiently large memory of dynamic storage duration to fit the instance of type T at the ...
Definition: ut0new.h:1960
T * ptr
Definition: ut0new.h:1922
void dealloc()
Invokes the destructor of instance of type T, if applicable.
Definition: ut0new.h:1969
Allocator that allows std::* containers to manage their memory through ut::malloc* and ut::free libra...
Definition: ut0new.h:2179
T & reference
Definition: ut0new.h:2183
void deallocate(pointer ptr, size_type n_elements=0)
Releases the memory allocated through ut::allocator<T>::allocate().
Definition: ut0new.h:2275
~allocator()=default
const T * const_pointer
Definition: ut0new.h:2182
ptrdiff_t difference_type
Definition: ut0new.h:2187
allocator< T, Allocator_base > & operator=(const allocator< T, Allocator_base > &)=default
allocator< T, Allocator_base > & operator=(allocator< T, Allocator_base > &&)=default
T value_type
Definition: ut0new.h:2185
allocator(const allocator< U, Allocator_base > &other)
Copy-construct a new instance of allocator with type T by using existing instance of allocator constr...
Definition: ut0new.h:2212
pointer allocate(size_type n_elements, const_pointer hint=nullptr)
Allocates chunk of memory that can hold n_elements objects of type T.
Definition: ut0new.h:2255
size_type max_size() const
Return the maximum number of objects that can be allocated by this allocator.
Definition: ut0new.h:2238
const T & const_reference
Definition: ut0new.h:2184
allocator(allocator< T, Allocator_base > &&)=default
bool operator!=(const ut::allocator< T, Allocator_base > &other) const
Non-equality of allocators instantiated with same types T.
Definition: ut0new.h:2230
allocator(PSI_memory_key key=mem_key_std)
Default constructor.
Definition: ut0new.h:2196
allocator(const allocator< T, Allocator_base > &)=default
size_t size_type
Definition: ut0new.h:2186
bool operator==(const ut::allocator< T, Allocator_base > &) const
Equality of allocators instantiated with same types T.
Definition: ut0new.h:2226
T * pointer
Definition: ut0new.h:2181
Implementation bits and pieces of include/ut0new.h.
unsigned int PSI_memory_key
Instrumented memory key.
Definition: psi_memory_bits.h:48
#define realloc(P, A)
Definition: lexyy.cc:916
#define free(A)
Definition: lexyy.cc:915
A macro that gives FILE without the directory name (e.g.
constexpr bool unlikely(bool expr)
Definition: my_compiler.h:55
static int count
Definition: myisam_ftdump.cc:42
Instrumentation helpers for memory allocation.
std::atomic< Type > N
Definition: ut0counter.h:224
Definition: ut0tuple.h:56
Definition: os0file.h:85
Definition: authentication.cc:35
Alignment
Enum class describing alignment-requirements.
Definition: lock_free_type.h:74
constexpr bool is_unbounded_array_v< T[]>
Definition: ut0new.h:2284
typename select_alloc_impl< Pfs_memory_instrumentation_on >::type select_alloc_impl_t
Just a small helper type which saves us some keystrokes.
Definition: aligned_alloc.h:689
constexpr size_t bounded_array_size_v< T[N]>
Definition: ut0new.h:2294
constexpr bool is_bounded_array_v
Definition: ut0new.h:2287
constexpr bool is_unbounded_array_v
Definition: ut0new.h:2282
typename select_large_page_alloc_impl< Pfs_memory_instrumentation_on >::type select_large_page_alloc_impl_t
Just a small helper type which saves us some keystrokes.
Definition: large_page_alloc.h:366
typename select_page_alloc_impl< Pfs_memory_instrumentation_on >::type select_page_alloc_impl_t
Just a small helper type which saves us some keystrokes.
Definition: page_alloc.h:427
typename select_malloc_impl< Pfs_memory_instrumentation_on, Array_specialization >::type select_malloc_impl_t
Just a small helper type which saves us some keystrokes.
Definition: alloc.h:428
constexpr size_t bounded_array_size_v
Definition: ut0new.h:2292
constexpr bool is_bounded_array_v< T[N]>
Definition: ut0new.h:2289
This file contains a set of libraries providing overloads for regular dynamic allocation routines whi...
Definition: aligned_alloc.h:47
void * aligned_zalloc_withkey(PSI_memory_key_t key, std::size_t size, std::size_t alignment) noexcept
Dynamically allocates zero-initialized storage of given size and at the address aligned to the reques...
Definition: ut0new.h:1524
T * new_arr_withkey(PSI_memory_key_t key, Args &&... args)
Dynamically allocates storage for an array of T's.
Definition: ut0new.h:872
size_t pfs_overhead() noexcept
Returns number of bytes that ut::malloc_*, ut::zalloc_*, ut::realloc_* and ut::new_* variants will be...
Definition: ut0new.h:1123
T * aligned_new_withkey(PSI_memory_key_t key, std::size_t alignment, Args &&... args)
Dynamically allocates storage for an object of type T at address aligned to the requested alignment.
Definition: ut0new.h:1592
void * zalloc_withkey(PSI_memory_key_t key, std::size_t size) noexcept
Dynamically allocates zero-initialized storage of given size.
Definition: ut0new.h:630
void * malloc_withkey(PSI_memory_key_t key, std::size_t size) noexcept
Dynamically allocates storage of given size.
Definition: ut0new.h:594
std::enable_if_t<!std::is_array< T >::value, std::unique_ptr< T, Deleter > > make_unique(Args &&... args)
Dynamically allocates storage for an object of type T.
Definition: ut0new.h:2335
T * aligned_new_arr(std::size_t alignment, Args &&... args)
Dynamically allocates storage for an array of T's at address aligned to the requested alignment.
Definition: ut0new.h:1836
void * aligned_zalloc(std::size_t size, std::size_t alignment) noexcept
Dynamically allocates zero-initialized storage of given size and at the address aligned to the reques...
Definition: ut0new.h:1547
void * malloc_large_page(std::size_t size) noexcept
Dynamically allocates memory backed up by large (huge) pages.
Definition: ut0new.h:1269
std::unordered_set< Key, std::hash< Key >, std::equal_to< Key >, ut::allocator< Key > > unordered_set
Definition: ut0new.h:2868
std::basic_ostringstream< char, std::char_traits< char >, ut::allocator< char > > ostringstream
Specialization of basic_ostringstream which uses ut::allocator.
Definition: ut0new.h:2851
T * new_(Args &&... args)
Dynamically allocates storage for an object of type T.
Definition: ut0new.h:792
allocation_low_level_info page_low_level_info(void *ptr) noexcept
Retrieves the pointer and size of the allocation provided by the OS.
Definition: ut0new.h:1207
std::vector< T, ut::allocator< T > > vector
Specialization of vector which uses allocator.
Definition: ut0new.h:2855
void aligned_delete_arr(T *ptr) noexcept
Releases storage which has been dynamically allocated through any of the aligned_new_arr_*() variants...
Definition: ut0new.h:1893
void delete_(T *ptr) noexcept
Releases storage which has been dynamically allocated through any of the ut::new*() variants.
Definition: ut0new.h:807
std::set< Key, Compare, ut::allocator< Key > > set
Specialization of set which uses ut_allocator.
Definition: ut0new.h:2863
void * malloc_page(std::size_t size) noexcept
Dynamically allocates system page-aligned storage of given size.
Definition: ut0new.h:1173
void * realloc_withkey(PSI_memory_key_t key, void *ptr, std::size_t size) noexcept
Upsizes or downsizes already dynamically allocated storage to the new size.
Definition: ut0new.h:673
T * new_withkey(PSI_memory_key_t key, Args &&... args)
Dynamically allocates storage for an object of type T.
Definition: ut0new.h:749
void delete_arr(T *ptr) noexcept
Releases storage which has been dynamically allocated through any of the ut::new_arr*() variants.
Definition: ut0new.h:1107
allocation_low_level_info large_page_low_level_info(void *ptr) noexcept
Retrieves the pointer and size of the allocation provided by the OS.
Definition: ut0new.h:1305
std::enable_if_t<!std::is_array< T >::value, std::unique_ptr< T, Deleter > > make_unique_aligned(size_t alignment, Args &&... args)
Dynamically allocates storage for an object of type T at address aligned to the requested alignment.
Definition: ut0new.h:2457
T * new_arr(Args &&... args)
Dynamically allocates storage for an array of T's.
Definition: ut0new.h:957
bool free_page(void *ptr) noexcept
Releases storage which has been dynamically allocated through any of the ut::malloc_page*() variants.
Definition: ut0new.h:1223
void * malloc_large_page_withkey(PSI_memory_key_t key, std::size_t size) noexcept
Dynamically allocates memory backed up by large (huge) pages.
Definition: ut0new.h:1245
size_t large_page_allocation_size(void *ptr) noexcept
Retrieves the total amount of bytes that are available for application code to use.
Definition: ut0new.h:1291
std::enable_if_t<!std::is_array< T >::value, std::shared_ptr< T > > make_shared_aligned(size_t alignment, Args &&... args)
Dynamically allocates storage for an object of type T at address aligned to the requested alignment.
Definition: ut0new.h:2711
std::map< Key, Value, Compare, ut::allocator< std::pair< const Key, Value > > > map
Specialization of map which uses ut_allocator.
Definition: ut0new.h:2873
void * malloc(std::size_t size) noexcept
Dynamically allocates storage of given size.
Definition: ut0new.h:615
std::conditional_t< !std::is_array< T >::value, std::unique_ptr< T, detail::Deleter< T > >, std::conditional_t< detail::is_unbounded_array_v< T >, std::unique_ptr< T, detail::Array_deleter< std::remove_extent_t< T > > >, void > > unique_ptr
The following is a common type that is returned by all the ut::make_unique (non-aligned) specializati...
Definition: ut0new.h:2436
T * aligned_new(std::size_t alignment, Args &&... args)
Dynamically allocates storage for an object of type T at address aligned to the requested alignment.
Definition: ut0new.h:1633
std::list< T, ut::allocator< T > > list
Specialization of list which uses ut_allocator.
Definition: ut0new.h:2859
void free(void *ptr) noexcept
Releases storage which has been dynamically allocated through any of the ut::malloc*(),...
Definition: ut0new.h:715
PSI_memory_key_t make_psi_memory_key(PSI_memory_key key)
Convenience helper function to create type-safe representation of PSI_memory_key.
Definition: ut0new.h:188
void * aligned_alloc_withkey(PSI_memory_key_t key, std::size_t size, std::size_t alignment) noexcept
Dynamically allocates storage of given size and at the address aligned to the requested alignment.
Definition: ut0new.h:1482
constexpr bool WITH_PFS_MEMORY
Definition: ut0new.h:578
void aligned_delete(T *ptr) noexcept
Releases storage which has been dynamically allocated through any of the aligned_new_*() variants.
Definition: ut0new.h:1648
void * malloc_page_withkey(PSI_memory_key_t key, std::size_t size) noexcept
Dynamically allocates system page-aligned storage of given size.
Definition: ut0new.h:1146
void * realloc(void *ptr, std::size_t size) noexcept
Upsizes or downsizes already dynamically allocated storage to the new size.
Definition: ut0new.h:701
std::enable_if_t<!std::is_array< T >::value, std::shared_ptr< T > > make_shared(Args &&... args)
Dynamically allocates storage for an object of type T.
Definition: ut0new.h:2571
size_t page_allocation_size(void *ptr) noexcept
Retrieves the total amount of bytes that are available for application code to use.
Definition: ut0new.h:1193
bool free_large_page(void *ptr) noexcept
Releases storage which has been dynamically allocated through any of the ut::malloc_large_page*() var...
Definition: ut0new.h:1322
T * aligned_new_arr_withkey(PSI_memory_key_t key, std::size_t alignment, Args &&... args)
Dynamically allocates storage for an array of T's at address aligned to the requested alignment.
Definition: ut0new.h:1713
void aligned_free(void *ptr) noexcept
Releases storage which has been dynamically allocated through any of the aligned_alloc_*() or aligned...
Definition: ut0new.h:1561
void * aligned_alloc(std::size_t size, std::size_t alignment) noexcept
Dynamically allocates storage of given size and at the address aligned to the requested alignment.
Definition: ut0new.h:1505
void * zalloc(std::size_t size) noexcept
Dynamically allocates zero-initialized storage of given size.
Definition: ut0new.h:651
The interface to the operating system process control primitives.
bool os_use_large_pages
Whether to use large pages in the buffer pool.
Definition: os0proc.cc:50
The interface to the operating system process and thread control primitives.
Performance schema instrumentation interface.
Performance schema instrumentation interface.
required string key
Definition: replication_asynchronous_connection_failover.proto:59
static MEM_ROOT mem
Definition: sql_servers.cc:98
Memory instrument information.
Definition: psi_memory_bits.h:57
Definition: ut0new.h:555
static constexpr int value
Definition: ut0new.h:556
Light-weight and type-safe wrapper which serves a purpose of being able to select proper ut::new_arr*...
Definition: ut0new.h:979
Count(size_t count)
Definition: ut0new.h:980
size_t m_count
Definition: ut0new.h:982
size_t operator()() const
Definition: ut0new.h:981
Light-weight and type-safe wrapper around the PSI_memory_key that eliminates the possibility of intro...
Definition: ut0new.h:176
PSI_memory_key operator()() const
Definition: ut0new.h:178
PSI_memory_key m_key
Definition: ut0new.h:179
PSI_memory_key_t(PSI_memory_key key)
Definition: ut0new.h:177
Can be used to extract pointer and size of the allocation provided by the OS.
Definition: ut0new.h:138
void * base_ptr
A pointer returned by the OS allocator.
Definition: ut0new.h:140
size_t allocation_size
The size of allocation that OS performed.
Definition: ut0new.h:142
Definition: ut0new.h:2221
Small wrapper which utilizes SFINAE to dispatch the call to appropriate aligned allocator implementat...
Definition: aligned_alloc.h:695
Definition: ut0new.h:2312
void operator()(T *ptr)
Definition: ut0new.h:2313
Definition: ut0new.h:2307
void operator()(T *ptr)
Definition: ut0new.h:2308
Small wrapper which utilizes SFINAE to dispatch the call to appropriate allocator implementation.
Definition: alloc.h:434
Definition: ut0new.h:2302
void operator()(T *ptr)
Definition: ut0new.h:2303
Definition: ut0new.h:2297
void operator()(T *ptr)
Definition: ut0new.h:2298
Small wrapper which utilizes SFINAE to dispatch the call to appropriate aligned allocator implementat...
Definition: large_page_alloc.h:372
Small wrapper which utilizes SFINAE to dispatch the call to appropriate aligned allocator implementat...
Definition: page_alloc.h:433
Definition: ut0new.h:2129
void * allocate_impl(size_t n_bytes)
Definition: ut0new.h:2138
const PSI_memory_key m_key
Definition: ut0new.h:2143
PSI_memory_key get_mem_key() const
Definition: ut0new.h:2136
allocator_base_pfs(PSI_memory_key key)
Definition: ut0new.h:2130
allocator_base_pfs(const allocator_base_pfs< U > &other)
Definition: ut0new.h:2133
Definition: ut0new.h:2119
void * allocate_impl(size_t n_bytes)
Definition: ut0new.h:2125
allocator_base(const allocator_base< U > &other)
Definition: ut0new.h:2123
allocator_base(PSI_memory_key)
Definition: ut0new.h:2120
Definition: ut0new.h:1329
Version control for database, common definitions, and include files.
#define UT_ARR_SIZE(a)
Definition: univ.i:525
Utilities for byte operations.
Utilities related to CPU cache.
Debug utilities for Innobase.
#define ut_ad(EXPR)
Debug assertion.
Definition: ut0dbg.h:68
#define ut_a(EXPR)
Abort execution if EXPR does not evaluate to nonzero.
Definition: ut0dbg.h:56
PSI_memory_key mem_key_ddl
Definition: ut0new.cc:59
void ut_new_boot_safe()
Setup the internal objects needed for ut::*_withkey() to operate.
Definition: ut0new.cc:136
PSI_memory_key mem_key_redo_log_archive_queue_element
PSI_memory_key mem_key_dict_stats_index_map_t
Definition: ut0new.cc:53
PSI_memory_key mem_key_buf_stat_per_index_t
Definition: ut0new.cc:49
PSI_memory_key mem_key_other
Definition: ut0new.cc:56
PSI_memory_info pfs_info_auto[n_auto]
PSI_memory_key auto_event_keys[n_auto]
constexpr int ut_new_get_key_by_base_file(const char *file, size_t len)
Retrieve a memory key (registered with PFS), given the file name of the caller.
Definition: ut0new.h:534
PSI_memory_key mem_key_fil_space_t
Definition: ut0new.cc:55
constexpr bool ut_string_begins_with(const char *a, const char *b, size_t b_len)
gcc 5 fails to evalutate costexprs at compile time.
Definition: ut0new.h:508
constexpr int ut_new_get_key_by_file(const char *file)
Retrieve a memory key (registered with PFS), given the file name of the caller.
Definition: ut0new.h:547
PSI_memory_key mem_key_ahi
Keys for registering allocations with performance schema.
Definition: ut0new.cc:46
void ut_new_boot()
Setup the internal objects needed for ut::*_withkey() to operate.
Definition: ut0new.cc:117
PSI_memory_key mem_key_partitioning
Definition: ut0new.cc:57
PSI_memory_key mem_key_archive
Definition: ut0new.cc:47
static constexpr const char * auto_event_names[]
List of filenames that allocate memory and are instrumented via PFS.
Definition: ut0new.h:244
PSI_memory_key mem_key_std
Definition: ut0new.cc:60
const size_t alloc_max_retries
Maximum number of retries to allocate memory.
Definition: ut0new.cc:42
PSI_memory_key mem_key_row_log_buf
Definition: ut0new.cc:58
PSI_memory_key mem_key_trx_sys_t_rw_trx_ids
Definition: ut0new.cc:61
static constexpr size_t n_auto
Definition: ut0new.h:445
PSI_memory_key mem_key_buf_buf_pool
Definition: ut0new.cc:48
constexpr size_t ut_len_without_extension(const char *file)
Find the length of the filename without its file extension.
Definition: ut0new.h:521
PSI_memory_key mem_key_undo_spaces
Definition: ut0new.cc:62
PSI_memory_key mem_key_ut_lock_free_hash_t
Definition: ut0new.cc:63
PSI_memory_key mem_key_clone
Memory key for clone.
Definition: ut0new.cc:51
PSI_memory_key mem_key_dict_stats_n_diff_on_level
Definition: ut0new.cc:54
PSI_memory_key mem_key_dict_stats_bg_recalc_pool_t
Definition: ut0new.cc:52
Various utilities.
#define PSI_NOT_INSTRUMENTED
Definition: validate_password_imp.cc:39