WL#14125: TempTable: Implement an upper bound for MMAP-backed allocations

Affects: Server-8.0   —   Status: Complete

This worklog is intended to provide a mechanism which will allow for more finer grained control over the TempTable memory-consumption. Currently, such mechanism exists only in part, where all heap-backed allocations are monitored and put under the control through the means of temptable_max_ram variable. This is missing for MMAP-backed allocations which are currently open-ended. In other words, there is no upper bound of memory that queries routed through TempTable may consume and this may result in virtual address wastage and other unwanted direct or indirect consequences.

Goal of this worklog is to implement monitoring and controlling mechanism for MMAP-backed allocations which will not allow to consume memory beyond certain threshold. Behavior should be configurable through the threshold which will be exposed through the means of new variable temptable_max_mmap. Setting this threshold to 0 will effectively disable all MMAP-backed allocations which subsumes deprecated temptable_use_mmap variable.

Functional requirements

FR1: New sys-var is introduced: temptable_max_mmap.

FR2: temptable_max_mmap defines upper limit of how much MMAP resources will TempTable consume at most.

FR3: temptable_max_mmap allowed range of values is [0, ULLONG_MAX].

FR4: temptable_max_mmap values are defined in byte units.

FR5: temptable_max_mmap default value is 1073741824 bytes (1GB).

FR6: temptable_max_mmap must co-exist with temptable_use_mmap (which will be deprecated in favor of former).

FR7: If temptable_use_mmap is set to ON, TempTable will consume no more than temptable_max_mmap bytes when allocating memory from MMAP.

FR8: If temptable_use_mmap is set to OFF, TempTable will consider this use-case as if temptable_max_mmap was set to zero (practically disables TempTable from using MMAP).

FR9: If temptable_use_mmap is set to OFF, and temptable_max_mmap is set to something different than zero, TempTable will still consider this use-case as if temptable_max_mmap was set to zero.

FR10: If temptable_use_mmap is set to ON, and temptable_max_mmap threshold is reached, it is implied that temptable_max_ram threshold is also reached.

FR11: If temptable_use_mmap is set to ON, and temptable_max_mmap threshold is reached, optimizer shall fallback to InnoDB tables.

FR12: If temptable_use_mmap is set to OFF, and temptable_max_ram threshold is reached, optimizer shall fallback to InnoDB tables.

FR13: Increment of Created_tmp_disk_tables sys-var can be used to validate if optimizer falls back (or spills over) to InnoDB tables.

FR14: Performance-schema memory/temptable/physical_ram event can be used to monitor temptable_max_ram watermarks.

FR15: Performance-schema memory/temptable/physical_disk event can be used to monitor temptable_max_mmap watermarks.

Non-functional requirements

NFR1: Performance shall not be impacted for use-cases which do not reach the temptable_max_mmap threshold.

NFR2: Performance may be impacted for use-cases which do reach the temptable_max_mmap threshold (as queries will start spilling over to temporary InnoDB tables).

NFR3: TempTable does not have control over performance for NFR2 use-cases.

NFR4: Clients will want to adjust the {temptable_max_ram, temptable_max_mmap} thresholds to meet their needs.

temptable_max_ram is reached temptable_max_mmap is reached temptable_use_mmap TempTable RAM-based table will be used TempTable MMAP-based table will be used InnoDB table will be used
N N N Y N N
N N Y Y N N
N Y N Y N N
N Y Y Y N N
Y N N N N Y
Y N Y N Y N
Y Y N N N Y
Y Y Y N N Y

temptable::MemoryMonitor shall be extended so it includes means to monitor and retrieve MMAP resources consumption. E.g. it should provide following interfaces:

struct MemoryMonitor {
  ...
  struct MMAP {
    static size_t increase(size_t bytes) { ... }
    static size_t decrease(size_t bytes) { ... }
    static size_t threshold() { return temptable_max_mmap; }
    static size_t consumption() { return ... }
  };
};

These interfaces should then be sufficient for temptable::Allocator to use whenever it allocates and deallocates block of memory from MMAP.

All possible combinations and edge-cases of {temptable_max_ram, temptable_max_mmap, temptable_use_mmap} shall be covered with unit-testing.

There shall be at least one MTR use-case proving that spill-over to InnoDB tables is handled correctly from optimizer PoV. By utilizing information tracked by Created_tmp_disk_tables, memory/temptable/physical_ram and memory/temptable/physical_disk one can validate the correctness from MTR.