There will be one or more new tables in PERFORMANCE_SCHEMA showing what MySQL is doing with memory.
Marc Alff, 2011-12-15: The following spec below is just a collection of historical notes. The spec needs to be written. ============================================================================== MEMORY table ------------ There is one row for each memory pool, which may mean there is a row for global, a lot of rows for per-thread, and separate rows for different pools. All counts are in bytes. MEMORY_TYPE VARCHAR MySQL or its storage engines use different allocation methods, and they might change depending on version or on compilation. Examples: "TCMalloc", "safemalloc". MEMORY_POOL VARCHAR A descriptor. Usually NULL. In an ideal world, we'd know if the memory was specifically for "buffers for storage engine X", etc. MEMORY_THREAD INT If it's global (process) memory, this is NULL. If it's per-thread memory, this is thread number, which may or may not be relatable to PROCESSLIST. MEMORY_MAX INT Maximum. If there is no such thing as a maximum, this is NULL. MEMORY_USE INT How much is in use currently. MEMORY_USE_MAX INT How much was in use at peak. High water mark. MEMORY_USE_AVG INT How much was in average usse, according to snapshots. MEMORY_BLOCK_SIZE INT And whatever else you see in MEM_ROOT. Getting the information ----------------------- It depends on how we're allocating. In include/my_alloc.h there is a structure MEM_ROOT, which has blocks free, used, and so on. A patch suggestion (BUG#43237) proposes adding fields from all MEM_ROOT instances into a global counter and using SHOW STATUS to display. Sometimes the malloc/free library has a way to look at details, for example TCMalloc's heap profiler http://goog-perftools.sourceforge.net/doc/heap_profiler.html If you just use GNU malloc(), then GNU mallinfo() will tell you http://www.gnu.org/software/libtool/manual/libc/Statistics-of-Malloc.html In WL#682 "Memory consumption estimation for x86/32 Linux", there is a suggestion about getrusage(). Instrumenting malloc/free calls ------------------------------- Primarily this worklog task is about the state of memory. We must also consider whether to monitor actions that affect memory, which for convention we refer to as "malloc" and "free" calls. WL#2360 mentions "memory malloc/free" as a possible class for events in performance_schema. The main thing we'd be monitoring here is elapsed time, but of course the amount would be in too. In the PERFORMANCE_SCHEMA.EVENTS_WAITS_CURRENT table, we'd have: EVENT_NAME "Wait/Memory/Allocator/safemalloc(etc.)/Server" SOURCE as usual TIMER_START as usual TIMER_END as usual TIMER_WAIT as usual SPINS perhaps the amount that's being allocated OBJECT_... NULL, except OBJECT_INSTANCE_ID = memory location NESTING_EVENT_ID as usual And its aggregations per thread etc. are, as usual, as usual. One possible objection to this table is that there might be no "wait" -- no operating system call, no significant time consumption. We'll have to do some tests before deciding we really want this. It will be interesting to see what "total" bytes allocated/freed is according to a summary table of EVENTS_WAITS_CURRENT, but you can't depend on such totals. Some memory might be released when a thread dies, or when a memory pool is reorganized. Mikiya Okuno wrote a feature request (BUG#55171 How much sort_buffer_size are actually used?). If we showed memory events, he could take samples. We'd have to distinguish "allocation within sort buffer" from other types. Instrumenting memset/memcpy/strcpy etc. --------------------------------------- There is a possible analogy with the specification for WL#4678 "PERFORMANCE_SCHEMA Instrumenting File IO". For files, we instrument "create file" + "read/write file". So for memory, we should instrument "create memory" i.e. malloc + "read/write memory" i.e. memset, memcpy, strcpy, and other functions that read/write memory. Naturally we would only want to instrument a very few such calls, when the implementor believes that the relative overhead is slow because there are many bytes, or perhaps in a special version with a compile option that by default is OFF (implementor-defined). For the name, a possible example is: "wait/memory/unmapped-global/maria/page-buffer" In fact the decision about the part of a name that follows "maria" would be entirely the business of Maria programmers. But for the parts of the name that precede "maria", we want to have a public worklog description like this one, because we don't want other storage engines to implement essentially the same thing with different names. By using worklog as a central registry, we ensure that a query for "all memory operations" will find memset done by all storage engines together. List of columns and data types: THREAD_ID as for mutexes EVENT_ID as for mutexes EVENT_NAME see above SOURCE as for mutexes TIMER_START as for mutexes TIMER_END as for mutexes TIMER_WAIT as for mutexes SPINS NULL OBJECT_SCHEMA implementor-defined, probably NULL OBJECT_NAME implementor-defined, probably NULL OBJECT_TYPE implementor-defined, probably NULL OBJECT_INSTANCE_BEGIN memory address (if it's memcpy, use target address) NESTING_EVENT_ID as for mutexes Original worklog description ---------------------------- The original description for this worklog task was: " Due to a request from Brian Aker ... SHOW [FULL] PROCESSLIST should have a new field which indicates the estimated amount of memory that the connection is using. There should be an equivalent column in INFORMATION_SCHEMA.PROCESSLIST. Add a new column to SHOW FULL PROCESSLIST Add a new column to INFORMATION_SCHEMA.PROCESSLIST - For SHOW, the new column is Memory_Used - For INFORMATION_SCHEMA, the new column is MEMORY_USED - Datatype for each column is INT - The column will display the estimated amount of memory used by the connection - The Memory_Used column should be displayed only with SHOW FULL PROCESSLIST, and not with SHOW PROCESSLIST " The current plan is very different. No use of SHOW, no use of INFORMATION_SCHEMA, no connection to PROCESSLIST. But there's no hurry to change the worklog task's title. References ---------- WL#682 "Memory consumption estimation for x86/32 Linux" WL#3675 "Falcon Diagnostics" BUG#41096 Agent needs DTrace probes for all dynamically allocated structures BUG#43237 "Add Bytes_allocated to SHOW STATUS for #bytes currently allocated in MEM_ROOT" (feature request) dev-private thread "Memory allocation debugging patch" [mysql intranet]/secure/mailarchive/mail.php?folder=4&mail=15631 dev-private thread "Re: Memory consumption" [mysql intranet]/secure/mailarchive/mail.php?folder=4&mail=8695