WL#643: adaptive block size (malloc optimization)

Affects: Server-7.1   —   Status: Assigned   —   Priority: Medium

>> After thinking about this for a couple of hours, I belive that in the 
>> future a good way would be to make the block sizes more adaptive (as 
>> the malloc would be 'perfect' if we would just know how much memory we 
>> would really need). A simple way to do this would be something like 
>> the following: 
>> - Remember for the mem_root how much memory was allocated this time. 
>> - For each free_root() adjust the block_size as follows: 
>> Adjust 'used_memory' between block_size/2 and block_size*2 
>> mean_memory_used= (mean_memory_used + used_memory)/2 
>> block_size= ALIGN_SIZE(mean_memory, 8192); 
>> The above would change block_size to slowly increase / decrease 
>> according to how the current connection allocates data.  It would 
>> also be quite inexpensive to do. 
>> What do you think about the above ? 
Sanja> Good idea! 
Sanja> But I think it is better to count how many memory used, but not 
It would be quite inexpensive to have both and then do some statistics 
to get the best usage. 
Sanja> Also very important for us size of preallocated block and it is 
Sanja> better to adjust it amount (IMHO). Thus memory root can provide interface 
Sanja> for adjustment, but external procedure will edjust preallocated memory 
Sanja> N calls for example. I.e. use it with calculating block_size, to avoid 
Sanja> frequently allocated second block. Even better to make preallocated block 
Sanja> adjustments if number of allocation second block will be high then 50% 
Sanja> (I mean memory pool of thd here which used to perform command) 
Sounds good. 
So please put back malloc optimization to your todo :) 
Estimation 7hrs (with testing)