WL#643: adaptive block size (malloc optimization)

>> After thinking about this for a couple of hours, I belive that in the 
>> future a good way would be to make the block sizes more adaptive (as 
>> the malloc would be 'perfect' if we would just know how much memory we 
>> would really need). A simple way to do this would be something like 
>> the following: 
>>  
>> - Remember for the mem_root how much memory was allocated this time. 
>> - For each free_root() adjust the block_size as follows: 
>>  
>> Adjust 'used_memory' between block_size/2 and block_size*2 
>>  
>> mean_memory_used= (mean_memory_used + used_memory)/2 
>>  
>> block_size= ALIGN_SIZE(mean_memory, 8192); 
>>  
>> The above would change block_size to slowly increase / decrease 
>> according to how the current connection allocates data.  It would 
>> also be quite inexpensive to do. 
>>  
>> What do you think about the above ? 
Sanja> Good idea! 
Sanja> But I think it is better to count how many memory used, but not 
+allocated. 
 
It would be quite inexpensive to have both and then do some statistics 
to get the best usage. 
 
Sanja> Also very important for us size of preallocated block and it is 
Sanja> better to adjust it amount (IMHO). Thus memory root can provide interface 
Sanja> for adjustment, but external procedure will edjust preallocated memory 
+every 
Sanja> N calls for example. I.e. use it with calculating block_size, to avoid 
+very 
Sanja> frequently allocated second block. Even better to make preallocated block 
Sanja> adjustments if number of allocation second block will be high then 50% 
+cases. 
Sanja> (I mean memory pool of thd here which used to perform command) 
 
Sounds good. 
 
So please put back malloc optimization to your todo :) 
 
Estimation 7hrs (with testing)