WL#5302: MyISAM mmap refactoring

Affects: Benchmarks-3.0   —   Status: Un-Assigned

Currently there are two MyISAM mmap implementations (for compressed and
uncompressed tables). They are wide spread across (and even outside of) MyISAM
code base.

The primary goal of this task is to make MyISAM mmap maintenance easier: merge
two implementations into one, comment the code and concentrate the code in a
single place.
User visible changes
--------------------
We will declare 'myisam_use_mmap' obsolete, still having it around for some
time. When 'myisam_use_mmap' is ON 'myisam_mmap_size' gets maximum value, when
OFF 'myisam_mmap_size' is 0.

'myisam_mmap_size' minimum allowed and default values should be changed to 0.
Will affect both compressed and uncompressed tables (currently only compressed
tables are affected).

New status variable 'myisam_mmap_used' should be added reflecting amount of
address space used for mmap.

MyISAM internal changes
-----------------------
madvise(SEQUENTIAL) will be removed from this implementation. It didn't work
properly anyway: was enabled only for compressed tables if myisam_use_mmap is
off, for the first scan after a table was opened for the first time.

Remap with concurrent inserts has no guaranteed number of inserts threshold any
more. Table may be remapped at any time after reaching threshold, when it has no
concurrent users. This may increase non-mmaped I/O.

SE API visible changed
----------------------
HA_OPEN_MMAP flag goes away - was used internally by MyISAM.

HA_EXTRA_MMAP flag is renamed to HA_EXTRA_UNUSED0. Server doesn't use this extra
function any more.

Implementation
--------------
We will implement all functionality in a new file, mi_mmap.c. Declarations are
to be put together into storage/myisam/myisamdef.h. If MMAP is not available,
hooks are to be defined as no-op.

my_bool myisam_use_mmap;
  Obsolete option. Moved to mi_mmap.c/myisamdef.h.

ulonglong myisam_mmap_size;
  Maximum allowed mapped address space. Moved to mi_mmap.c/myisamdef.h.

ulonglong myisam_mmap_used;
  Sum of all mappings lengths. Moved to mi_mmap_c/myisamdef.h.

static mysql_mutex_t myisam_mmap_used_lock;
  Mutex for myisam_mmap_used. Moved to mi_mmap.c.

my_off_t MYISAM_SHARE::mmapped_length;
  Length of memory mapping. Stays as is.

uint MYISAM_SHARE::nonmmapped_inserts;
  Number of out of memory mapping inserts. Stays as is.

uchar *MYISAM_SHARE::file_map;
  Memory mapping. Stays as is.

#define MAX_NONMMAPPED_INSERTS 1000
  Threshold of nonmmapped inserts. When reached, try to remap.

#define MEMMAP_EXTRA_MARGIN 7
  Moved to myisamdef.h.

void _mi_mmap_init_hook(void);
  Initialize mutex for 'myisam_mmap_used' variable. Must be called from
  myisam_init().

void _mi_mmap_deinit_hook(void);
  Destroy mutex for 'myisam_mmap_used' variable. Must be calledx from
  mi_panic(), if flag is HA_PANIC_CLOSE after all tables are closed.

void _mi_mmap_open_hook(MI_INFO *info);
  Initialize memory mapping, setup mmapped I/O handlers. Merge implementations
  of mi_dynmap_file() and _mi_memmap_file(). Must be called from mi_open()
  when MYISAM_SHARE is created and test_if_locked & HA_OPEN_TMP_TABLE is false.

void _mi_mmap_close_hook(MI_INFO *info);
  Destroy memory mapping. Must be called from mi_close() when MYISAM_SHARE is
  destroyed.

void _mi_mmap_lock_hook(MI_INFO *info);
  Check if remap is needed and remap if there're no active users. Must be called
  from mi_lock_database(F_RDLCK/F_WRLCK). May be called only for uncompressed
  tables.

void _mi_mmap_truncate_hook(MI_INFO *info);
  Destroy memory mapping before truncation. Setup I/O handlers for non-mmapped
  I/O. Can only be called with uncompressed tables.

static size_t _mi_mmap_pread(MI_INFO *info, uchar *Buffer,
                             size_t Count, my_off_t offset, myf MyFlags);
static size_t _mi_mmap_pwrite(MI_INFO *info, uchar *Buffer,
                              size_t Count, my_off_t offset, myf MyFlags);
  Memory mapped I/O handlers.
Analysis basing on mysql-next-mr (2010-03-19).

include/my_base.h
-----------------
Remove HA_OPEN_MMAP. Document HA_EXTRA_MMAP as obsolete.

include/myisam.h
----------------
Move myisam_mmap_size and myisam_mmap_used declarations, MEMMAP_EXTRA_MARGIN
definition, THR_LOCK_myisam_mmap declaration into MyISAM private headers.

mysys/my_init.c, mysys/mysys_priv.h, mysys/my_thr_init.c
--------------------------------------------------------
Move THR_LOCK_myisam_mmap to MyISAM.

sql/mysqld.cc, sql/mysql_priv.h
-------------------------------
Move myisam_use_mmap to MyISAM (should already be available as MyISAM plugin
option).

sql/records.cc
--------------
Remove extra(HA_EXTRA_MMAP). Mapping must be setup on open.

storage/myisam/ha_myisam.cc
---------------------------
Set minimum value for myisam_mmap_size to 0.
Remove mmap related code from ha_myisam::open().
Remove mmap related code from ha_myisam::extra().
Add myisam_mmap_used as status variable.

storage/myisam/myisamdef.h
--------------------------
Remove mmap_lock, mi_key_rwlock_MYISAM_SHARE_mmap_lock declarations.
Remove mi_mmap_pread(), mi_mmap_pwrite() declarations.
Remove MEMMAP_USED definition.