WL#5302: MyISAM mmap refactoring
Affects: Benchmarks-3.0
—
Status: Un-Assigned
Currently there are two MyISAM mmap implementations (for compressed and uncompressed tables). They are wide spread across (and even outside of) MyISAM code base. The primary goal of this task is to make MyISAM mmap maintenance easier: merge two implementations into one, comment the code and concentrate the code in a single place.
User visible changes -------------------- We will declare 'myisam_use_mmap' obsolete, still having it around for some time. When 'myisam_use_mmap' is ON 'myisam_mmap_size' gets maximum value, when OFF 'myisam_mmap_size' is 0. 'myisam_mmap_size' minimum allowed and default values should be changed to 0. Will affect both compressed and uncompressed tables (currently only compressed tables are affected). New status variable 'myisam_mmap_used' should be added reflecting amount of address space used for mmap. MyISAM internal changes ----------------------- madvise(SEQUENTIAL) will be removed from this implementation. It didn't work properly anyway: was enabled only for compressed tables if myisam_use_mmap is off, for the first scan after a table was opened for the first time. Remap with concurrent inserts has no guaranteed number of inserts threshold any more. Table may be remapped at any time after reaching threshold, when it has no concurrent users. This may increase non-mmaped I/O. SE API visible changed ---------------------- HA_OPEN_MMAP flag goes away - was used internally by MyISAM. HA_EXTRA_MMAP flag is renamed to HA_EXTRA_UNUSED0. Server doesn't use this extra function any more. Implementation -------------- We will implement all functionality in a new file, mi_mmap.c. Declarations are to be put together into storage/myisam/myisamdef.h. If MMAP is not available, hooks are to be defined as no-op. my_bool myisam_use_mmap; Obsolete option. Moved to mi_mmap.c/myisamdef.h. ulonglong myisam_mmap_size; Maximum allowed mapped address space. Moved to mi_mmap.c/myisamdef.h. ulonglong myisam_mmap_used; Sum of all mappings lengths. Moved to mi_mmap_c/myisamdef.h. static mysql_mutex_t myisam_mmap_used_lock; Mutex for myisam_mmap_used. Moved to mi_mmap.c. my_off_t MYISAM_SHARE::mmapped_length; Length of memory mapping. Stays as is. uint MYISAM_SHARE::nonmmapped_inserts; Number of out of memory mapping inserts. Stays as is. uchar *MYISAM_SHARE::file_map; Memory mapping. Stays as is. #define MAX_NONMMAPPED_INSERTS 1000 Threshold of nonmmapped inserts. When reached, try to remap. #define MEMMAP_EXTRA_MARGIN 7 Moved to myisamdef.h. void _mi_mmap_init_hook(void); Initialize mutex for 'myisam_mmap_used' variable. Must be called from myisam_init(). void _mi_mmap_deinit_hook(void); Destroy mutex for 'myisam_mmap_used' variable. Must be calledx from mi_panic(), if flag is HA_PANIC_CLOSE after all tables are closed. void _mi_mmap_open_hook(MI_INFO *info); Initialize memory mapping, setup mmapped I/O handlers. Merge implementations of mi_dynmap_file() and _mi_memmap_file(). Must be called from mi_open() when MYISAM_SHARE is created and test_if_locked & HA_OPEN_TMP_TABLE is false. void _mi_mmap_close_hook(MI_INFO *info); Destroy memory mapping. Must be called from mi_close() when MYISAM_SHARE is destroyed. void _mi_mmap_lock_hook(MI_INFO *info); Check if remap is needed and remap if there're no active users. Must be called from mi_lock_database(F_RDLCK/F_WRLCK). May be called only for uncompressed tables. void _mi_mmap_truncate_hook(MI_INFO *info); Destroy memory mapping before truncation. Setup I/O handlers for non-mmapped I/O. Can only be called with uncompressed tables. static size_t _mi_mmap_pread(MI_INFO *info, uchar *Buffer, size_t Count, my_off_t offset, myf MyFlags); static size_t _mi_mmap_pwrite(MI_INFO *info, uchar *Buffer, size_t Count, my_off_t offset, myf MyFlags); Memory mapped I/O handlers.
Analysis basing on mysql-next-mr (2010-03-19). include/my_base.h ----------------- Remove HA_OPEN_MMAP. Document HA_EXTRA_MMAP as obsolete. include/myisam.h ---------------- Move myisam_mmap_size and myisam_mmap_used declarations, MEMMAP_EXTRA_MARGIN definition, THR_LOCK_myisam_mmap declaration into MyISAM private headers. mysys/my_init.c, mysys/mysys_priv.h, mysys/my_thr_init.c -------------------------------------------------------- Move THR_LOCK_myisam_mmap to MyISAM. sql/mysqld.cc, sql/mysql_priv.h ------------------------------- Move myisam_use_mmap to MyISAM (should already be available as MyISAM plugin option). sql/records.cc -------------- Remove extra(HA_EXTRA_MMAP). Mapping must be setup on open. storage/myisam/ha_myisam.cc --------------------------- Set minimum value for myisam_mmap_size to 0. Remove mmap related code from ha_myisam::open(). Remove mmap related code from ha_myisam::extra(). Add myisam_mmap_used as status variable. storage/myisam/myisamdef.h -------------------------- Remove mmap_lock, mi_key_rwlock_MYISAM_SHARE_mmap_lock declarations. Remove mi_mmap_pread(), mi_mmap_pwrite() declarations. Remove MEMMAP_USED definition.
Copyright (c) 2000, 2024, Oracle Corporation and/or its affiliates. All rights reserved.