WL#3147: MyISAM repair code cleanup
Affects: Server-7.1 — Status: Un-Assigned — Priority: Medium
Remove code duplication and fix possible numeric overflow in the MyISAM repair code. In the course of analyzing BUG#11527 (Setting myisam_repair_threads to >1 leads to corruption), I found two areas of code duplication and a possible integer overflow with very big tables (in addition to the overflow that caused the bug).
Copyright (c) 2001-2007 by MySQL AB. All rights reserved. Details: All of this happens in myisam/sort.c. thr_find_all_keys() duplicates code from _create_index_by_sort() and find_all_keys(). I suggest to move the duplicated part of _create_index_by_sort() into find_all_keys() and just call find_all_keys() from thr_find_all_keys(). In the duplicated code from _create_index_by_sort() we compute with the number of records. If this is very big, a numeric overflow can happen. An additional condition can check for this. What I mean is this: - if ((my_off_t) (records + 1) * (sort_length + sizeof(char*)) <= (my_off_t) memavl) + /* + With true varchar we could have records much shorter than key_length. + We could have so many of these that records * key_length overflows + my_off_t. So we take a two step decision here. + */ + if ((records < memavl) && + (my_off_t) (records + 1) * (sort_length + sizeof(char*)) <= (my_off_t) memavl) Also the variables 'maxbuffer' and 'skr' should be of type ' ha_rows' to be safe.
Copyright (c) 2000, 2016, Oracle Corporation and/or its affiliates. All rights reserved.