WL#4439: Multi Range Read implementation for Falcon
Affects: Server-9.x
—
Status: Un-Assigned
Implement Multi Range Read for Falcon storage engine. Falcon already internally does something similar to DS-MRR [*], but it is limited to individual ranges/index lookups. We could extend this to work across multiple ranges. [*] See WL#2475
1. Why this is needed and existing MRR roots in Falcon 2. Falcon/MRR implementation 2.1 Use Falcon's own memory 2.2 No scan interruption/continuation 2.3 Range association done by SQL layer 2.3 Falcon/MRR scan may return out-of-range records 1. Why this is needed and existing MRR roots in Falcon ------------------------------------------------------ When MySQL asks falcon to scan a range by calling falconTable->read_range_first() falconTable->read_range_next() ... falconTable->read_range_next() or falconTable->index_read_map() falconTable->index_next_same() ... falconTable->index_next_same() Falcon will reads records in record number order - it will first collect record numbers in a sparse bitmap and then do a record retrieval sweep. When one runs a join SELECT * FROM City, Airport WHERE City.ID= Airport.City when handler calls will be as follows: city->rnd_next() = 'Boston'; airport->index_read('Boston') = 'Logan' airport->index_next_same() = EOF city->rnd_next() = 'Moscow'; airport->index_read('Moscow') = 'Sheremetyevo' airport->index_next_same() = 'Vnukovo' airport->index_next_same() = 'Domodedovo' airport->index_next_same() = EOF city->rnd_next() = 'Paris'; airport->index_read('Paris') = 'Charles De Gaulle' airport->index_next_same() = 'Orly' airport->index_next_same() = EOF city->rnd_next() = 'Danvers'; airport->index_read('Danvers') = EOF Lots of ranges with small number of records in each (typical for joins) will break Falcon's sweep-read strategy /* Btw, customers do file CSC cases with *huge* ranges, e.g. SELECT * FROM tbl WHERE key IN (val1, ......, val10K) */ 2. Falcon/MRR implementation ---------------------------- The SQL Layer part is ready: there is MRR interface (WL#2474) and Batched Key Access (WL#2771). We just need to extend Falcon's sweep-read functionality to span across ranges. 2.1 Use Falcon's own memory --------------------------- MRR interface is designed with centralized buffer space management in mind - the SQL layer is supposed to allocate a single MRR buffer and then share it with all involved storage engines and tables (this is a step in a direction towards being able to control memory usage). Decision made at Manchester,MA meeting: Falcon will still use its own internal memory buffers, like it does for regular range/ref scans. This is more in line with Falcon's architecture. The case when there is not enough memory for the record bitmap is considered extreme, we intentionally do not handle it (see all arguments re 64 bit architectures, linux's overcommit etc) 2.2 No scan interruption/continuation ------------------------------------- Following the direction of the previous decision: since we assume there is enough memory for the record buffer, the entire Falcon/MRR scan can be done in one "collect rowids, retrieve records" iteration. This makes Falcon/MRR code much easier as Falcon has no natural way to interrupt an index scan and then continue (Sergey P tried to add it and it proved to be non-trivial. Non-triviality of this has been confirmed by Ann). 2.3 Range association done by SQL layer --------------------------------------- Falcon accumulates rowids (falcon's term is "record numbers") in a sparse bitmap, which means that at record retrieval stage it won't be able to tell which range the record is expected to be in. The apparent resolution is to add the code that will use a hash table to determine which range the obtained record falls into (hash table is enough because we're primarily interested in equality ranges). This will be done at the SQL layer. Falcon/MRR will only set MRR_NO_ASSOCIATION flag to signal that it is not able to do range association. 2.3 Falcon/MRR scan may return out-of-range records --------------------------------------------------- See previous section. In falcon, it might turn out that the record has been altered by another transaction and now is outside of the scanned range. For single range scans, Falcon code checks if the record falls within the range before returning it. In MRR scan Falcon will not be able to do that because it would not know which of the ranges the record is supposed to be in. Since range association is done at the SQL layer, the check if returned record is contained within some of the scanned ranges will be done there as well. That is, Falcon/MRR scan may return records that do not fall into any range. The caller is expected to filter them out.
None needed as HLS is detailed enough.
Copyright (c) 2000, 2024, Oracle Corporation and/or its affiliates. All rights reserved.