WL#9263: InnoDB: Enable partial access of LOB using multiple zlib streams

Affects: Server-8.0   —   Status: Complete

Current Behaviour:

When a table is created with compressed row format by specifying
ROW_FORMAT=COMPRESSED, all the index pages and BLOB pages are
compressed by making use of the zlib library.

Currently compressed BLOBs are stored as a single stream.  So to
decompress and access any portion of the BLOB, we have to decompress
from the beginning and proceed till the end.  This makes partial fetch
impossible.  For example, if we want to fetch the last 20 bytes of a
LOB, then we need to decompress the LOB from the beginning till the
end and then only we can access those 20 bytes.  

To solve this issue what we need is that the LOB must be compressed
into smaller zlib streams. This way we can avoid decompressing the
full LOB when only a small portion is needed to be read or modified.

One single LOB is stored into many smaller zlib streams instead of one
big zlib stream. The smaller zlib streams are placed one after the
other.  So the 2nd zlib stream starts where the 1st zlib stream ended,
the 3rd zlib stream starts where the 2nd zlib stream ends and so on.
So we can read the full LOB by sequentially processing the zlib

The Algorithm:

The input is an uncompressed BLOB data.  And the output is compressed
BLOB data in a sequence of zlib streams.  The input is split into
128KB chunks.  And each of those 128KB of uncompressed data is
compressed into a single zlib stream.  Then each of these zlib streams
are placed sequentially.

Insert Operation:

All inserts will follow the new approach whereby each LOB data page will be
stored as a sequence of smaller zlib streams.

Fetch Operation:

The fetch operation will be capable of handling old and new formats.  This is
necessary for backwards compatibility.

Destroy Operation:

This will be used by the purge.  Purge will also handle both old and new

Scope of this worklog:

The scope of this worklog is to change the way the LOB is stored
within InnoDB.  It deals with only the full LOB.  It does not deal
with partial fetch, or modify operations.  

Functional Requirements:

This worklog doesn't add new functionalities.  There is no externally visible
new functionality.  Everything should work as before.  

Non-Functional Requirements:

* There should be no memory leak or other memory issues.
* There should be no performance regression.  

The following functions can be provided to implement the feature.

Insert Operation:

Insertion of compressed LOB is performed by the zInserter class.  This class
provides the zInserter::write() member function to write an array of compressed
LOB data.  This worklog will introduce a new member function zInserter::insert()
that will write an array of compressed LOB data using the new approach.

Fetch Operation:

To fetch an LOB written in the old format we will use zReader::fetch(), the
current member function to read LOB.  This worklog will introduce a new member
function to read LOB in new format - zReader::read().

Convert Operation:

This worklog will introduce zInserter::convert_old_to_new() member function that
will be used to convert LOB written in old format to LOB written in new format.

Destroy Operation:

This should be largely unaffected by this worklog because it doesn't
read/interpret the zlib stream.