WL#4180: Persistent object versions

Affects: Server-Prototype Only   —   Status: Un-Assigned   —   Priority: Medium

Implement persistent object versions, that are stored along with object definition
for the following objects:
 * tables
 * views
 * stored procedures
 * stored functions

 * provide a persistent version for validation of prepared statements (WL#4165).
   The current suggestion of WL#4165 is to use non-persistent object versions,
   since they require no changes in the data dictionary storage format.
   The drawback, however, is that non-persistent versions are prone to 
   "false negatives" during statement validation (that is,
   render some valid statements as invalid).
 * implement a truly online ALTER TABLE. Online ALTER TABLE
   today must wait for all old transactions to finish to be able to
   remove the old version of the table from the table definition cache.
   Currently the table definition
   cache is a hash with key being <db>.<table_name>. If each table has
   a version, the table definition cache can include it into the key,
   or keep two versions of the table in the cache in some other way,
   and thus it will be possible to ship the old version of TABLE_SHARE
   to "old" transactions, and the new version of it to "new" transactions,
   i.e. transactions that started after online ALTER TABLE completion.

 1. objects of different types may have the same version value
 2. objects of the same type with different names may have the same version value
 3. objects of the same type *and* having the same name (i.e. a table
that was dropped and created anew) may (but does not have to) have the same
version value but only if other metadata is identical.
 4. in all other cases, object versions should never be equal

Less strict requirements will lead to "false positives" in statement validation
of WL#4165.

Choice of object version

1) Global counter

Implement a system table that stores a global counter incremented upon each DDL

 - allows to track relative age of objects, i.e. say that this stored procedure
is older than this view
 - an object with the same name that was dropped and created anew gets a new

 - each DDL statement needs access to the system table with the global counter

2) Local counter.

Store a local counter along with the definition of the object. The counter
may start at 0, and then be incremented with each alteration of an object.

Not acceptable as violates requirement #3. Listed here only because is a popular

3) Version value derived from object metadata 
 3a. MD5 of object definition.

 * does not have to be persistent -- and therefore allows for backward
compatibility with data dictionaries of 5.0 and 5.1.
 * yields the same value for the same metadata

 * size is 32 bytes. Considered neglectible.
 * cost of calculation: O(metadata length) Considered neglectible.

 * size: 36 bytes
 * calculation cost: O(1) + cost of high-precision timer

5) timestamp (high-precision, probably)
  * this is a simplified version of the 4th choice (timestamp is a part of UUID)

6) Simplified UUID, made as a combination of the current time (unixtime since
the start of an epoch) and a pseudo-random number, to resolve possible
collisions within timer resolution (1 second)

7) A two-part version number, where first part only changes when
table layout has changed (column order, names and types). The second
part is built using one of the algorithms listed above. The first part
can be used to track "incompatible" changes to a table or a view.

8) At the architecture board in Stockholm, March 2008 it was decided to
use the following object version format:
1 byte (approx.) - format type
1 byte (approx.) - length of the version. Could be collapsed with format type
blob - version body
Version body: two-part number that consists of high and low part.
High part changes after offline ALTER, low part changes
when after online ALTER
This object version should be introduced as part or after WL#2760 (New Data

BUG#15638 need an id for db objects