WL#4218: Online Backup : Test Transactional Consistency (using invariants)

MySQL backup is supposed to make a transaction consistent backup
image at time t of the databases that are backed up. This means
that at restore time, all transactions committed before time t
should be restored but no transactions committed after time t. In
MySQL backup terminology, the point t in time is called "the
validity point".

This WL will test that transactions executed concurrently with
backup are either completely restored or not restored at all. No
transaction should be partially represented after restore.

The following figure illustrates backup and concurrent transactions. In this
figure, transactions T1-T4 should be completely restored because they all
committed before time t. Transactions T5-T6, on the other hand, should not be
restored at all since they committed after time t:

========================================

T1  T2                             Time  
*   *                               |
*   *   T3                          |
C   *   *                           |
    *   *              Backup       |
    *   *                *          |
    C   *                *          |
        *   T4           *          |
        C   *            *          |
            *   T5       *          |
            *   *        *          |
            C   *        *          |
--------------- * ----- (vp) ---- Time t
                *        *          |
                *   T6   *          |
                C   *    *          |
                    *  (done)       |
                    C               V
  
========================================



Note: A similar property is checked in the validity point tests
backup_vp_[nontx|tx]. However, in these tests emphasis is put on
alignment of transactions with the validity point. This implies a
testing methodology where execution of the BACKUP command is
fine-controlled using debug synchronization points. 

Note: there is also a WL for testing VP without micromanaging
synchronization points: WL#4769

Note: This test was discussed and planned during Backup Camp in Stockholm, Jan 2009.

This worklog shall produce a suite of tests that makes use of
invariants to check that backup makes transactionally consistent
backup images. In other words, verify that:

"Any transaction executing concurrently with backup is either
completely included in the database after restore, or is not
included at all".

Test methodology:
-----------------
M1 A databases will be filled with data simulating bank accounts
M2 The sum of money across all accounts shall be exactly 1
   million dollar
M3 The random query generator (RQG) will run alone for a short
   while before backup is started.
M4 Once backup has completed, the RQG is stopped, and restore is
   executed.
M5 When restore has completed, the tables are inspected. The sum
   of money across all accounts should still be 1 million dollar.
   Any other sum indicates that a transaction is only partially
   restored.

Requirements for concurrent transactions:
-----------------------------------------
R1 The concurrent load must be transactional
R2 There shall be at least 5 clients running concurrently with backup
R3 Together, the transactions shall include all SQL DML operators
   - delete 
   - insert
   - update
   - select
R4 All transactions shall consist of at least 2 operations.
R5 All transactions that commit shall preserve this rule: "Any
   money withdrawn from an account has to be deposited in another
   account". 
R6 A subset of the transactions shall be aborted. These
   transactions do not have to follow rule R5.

Test pass or failure:
---------------------
The test has passed if and only if backup and restore succeeds and
the invariant holds. In other words, the sum of money across all
accounts is exactly 1 million dollar after restore.