WL#3598: Disk Durable Commit

Documentation of Discussion 7 november 2006 14.30-15.30
Mikael Ronström, Jonas Oreland

The idea with the change is to supply a new flag on the NDB API that makes it 
possible
to ensure that a transaction is disk durable when reported back to the 
application.

The following is a short description of the implementation.
1) In normal operation the transaction is executed more or less as usual with the 
following
exceptions/changes:
 - Commit Signals carry one additional word which is the number of operations in 
the transaction
 - Commit Log Records gets a new type with three additional things, Operation Id, 
No Of Operations and
   Transaction Id
 - The log record is forced to disk during the execution of the Commit signal in 
LQH before sending it to the
next node. Locks are always released in the Complete Phase for this type of 
transaction.

2) During System Restart the following changes are made.
 - When finding a CommitSafe log record of GCI <= LastGCIToRestore then nothing 
special is done, it is executed
as usual. When GCI > LastGCIToRestore then a note about it is sent to TC
 - Execution of the log must continue until the last page written and not only to 
the LastGCICompleted record
 - When completed Log execution inform TC, TC will then decide which transactions 
to commit and which to abort.
 - TC will inform LQH about which of these special log records to execute, it will 
also inform about those which got
aborted by sending an invalidate message for those (probably same signal with a 
flag)
 - After executing the log and handling the TC completion then also ensure that 
two new GCI Completed log records
are written and synch those with the system file in DIH.


Time Plan:
1) Adapt Commit Signal, 1 day
2) New Read + Write Redo Log Entries, 1 day
3) New module in TC, 1 week
4) GCI Jumps 2 forward as part of SR as shown above, 1 week
5) Execute or Invalidate special log record at response from TC, 3 days
6) New flag in NDB API + TCKEYREQ + LQHKEYREQ, 1 day
7) Write Synched Log in LQH, 1 week

In total 1 month of work for someone knowledgeable in the NDB kernel trade.
This brings it to alpha/beta level, probably one month of extra debug+test is 
needed
to bring it to GA level.