WL#3584: Global Transaction Identifiers (GTIDs)

Affects: Server-5.6   —   Status: Complete

==== SUMMARY ====

Implement globally unique positions of transactions in the binary log.
Purpose: solve several problems with automatic fail-over, circular replication, 
and fault tolerance.

==== PROBLEMS ====

P1 The binary log is missing information required to do automatic failover.
   Currently, failover requires manual intervention.

P2 Failover in a circular topology requires extra manual intervention to
   prevent binary log events from replicating forever in the circle and cause
   data corruption.

P3 The current format of binary log positions (filename + byte offset) has
   multiple problems:
    - local to the machine: a transaction changes position when logged on slave
    - hard to use
    - tied to a specific implementation
    - the position of one transaction cannot be used to predict the position
      of the next transaction without inspecting the binary log
    - not fault tolerant: if user specifies an invalid position, then the
      position is accepted and can cause data corruption

The purpose of this worklog is to fix the above problems. As side-effects, this 
will also fix a number of bugs (see references).

==== SOLUTION ====

S1 Introduce Global Transaction Identifiers (GTIDs). GTIDs are simply the
   server_uuid together with a sequence number:

     UGID = (server_uuid, sequence_number)

   Every transaction has its own GTID. GTIDs are propagated to slaves without
   transformation (so that they are indeed global).

S2 Ensure that a (slave) server never executes a transaction more than once.

==== REFERENCES ====

- Jan's ideas:
  http://jan.kneschke.de/2007/8/8/mysql-proxy-and-a-global-transaction-id

- One of the things we want to make simpler is
  the current procedure for fail-over in circular replication
  (Guilhem's text):
  http://dev.mysql.com/news-and-events/newsletter/2003-05/a0000000127.html

- Google's patch:
  http://code.google.com/p/google-mysql-tools/wiki/GlobalTransactionIds
  https://code.launchpad.net/~jtolmer/mysql-server/global-trx-ids

- Related blogs:
  http://www.techcrunch.com/2008/05/31/hey-twitter-i-have-a-few-questions-too/
  http://mysqldatabaseadministration.blogspot.com/2008/05/michael-arrington-
asks-twitter-few.html
  http://blog.twitter.com/2008/05/its-not-rocket-science-but-its-our-work.html

- Other work at MySQL related to binlog positions:
  WL#4047 Function to get last binary log position
  WL#3835 Slave tables to track progress of replication
  WL#3735 Replication begin/end log markers
The full high-level specification is in the attached file wl3584-hls-N.odt / 
wl3584-hls-N.pdf. The highest-numbered file is the latest version.

Old versions are kept for reference.