WL#4618: RBR: extended table metadata in the binary log

Affects: Server-Prototype Only   —   Status: Un-Assigned   —   Priority: Medium

EXECUTIVE SUMMARY
=================

This worklog extends the table metadata that is written to the binary
log in row based logging. As of today for each table, a
Table_map_log_event is written.  It contains information about the
database name, the table name, the number of columns the column types
and a bit set stating which columns can be NULL and those that cannot.

SCENARIO 1 - Character Sets
===========================

MySQL currently supports statement based replication between
tables with columns having slightly different data types.
It includes columns with different character sets.

If you create this table on master:

CREATE TABLE t1 (a varchar(10) character set latin1);

  and this table on slave:

CREATE TABLE t1 (a varchar(10) character set utf8);

then statement based replication works fine.
When slave re-parses a statement, it converts strings to utf8
from character_set_client of the event to utf8.

Row based replication doesn't work because
it operates with binary images. So the strings in
binary log are represented in latin1 character set,
and no conversion happens to utf8.

Possible solution how to make it work:
- Extend table map event to transfer character set ID together
  with other column meta data, like data type, length and so on.
- If character set of the data in the image is not the same with
character of the column, then apply character set conversion.


SCENARIO 2 - Signedness
=======================

If using different signed columns on master and slave, the slave
may go out of sync because there is no information shipped with
the table metadata regarding the signedness of the field.

For example (if using @@slave_type_conversions='ALL_NON_LOSSY'):

MASTER> CREATE TABLE t (c1 INT UNSIGNED);
 SLAVE> ALTER TABLE t CHANGE COLUMN c1 c1 BIGINT SIGNED;
MASTER> INSERT INTO t(c1) values (3916586877);
MASTER> SELECT * FROM t; ---> 3916586877
 SLAVE> SELECT * FROM t; ---> -378380419

SCENARIO 3 - Enable External Programs to Decode Info in RBR
===========================================================

Some [3] have, for instance, created their own Table_metadata_log_event,
which contains information on SIGNEDNESS, column names, column comments,
etc.

REFERENCES
==========

[1] BUG#15831300: SLAVE_TYPE_CONVERSIONS=ALL_NON_LOSSY NOT WORKING AS EXPECTED
[2] WL#4097: RBR: replication between different character sets and lengths
[3] https://github.com/twitter/mysql/wiki/Change-History#wiki-5.5.24.t7