WL#7205: Make the MTR rpl suite GTID_MODE agnostic
Affects: Server-5.6 — Status: Complete — Priority: Medium
GOAL ---- This worklog aims at making the MTR rpl suite GTID_MODE agnostic, so that we can have run it daily on pushbuild with --gtid-mode=ON. INTRODUCTION ------------ To keep coverage figures acceptable for the GTIDs feature, we want to start running the rpl suite with the following switches: --gtid-mode=on --enforce-gtid-consistency --log-slave-updates But due to the fact that MTR is result file oriented, we end up in a bad situation to make this happen (mostly because commands like SHOW BINLOG EVENTS or the output of mysqlbinlog prints out more information when GTIDs are turned on. This breaks many tests due to result file difference. There are more problems, but these are the most recurring ones. There are some tests, which rely on binlog positions or non-supported features when --gtid-mode=ON, that will fail, those tests must be identified and protected with include/not_gtid_enabled.inc. We should deprecate –-sync_slave_with_master (and lookalikes) and only use include/sync_slave_sql_with_master.inc (or siblings) which already know how to wait for positions or GTIDs. The goal of this worklog is to fix this. REFERENCES ---------- WL#6663: Make the MTR binlog suite GTID_MODE agnostic WL#6921: Migrate rpl suite to run with innodb engine
No functional or non-functional requirements.
we will need to add this Run for the default.daily push in the pb2. perl mysql-test-run.pl --force --timer --debug-server --parallel=auto -- experimental=collections/default.experimental --comment=rpl_gtid-debug -- vardir=var-rpl_gtid-debug --suite=rpl --mysqld=--binlog-checksum=CRC32 -- mysqld=--enforce-gtid-consistency --mysqld=--log-slave-updates --mysqld=--gtid- mode=on --skip-test-list=collections/disabled-gtid-on.list PROBLEMS ========= We want to run the rpl suite periodically with --gtid-mode=on. However, one cannot just turn on the necessary mysqld switches and expect the test cases to pass. There are several problems/failures due to: P1. result file differences due to extra output on SHOW BINLOG EVENTS P2. tests that mix transactional and non transactional engines in the same statement/transaction P3. the fact that some tests do not require the slave updates to be ON and with GTID_MODE=ON we fail to start the server. P4. Currently, even with --gtid-mode=on --enforce-gtid-consistency --log-slave-updates server options slave still connects to master using MASTER_AUTO_POSITION= 0. P5. BUGS SOLUTION ========= - To fix P1, Sven has created a filter in MTR + Perl that can filter out the GTID additional events from the result files in some cases. In particular, those that resort to --source include/show_binlog_events.inc However, this is not enough. When not enough, we create two wrappers for the test case. One works on traditional positioning and the other on gtid positioning. The former keeps the test name, the latter gets the additional name component: _gtid_ . For example: - rpl.rpl_binlog_errors becomes rpl.rpl_gtid_binlog_errors These should wrap the original test case that is moved into (keeping the original name, with the extension renamed from .test to .inc). Again, for the rpl_binlog_errors example: mysql-test/extra/rpl_tests/rpl_binlog_error.inc - Fixing P2 These tests are not supported. Thence they shall be added to the skip-test-list regarding gtid-mode=on. No point on testing these. - Fixing P3 We will need to fix the test that it does run with GTID_MODE=ON or if the test does not need to be run for GTID_MODE=ON we need to disable the test for GTID_MODE=ON. - Fixing P4 We will set MASTER_AUTO_POSITION to 1 by setting --let $use_gtids= 1 *before* including master-slave.inc. This setting must be automatic on rpl_init.inc before calling rpl_change_topology.inc. And it should be unset on rpl_end.inc before calling again rpl_change_topology.inc and reverts CHANGE MASTER to MASTER_AUTO_POSITION= 0. - Fixing P5 Report the bugs. Once reported the tests will be disabled till the bug are fixed. rpl.rpl_insert, rpl.rpl_mix_insert_delay : BUG#17803002 waiting to be fixed rpl.rpl_drop_temp, rpl.rpl_failed_optimize: BUG#17803141 waiting to be fixed These tests are disabled for now on this run only, until the bugs are fixed. What will change on new rpl tests ================================== 1) Instead of using MTR command --sync_slave_with_master use --source include/sync_slave_sql_with_master.inc The include file will use the appropriate mechanism to synchronize the slave with master: a) file+positions if GTID_MODE=OFF b) GTIDs if GTID_MODE=ON There are other include files for different sync points: * sync_slave_io_with_master.inc: only synchronizes IO thread * sync_slave_sql_with_io.inc: only synchronizes SQL thread with IO thread. 2) If test logic depends on master binlog files and positions, e.g. it has to assert binlog positions, or depends on GTID_MODE=OFF please use --source include/not_gtid_enabled.inc so that it is skipped when GTID_MODE=ON. 3) If test uses any of the non-supported statements/scenarios listed on http://dev.mysql.com/doc/refman/5.6/en/replication-gtids-restrictions.html please use --source include/not_gtid_enabled.inc so that it is skipped when GTID_MODE=ON. 4) If test depends on GTID_MODE=ON please use --source include/have_gtid.inc so that it is skipped when GTID_MODE=OFF. How to test with GTID_MODE=ON =============================== > $ perl mysql-test-run.pl --mysqld=--enforce-gtid-consistency --mysqld=-- log-slave-updates --mysqld=--gtid-mode=on TESTNAME PB2 ===== rpl suite with GTID_MODE=ON will run daily and weekly (with big tests enabled) on PB2
Copyright (c) 2000, 2017, Oracle Corporation and/or its affiliates. All rights reserved.