WL#10470: Defaults: Enable Binary Log

EXECUTIVE SUMMARY
=================

This worklog changes the log-bin default. After this worklog is implemented
the default will change from OFF to ON.

Rationale
=========

Nearly all production installations have the binary log enabled as it's used
for replication and point-in-time recovery.

Given that, we should have it enabled by default for the following reasons:
1. We eliminate one configuration step for users.
  1A. Enabling it later requires a mysqld restart.
2. We get more production-like internal testing of the server.
3. We can better know and face the performance impact of the binary log.


User Stories
============

- As a MySQL DBA I want the binary log turned on by default so that
  I can extend my asynchronous replication seamlessly by just spinning
  up a new server without having to tweak it much.

- As a MySQL DBA I want the binary log turned on by default so that
  I easily setup and add a new server in a replicaset within InnoDB
  cluster, by just spinning up a new server without having to tweak
  it much.

- As a MySQL DBA I want the binary log turned on by default so I
  can take advantage from its infrastructure easily while deploying
  a MySQL server together with other message bus (like Kafka) or
  indexing infrastructure (like lucene).

- As a MySQL User I want to spend just a few seconds setting up a
  full fledged MySQL server before I start using so that I can start
  prototyping quickly against it.

- As a MySQL User I want to have the least configuration impedance so
  I can start using MySQL and go from prototype to a realistic deployment
  with the minimum hassle.

- As a MySQL user I want to have the binary log on by default, so that
  I don't have to do a server restart later on, when I decide to enable
  it.

As a matter of fact, it all boils down to removing configuration overhead
for a practical deployment of the MySQL server.

Development Stories
===================

- As a MySQL tester we want the binary log turned on by default so that
  we remove the overhead of planning and contemplating the fact that the
  binary log has to be turned on on our test plans. It comes naturally.

- As a MySQL performance engineer we want the binary log to be on by
  default so that we continue to focus on replication performance and
  efficiency without having to keep in mind to turn it on.

As a matter of fact, it all boils down, again, to removing the impedance
and planning overhead to consider servers with binary logs ON in some
scenarios. It is better that binary log is an opt-out rather than an
opt-in thing.


Existing behavior
=================

1. By default binary logging is DISABLED, and, corresponding variables are set
   as follows:
   log_bin= OFF
   log_bin_basename= 
   log_bin_index= 

2. On providing --log-bin option without string or empty string, binary logging
   is ENABLED, and, corresponding variables are set as follows:
   log_bin= ON
   log_bin_basename= {datadir}/hostname-bin
   log_bin_index= {datadir}/hostname-bin.index.

3. The default value of server-id is 0.

4. The default relay log file names are {datadir}/host_name-relay-bin.xxxxxx.

5. The default relay log index file name is host_name-relay-bin.index.

6. Same argument can be specified for both --log-bin and --relay-log. This can
   result into undesired behaviors on server.

7. If --log-bin option is specified without specifying a --server-id, then
   server is not allowed to initialize.

8. If server is started with --skip-log-bin --log-bin, then binary logging is
   ENABLED, and, corresponding variables are set as follows:
   log_bin= ON
   log_bin_basename= {datadir}/0
   log_bin_index= {datadir}/0.index


Server Changes
==============

1. Now, default binary logging is ENABLED, and, corresponding variables are set
   as follows:
   log_bin= ON
   log_bin_basename= {datadir}/binlog
   log_bin_index= {datadir}/binlog.index

2. On setting --log-bin option without string or empty string, binary logging
   is ENABLED, and, corresponding variables are set as follows:
   log_bin= ON
   log_bin_basename= {datadir}/binlog
   log_bin_index= {datadir}/binlog.index.

3. The default value of server-id is changed to 1.

4. The default relay log file names are {datadir}/relaylog.xxxxxx.

5. The default relay log index file name is relaylog.index.

6. Server is not allowed to start when binary and relay logs have same base
   name. Either using --log-bin and --relay-log with same non-empty parameter,
   using --log-bin=relaylog or --relaylog=binlog.

7. If --log-bin option is specified without specifying a --server-id, then
   server is now allowed to initialize with a warning.

8. If server is started with --skip-log-bin --log-bin, the binary logging is
   ENABLED, and, corresponding variables are now set properly as follows:
   log_bin= ON
   log_bin_basename= {datadir}/binlog
   log_bin_index= {datadir}/binlog.index.


Following existing behaviors (9,10,11) are not changed and must work as earlier:

9. Binary logging remains disabled on --initialize[-insecure] by default.

10. --log-bin=anyuserprovidedname
    If a user provides string to --log-bin option, then binary logging is
    ENABLED using {datadir}/anyuserprovidedname.xxxxxx.

11. --skip-log-bin, --disable-log-bin
    If a user provides --skip-log-bin or --disable-log-bin, then binary logging
    is DISABLED.


Upgrade Story
=============
There is an upgrade impact with this WL. Consider the example case as below:-

MySQL 5.7: --log-bin
  index file name: .index

MySQL 8.0: --log-bin
  index file name: binlog.index

When the server starts up, it will not be able to find the existing binlog
files, because it will create the binlog.index file instead of reading
.index.

The workarounds are:
  1. rename the file .index to binlog.index
  2. specify the index file --log-bin-index=.index


Acceptance Criteria
===================
1. These settings should not cause any instability/failures in our automated
   tests.
2. These settings should not cause more than an average 20% regression in SysQA
   write perf tests.
3. User must be able to turn off binary logging by using --skip-log-bin.


Removed
=======
This worklog earlier suggested to change:
1. Default for server-id to a random 32-bit integer. This was dismissed because:
 - We consider setting default to 1 with this WL.

2. Set binary log file name to 'mysqld-bin'. This was was dismissed because
   'binlog' looks more suitable and user friendly.

Functional Requirements:
========================
Code level:-
-------------
F1: If --log-bin, --skip/disable-log-bin, --loose-*log-bin are not provided,
    then by default log-bin must be enabled using {datadir}/binlog.xxxxxx.

F2: If user provides --log-bin, or --log-bin=, then log-bin must be enabled
    using {datadir}/binlog.xxxxxx.

F3: If user provides --log-bin='mysqld-bin', then log-bin must be enabled using
    {datadir}/mysqld-bin.xxxxxx.

F4: If user provides --skip-log-bin, or --disable-log-bin, then binary logging
    must be disabled.

F5: If user provides both --skip/disable-log-bin and --log-bin in options, then
    which ever option is provided later must be considered.
    For example:- '.. --log-bin --skip-log-bin ..' makes log-bin disabled.

F6: The default server-id must be 1.

F7: Server-id must be changeable dynamically according to the infrastructure.

F8: Binary logging must remain disabled on --initialize[-insecure] by default.

F9: Must emit a warning at initialize time that the server-id needs to be set
    according to the infrastructure.

F10: The default relay log file names and index file must be using
     {datadir}/relaylog.xxxxxx and {datadir}/relaylog.index respectively.

F11: Server must fail to start when binary and relay logs base names are set to
     same.


Test framework:-
-----------------
FR1: Update the MTR .result files and JET files to match with new output.

FR2: Update the MTR .test files and JET files to work fine with new changes.

FR3: Update mysql-test-run.pl and related files for new changes.

FR4: Update command line setup for PushBuild.

FR5: Testcases which require binary logging must pass when log-bin is enabled.

FR6: Testcases which require binary logging must be skipped when skip-log-bin
     is used.

FR7: Testcases which have no impact due to binary logging must pass with either
     log-bin or skip-log-bin.

FR8: Testcases which need binary logging to be disabled must be skipped with
     log-bin.

FR9: Testcases which need binary logging to be disabled must pass with
     skip-log-bin.


Non-functional Requirements:
=============================
NFR1: The enabling of log-bin must not cause more than an average 20%
      regression in SysQA perf tests.

User Interface
==============

- log-bin is ON by default (used to be OFF)
  - binary log index default filename changes from .index to 
binlog.index
  - binary log files default filename prefix changes from .XXXXXX to 
binlog.XXXXXX
- relay log index default filename changes from -relay-bin.index to 
relaylog.index
- relay log files default filename prefix changes from -relay-
bin.XXXXXX to relaylog.XXXXXX
- server-id default changes from 0 to 1.
- initialize and initialize-insecure must emit a warning stating that the
  server-id needs to be adjusted, when a server-id is not specified.
- server fails to start if binary and relay logs are set to have same base
  names.


Observability
=============

Nothing is removed. Only the server defaults are changed. And, following
two messages (one error, one warning) have been introduced:
ER_RPL_CANT_HAVE_SAME_BASENAME
 - To avoid log-bin and relay-log from having the same basename value.

ER_WARN_NO_SERVERID_SPECIFIED
 - Warning for users to specify --server-id along with log-bin during
   server initialization.


Security
========

No security consideration, other than the fact that users can now fill up
the storage space more quickly when compared to a binlogless deployment.
This has to be documented so that DBAs are aware of this change.


Upgrade/Downgrade
=================

There is upgrade/downgrade impact with this WL. But, workarounds are there in
place.

For example:-
MySQL 5.7: --log-bin
  index file name: .index

MySQL 8.0: --log-bin
  index file name: binlog.index

When the server starts up, it will not be able to find the existing binlog
files, because it will create the binlog.index file instead of reading
.index.

The workarounds are:
  1. rename the file .index to binlog.index
  2. specify the index file --log-bin-index=.index

This has to be documented so that DBAs are aware of this.


Performance
===========
These settings should not cause more than an average 20% regression in SysQA
write perf tests.