WL#9073: Add an MTR option --no-skip to force all the tests to run

Problem description:

1. In MTR, tests are skipped if all pre-requisites of the test are not 
met. This can be risky because we may end up with tests that are never run because 
the conditions are never met in the regression test environment. for ex -
   In case of a missing plugin or a utility on which the test depends, the test 
will skip. In these cases, the skip behaviour is incorrect because a genuine 
dependency miss will go undetected in MTR test runs.

2. Second part of the task is to identify the tests for which skip is the correct 
behaviour. These tests should continue to skip to avoid unnecessary failures. for 
ex - 
   a. OS dependent files.
   b. Different binlog formats. - Since replication suite same test with all 3
binlog formats. Some tests cannot run with some formats and should be skipped
   c. Different compile options.

3. Third part of the task is, we do not want to change user experience. 


Proposed solution:

1. A new MTR option (--no-skip) will be introduced. This option will allow each 
MTR test to run without any skip. This will help us solve the first problem 
stated.

2. An excluded list will be maintained to keep track of tests which should 
continue to skip to avoid unwanted failures. This will help us solve the second 
problem stated.

3. The default value of this introduced variable will be OFF. This implies that 
users will not be forced to run all the tests and they will not have to deal with 
unwanted failures due to missing dependencies. To use/not-use this new option will 
be at the user's disposal.

FR1 : There shall be an MTR option , --no-skip that prevents tests from skipping 
when needed dependencies are not met

FR2 : There shall be an option to specify a list of inc files and tests that shall 
continue to be skipped even if --no-skip option is specified. This is needed to 
check for conditions (eg: OS dependency) that are impossible to satisfy always.

FR3 : The --no-skip option must be OFF by default and tests shall be skipped when 
dependencies are not met

An exclude list of include(.inc) and test(.test) files is created and kept at 
the location include/excludedList.list

Based on the excluded list, following tests will continue to skip:

1. All tests which will source the include files mentioned in the excluded list.
2. All tests which are mentioned in the excluded list.

The exclude list needs to be updated in future in case a new MTR test is 
introduced which we would like to skip depending on a newly introduced 
condition.

Below is the current excluded list:

/* OS dependent files*/

Reason for inclusion: Below files are OS specific and tests will run only on 
selective platform where conditions are met and skip on others.

1.  have_32bit.inc
2.  have_64bit.inc
3.  have_case_insensitive_file_system.inc
4.  have_case_sensitive_file_system.inc
5.  have_crypt.inc /*supported on Linux platform */
6.  have_lowercase0.inc
7.  have_lowercase1.inc
8.  have_lowercase2.inc
9.  linux.inc
10. windows.inc
11. not_windows.inc
12. have_open_file_limit_more_than_5000.inc
13. have_partition_open_file_limit.inc
14. table_compress_2.test /*checks for DATADIR that supports hole punching*/
15. table_compress_3.test /*checks for DATADIR that supports hole punching*/

/* Client uses the posix locale as default client character set.*/

/* Older unices have the whole collection by default. Also el6 and el7 where 
test runs fine. But el8 is like modern ubuntu, with just a small set of utf8 
locales. On Ubuntu, there is a list of supported locales 
(/usr/share/i18n/SUPPORTED). From this list, new locales may be generated with a 
locale generator (locale-gen). The reason for this change is that the whole 
collection of locales have gotten pretty huge (and is still growing) and 
increases the size of glibc

The reason for retaining this inc file in the excluded list is that new Unix-
installations may not have the locales needed for the test and without the inc-
file the test will fail. Also, the test is not supported on Windows. */

16. have_posix_locale.inc

/* Binlog formats */

Reason for inclusion: Below files are included because rpl suite runs each test 
with 3 different binlog format combinations. The test may not be meant to run on 
all binlog formats and will skip.

17. have_binlog_format_mixed_or_row.inc
18. have_binlog_format_mixed_or_statement.inc
19. have_binlog_format_mixed.inc
20. have_binlog_format_row_or_statement.inc
21. have_binlog_format_row.inc
22. have_binlog_format_statement.inc

/* Compile options */

Reason for inclusion: Below files are included because few tests are meant to be 
tested with specific compile options which is not available in all builds.

/* Test requires --debug-server option*/
23. have_debug_sync.inc 
24. have_debug.inc
25. have_nodebug.inc
26. mysql_have_debug.inc

/*have_log_bin.inc will be set using an opt file with option (--log-bin). The 
inc file is removed from the excluded list. There will be no duplicate runs 
because the usage of have_log_bin.inc is not eliminated from the tests due to 
internal MTR dependency on this inc file being sourced. */

/*not_log_bin.inc can be set using an opt file with option (--skip-log-bin) and 
usage of this inc can be removed from the test. However, this will cause the 
same tests to run again when --log-bin option will be used to run the suites. 
This will cause duplicate test runs leading to increased test execution 
time.Removal of not_log_bin.inc does not come in the scope of this worklog and 
will be done seperately.*/

27. not_log_bin.inc

/* Test is currently running on Dev build linux-x86-64bit-devbld */
28. mysqlbinlog_have_debug.inc
29. have_innochecksum_debug.inc

/*Test with yassl is supported on community build */
30. have_yassl.inc

/*Test with openssl is supported on commercial build */

31. have_openssl.inc
32. have_openssl_binary.inc /*checks for openssl binary */

/* variable have_ssl is compiled default YES but in case server is started with 
--skip-ssl  or if the test is run with invalid certificates; in that case 
have_ssl value would be NO or DISABLED */ 

33. have_ssl_communication.inc /*checks for have_ssl value to be YES*/
34. have_ssl_crypto_functs.inc /*checks for have_ssl value to be YES|DISABLED*/

/* Innodb page size */

Reason: Page sizes 4K,8K,16K,32K,64K is set within the test using a custom 
bootstrap file. However, below is a special case which can run with 4K,8K and 
16K at the same time and hence required to skip tests on 32K and 64K

35. have_innodb_max_16k.inc

/* IP versions */
36. have_ipv4_mapped.inc

/* System variable which checks whether server has statement_timeout feature or
   not.The value can be NO if the background thread used by this feature could
   not be initialized during server start */

37. have_statement_timeout.inc

/*Miscellaneous */

Reason for inclusion: Below .inc files are included because of the need of 
running the test with different protocols, different builds.

38. have_ps_protocol.inc
39. have_no_protocol.inc
40. no_view_protocol.inc
41. no_sp_protocol.inc
42. no_opt_trace_protocol.inc
43. no_explain_protocol.inc
44. no_cursor_protocol.inc
45. have_embedded.inc
46. not_embedded.inc
47. have_mysqld_safe.inc
48. have_valgrind.inc


/* Few skipped test-cases */

Reason for inclusion: Few tests have custom skip logic written within the test 
and needs to be skipped.

/* checks for mysql_embedded executables to be present */
49. mysql_embedded.test

When an MTR test is run with (--no-skip) option, MTR (mysql-test-run.pl) will pass 
the --no-skip option & the excluded list of include files to the mysql client 
(mysqltest.cc)

The mysql client will read the test file and also the excluded include files and 
based on the match or no-match take the decision to skip or run the test.

In case --no-skip option is not passed, the excluded list will also be ignored by 
MTR (mysql-test-run.pl). The test will run as per current behaviour.

Below are the list of plugins that will be benefited by this implementation.

have_archive_plugin.inc
have_audit_log_plugin.inc
have_blackhole_plugin.inc
have_blackhole.inc
have_daemon_example_plugin.inc
have_example_plugin.inc
have_federated_plugin.inc
have_group_replication_plugin.inc
have_mecab.inc
have_memcached_plugin.inc
have_mysql_no_login_plugin.inc
have_ngram.inc
have_not_innodb_plugin.inc
have_null_audit_plugin.inc
have_partition.inc
have_plugin_auth.inc
have_plugin_interface.inc
have_plugin_server.inc
have_simple_parser.inc
have_udf.inc
have_validate_password_plugin.inc