 |
MySQL 8.0.39
Source Code Documentation
|
|
MySQL 8.0.39
Source Code Documentation
|
The mysql-test-run.pl Perl script is the main application used to run the MySQL test suite.
It invokes mysqltest to run individual test cases.
Invoke mysql-test-run.pl in the mysql-test directory like this:
shell> mysql-test-run.pl [options] [test_name] ...
Each test_name argument names a test case. The test case file that corresponds to the test name is t/test_name.test.
For each test_name argument, mysql-test-run.pl runs the named test case. With no test_name arguments, mysql-test-run.pl runs all .test files in the t subdirectory.
If no suffix is given for the test name, a suffix of .test is assumed. Any leading path name is ignored. These commands are equivalent:
shell> mysql-test-run.pl mytest shell> mysql-test-run.pl mytest.test shell> mysql-test-run.pl t/mytest.test
A suite name can be given as part of the test name. That is, the syntax for naming a test is:
[suite_name.]test_name[.suffix]
If a suite name is given, mysql-test-run.pl looks in that suite for the test. The test file corresponding to a test named suite_name.test_name is found in suite/suite_name/t/test_name.test. There is also an implicit suite name main for the tests in the top t directory. With no suite name, mysql-test-run.pl looks in the default list of suites for a match and runs the test in any suites where it finds the test. Suppose that the default suite list is main, binlog, rpl, and that a test mytest.test exists in the main and rpl suites. With an argument of mytest or mytest.test, mysql-test-run.pl will run mytest.test from the main and rpl suites.
To run a family of test cases for which the names share a common prefix, use the --do-test=prefix option. For example, --do-test=rpl runs the replication tests (test cases that have names beginning with rpl). --skip-test has the opposite effect of skipping test cases for which the names share a common prefix.
The argument for the --do-test and --skip-test options also allows more flexible specification of which tests to perform or skip. If the argument contains a pattern metacharacter other than a lone period, it is interpreted as a Perl regular expression and applies to test names that match the pattern. If the argument contains a lone period or does not contain any pattern metacharacters, it is interpreted the same way as previously and matches test names that begin with the argument value. For example, --do-test=testa matches tests that begin with testa, --do-test=main.testa matches tests in the main test suite that begin with testa, and --do-test=main.*testa matches test names that contain main followed by testa with anything in between. In the latter case, the pattern match is not anchored to the beginning of the test name, so it also matches names such as xmainytesta.
It is also possible to put a list of test names in a file and have mysql-test-run.pl run those tests, using the option --do-test-list=file. The tests should be listed one per line in the file. A line beginning with # indicates a comment and is ignored. Similary an empty line in the file is also ignored.
mysql-test-run.pl also supports a --do-suite option, which is similar to --do-test but permits specifying entire suites of tests to run.
To perform setup prior to running tests, mysql-test-run.pl needs to invoke mysqld with the --initialize and --skip-grant-tables options. If MySQL was built with the compiler flag -DDISABLE_GRANT_OPTIONS, then --initialize, --skip-grant-tables, and --init-file will be disabled. To handle this, set the MYSQLD_BOOTSTRAP environment variable to the full path name of a server that has all options enabled. mysql-test-run.pl will use that server to perform setup; it is not used to run the tests.
The init_file test will fail if --init-file is disabled. This is an expected failure in this case.
To run mysql-test-run.pl on Windows, you'll need either Cygwin or ActiveState Perl to run it. You may also need to install the modules required by the script. To run the test script, change location into the mysql-test directory, set the MTR_VS_CONFIG environment variable to the configuration you selected earlier (or use the --vs-config option), and invoke mysql-test-run.pl. For example (using Cygwin and the bash shell):
shell> cd mysql-test shell> export MTR_VS_CONFIG=debug shell> ./mysqltest-run.pl --force --timer shell> ./mysqltest-run.pl --force --timer --ps-protocol
mysql-test-run.pl uses several environment variables. Some of them are listed in the following table. Some of these are set from the outside and used by mysql-test-run.pl, others are set by mysql-test-run.pl instead, and may be referred to in tests.
| Variable | Description |
|---|---|
EXE_MYSQL | Path to the mysql client executable |
MTR_BUILD_THREAD | If set, defines which port number range is used for the server |
MTR_MAX_PARALLEL | If set, defines maximum number of parallel threads if --parallel=auto is given |
MTR_MEM | If set to anything, will run tests with files in "memory" using tmpfs or ramdisk. Not available on Windows. Same as --mem option |
MTR_NAME_TIMEOUT | Setting of a timeout in minutes or seconds, corresponding to command line option --name-timeout. Avaliable timeout names are TESTCASE, SUITE (both in minutes) and START, SHUTDOWN, CTEST (all in seconds). MTR_CTEST_TIMEOUT is for ctest unit tests. |
MTR_PARALLEL | If set, defines number of parallel threads executing tests. Same as |
MTR_PORT_BASE | If set, defines which port number range is used for the server |
MTR_RECORD | Set to 1 if a MTR run is started with --record option, 0 otherwise. |
MTR_UNIQUE_IDS_DIR | The method in which the free ports are allocated is by maintaining a list under the unique ids directory. In case there are multiple chroot environments on the same host, then the same set of ports may get allocated for all environments, because there will be multiple unique ids directories in different physical locations, but MTR assumes it is the same directory. This will lead to a conflict while reserving ports for the different MTR runs. To avoid this problem, when using chroot environments, MTR_UNIQUE_IDS_DIR environment variable can be set to a common location, so that all the MTR processes will use the same unique ids directory. Users have to make sure this variable is set to the same path on all environments, and that this common path is mounted on all the environments. |
MYSQL | Full path to mysql client executable ,along with options passed to server. |
MYSQL_BIN_PATH | Path to the directory having mysql binaries. |
MYSQL_CLIENT_BIN_PATH | Path to the directory having client binaries. |
MYSQL_CONFIG_EDITOR | Path name to mysql_config_editor binary. |
MYSQL_OPTIONS | Options passed to server when connecting from mysql client. |
MYSQL_TEST | Path name to mysqltest binary |
MYSQL_TEST_DIR | Full path to the mysql-test directory where tests are being run from |
MYSQL_TEST_LOGIN_FILE | Path name to login file used by mysql_config_editor. If not set, the default is $HOME/.mylogin.cnf, or %APPDATA%\MySQL\.mylogin.cnf on Windows. |
MYSQL_TMP_DIR | Path to temp directory used for temporary files during tests |
MYSQLD | Full path to server executable used in tests. |
MYSQLD_BOOTSTRAP | Full path name to mysqld that has all options enabled |
MYSQLD_BOOTSTRAP_CMD | Full command line used for initial database setup for this test batch |
MYSQLD_CMD | Command line for starting server as used in tests, with the minimum set of required arguments. |
MYSQLTEST_VARDIR | Path name to the var directory that is used for logs, temporary files, and so forth |
NUMBER_OF_CPUS | Defines number of processors. |
TSAN_OPTIONS | Path name to a file containing ThreadSanitizer suppressions. |
The variable MTR_PORT_BASE is a more logical replacement for the original variable MTR_BUILD_THREAD. It gives the actual port number directly (will be rounded down to a multiple of 10). If you use MTR_BUILD_THREAD, the port number is found by multiplying this by 10 and adding 10000.
The variable $MYSQL_OPTIONS contains only the options used while connecting to server from mysql client . whereas variable $MYSQL contains path to mysql client executable along with options .
Tests sometimes rely on certain environment variables being defined. For example, certain tests assume that MYSQL_TEST is defined so that mysqltest can invoke itself with exec $MYSQL_TEST.
Other tests may refer to the last three variables listed in the preceding table, to locate files to read or write. For example, tests that need to create files will typically put them in $MYSQL_TMP_DIR/file_name.
The variable $MYSQLD_CMD will include any server options added with the --mysqld option to mysql-test-run.pl, but will not include server options added specifically for the currently running test.
mysql-test-run.pl supports the options in the following list. An argument of – tells mysql-test-run.pl not to process any following arguments as options.
--help, -h
Display a help message and exit.
--accept-test-fail
Do not print an error and do not give exit 1 if some tests failed, but test run was completed. This also enables –force.
--big-test
Allow tests marked as "big" to run. Tests can be thus marked by including the line --source include/big_test.inc, and they will only be run if this option is given, or if the environment variable BIG_TEST is set to 1.
This is typically done for tests that take very long to run, or that use very much resources, so that they are not suitable for running as part of a normal test suite run.
If both --big-test and --only-big-tests are given, --only-big-tests is ignored.
--boot-dbx
Run the mysqld server used for bootstrapping the database through the dbx debugger.
--boot-ddd
Run the mysqld server used for bootstrapping the database through the ddd debugger.
--boot-gdb
Run the mysqld server used for bootstrapping the database through the gdb debugger.
See also the --manual-boot-gdb option.
--build-thread=number
Specify a number to calculate port numbers from. The formula is 10 * build_thread + 10000. Instead of a number, it can be set to auto, which is also the default value, in which case mysql-test-run.pl will allocate a number unique to this host.
The value (number or auto) can also be set with the MTR_BUILD_THREAD environment variable.
This option is kept for backward compatibility. The more logical --port-base is recommended instead.
--callgrind
Instructs valgrind to use callgrind.
--charset-for-testdb=charset_name
Specify the default character set for the test database. The default value is latin1.
--check-testcases
Check test cases for side effects. This is done by checking the system state before and after each test case. If there is any difference, the test case is marked as failed because of it.
Similarly, when --check-testcases option is enabled, MTR does additional check for missing .result file and a test case not having its corresponding .result file is marked as failed.
This check is enabled by default. To disable it, use the --nocheck-testcases option.
--clean-vardir
Clean up the var directory with logs and test results etc. after the test run, but only if there were no test failures. This option only has effect if also running with option --mem. The intent is to alleviate the problem of using up memory for test results, in cases where many different test runs are being done on the same host.
--client-bindir=path
The path to the directory where client binaries are located.
--client-dbx
Start mysqltest in the dbx debugger.
--client-ddd
Start mysqltest in the ddd debugger.
--client-debugger=debugger
Start mysqltest in the named debugger.
--client-gdb
Start mysqltest in the gdb debugger.
--client-libdir=path
The path to the directory where client libraries are located.
--colored-diff
Colorize the diff part of the output.
When this option is enabled, mysql-test-run.pl passes --colored-diff option to mysqltest. If colored diff is enabled, mysqltest uses diff command with --color='always' option to print the colored diff.
--color option for diff command is available from GNU diffutils version 3.4. --combination=value
Extra option to pass to mysqld. The value should consist of a single mysqld option including dashes. This option is similar to --mysqld but has a different effect. mysql-test-run.pl executes multiple test runs, using the options for each instance of --combination in successive runs. If --combination is given only once, it has no effect and behaves similar to --mysqld option.
If a test being run, passes the same --mysqld option from an opt file which is also passed from --combination, then that --combination run is skipped because opt file always takes higher precedence over command line options. The test will just run once using the options provided in the opt file.
For test runs specific to a given test suite, an alternative to the use of --combination is to create a combinations file in the suite directory. The file should contain a section of options for each test run. See Passing Options from mysql-test-run.pl to mysqld or mysqltest.
For test specific combinations, onc can use test_name.combinations file in the test directory.
--comment=str
Write str to the output within lines filled with #, as a form of banner.
--compress
Compress all information sent between the client and the server if both support compression.
--cursor-protocol
Pass the --cursor-protocol option to mysqltest (implies --ps-protocol).
--dbx
Start mysqld in the dbx debugger.
--ddd
Start mysqld in the ddd debugger.
--debug
Dump trace output for all clients and servers.
--debugger=debugger
Start mysqld using the named debugger.
--debug-common
This option works similar to --debug but turns on debug only for the debug macro keywords query, info, error, enter, exit which are considered the most commonly used.
--debug-server
Runs mysqld.debug (if available) instead of mysqld as server. If it does find mysqld.debug, it will search for plugin libraries in a subdirectory debug under the directory where it's normally located. This option does not turn on trace output and is independent of the debug option.
--debug-sync-timeout=seconds
Controls whether the Debug Sync facility for testing and debugging is enabled. The option value is a timeout in seconds. The default value is 300. A value of 0 disables Debug Sync. The value of this option also becomes the default timeout for individual synchronization points.
mysql-test-run.pl passes --loose-debug-sync-timeout=seconds to mysqld. The --loose prefix is used so that mysqld does not fail if Debug Sync is not compiled in.
For information about using the Debug Sync facility for testing, see Thread Synchronization in Test Cases.
--default-myisam
Use MyISAM as the default storage engine for all except InnoDB-specific tests. This option is off by default.
--defaults-file=file_name
Use the named file as fixed config file template for all tests.
--defaults_extra_file=file_name
Add setting from the named file to all generated configs.
--do-suite=prefix or regex
Run all test cases from suites having a name that begins with the given prefix value or matches the regular expression. If the argument matches no existing suite, mysql-test-run.pl aborts.
The argument for the --do-suite option allows more flexible specification of which tests to perform. See the description of the --do-test option for details.
In case a suite or a list of suites is specified using the --suite option along with a pattern or regular expression given through --do-suite, only suite(s) from the list which match the prefix or regular expression are picked up.
When the --do-suite option is specified more than once on the command line, only the last occurrence is considered.
--do-test=prefix or regex
Run all test cases having a name that begins with the given prefix value or matches the regular expression. This option provides a convenient way to run a family of similarly named tests.
The argument for the --do-test option allows more flexible specification of which tests to perform. If the argument contains a pattern metacharacter other than a lone period, it is interpreted as a Perl regular expression and applies to test names that match the pattern. If the argument contains a lone period or does not contain any pattern metacharacters, it is interpreted the same way as previously and matches test names that begin with the argument value. For example, --do-test=testa matches tests that begin with testa, --do-test=main.testa matches tests in the main test suite that begin with testa, and --do-test=main.*testa matches test names that contain main followed by testa with anything in between. In the latter case, the pattern match is not anchored to the beginning of the test name, so it also matches names such as xmainytestz.
--do-test-list=file
Run all tests listed in the file file. In this file, tests should be listed one per line in any of the following forms.
<test_name> <test_name>.test <suite_name>.<test_name> <suite_name>.<test_name>.test path_to_the_test_file
Path to the test file should either be an absolute path or a relative path from base directory.
To run a test present in a non-default test suite, either the test name qualified with the suite name i.e suite_name.test_name[.test] or a path to the test file should be mentioned.
A line beginning with # will be ignored and is considered as a comment. Similarly an empty line in the file is also ignored.
For example, consider a test list file test_list containing following tests.
# Main suite test '1st.test' 1st 1st.test main.1st main.1st.test mysql-test/t/1st.test # GIS suite test 'geohash_functions.test' gis.geohash_functions mysql-test/suite/gis/t/geohash_functions.test # Non-default suite 'test_services' test_services.test_services mysql-test/suite/test_services/t/test_services.test # Internal suite test 'i_main.user_var' i_main.user_var internal/mysql-test/suite/i_main/t/user_var.test
Following command runs the list of tests mentioned in the file.
./mysql-test-run.pl --do-test-list=test_list
If both --suite and --do-test-list are given, and if suite name is not part of the test names listed in the file, then those tests are searched only in the suites mentioned in --suite option.
--enable-disabled
Ignore any disabled.def file, and run also tests marked as disbaled. Success or failure of those tests will be reported the same way as other tests.
--explain-protocol
Run EXPLAIN EXTENDED on all SELECT, INSERT, REPLACE, UPDATE and DELETE queries.
--extern option=value
Use an already running server. The option/value pair is what is needed by the mysql client to connect to the server. Each --extern can only take one option/value pair as argument, so it you need more you need to repeat --extern for each of them.
./mysql-test-run.pl --extern socket=var/tmp/mysqld.1.sock main.1st
During server initialization, mysql-test-run.pl creates a default database test which is used while establishing an initial connection with the server. When running a test case with –extern option, if test database doesn't exist on an already running server, then the test case will fail with following error.
mysqltest: Could not open connection 'default': 1049 Unknown database 'test'
So if you're running a test case on an already running server using –extern option, it is necessary to create test database on the server before running the test case if it doesn't exist.
Similarly, mysql-test-run.pl creates mtr database during server initialization which contains tables and procedures used to suppress global or test specific warnings. If a test case refers to this database and the database doesn't exist on running server, then the test case will fail.
--fast
Do not perform controlled shutdown when servers need to be restarted or at the end of the test run. This is equivalent to using --shutdown-timeout=0.
--force
Normally, mysql-test-run.pl exits if a test case fails. --force causes execution to continue regardless of test case failure.
--force-restart
Always restart the server(s) between each tast case, whether it's needed or not. Will also restart between repeated runs of the same test case. This may be useful e.g. when looking for the source of a memory leak, as there will only have been one test run before the server exits.
Enabling this option will cause mysql-test-run.pl to restart the server(s) on a newly intialized data directory.
This option can also be mentioned in test-name-master.opt file for a test case to restart the server(s) before it runs.
--gcov
Run tests with the gcov test coverage tool.
--gdb
Start mysqld in the gdb debugger.
--gprof
Run tests with the gprof profiling tool.
--include-ndbcluster, --include-ndb
Run also tests that need Cluster.
--initialize=value
Extra bootstrap options that need be to be passed during the server initialization process can be passed as a value to --initialize. The server is then started with the specified value. Only one option may be specified in value; to specify more than one, use additional --initialize options.
Options passed as a value to --mysqld will have precedence over the options passed to --initialize, regardless of the order in which they passed on the command line.
--json-explain-protocol
Run EXPLAIN FORMAT=JSON on all SELECT, INSERT, REPLACE, UPDATE and DELETE queries.
--manual-boot-gdb
This option is similar to --boot-gdb but attaches the debugger to the server during the bootstrapping process, permitting the use of a remote debugger.
--manual-dbx
Use a server that has already been started by the user in the dbx debugger.
--manual-ddd
Use a server that has already been started by the user in the ddd debugger.
--manual-debug
Use a server that has already been started by the user in a debugger.
--manual-gdb
Use a server that has already been started by the user in the gdb debugger.
--mark-progress
Marks progress with timing (in milliseconds) and line number in var/log/testname.progress.
--max-connections=num
The maximum number of simultaneous server connections that may be used per test. If not set, the maximum is 128. Minimum allowed limit is 8, maximum is 5120. Corresponds to the same option for mysqltest.
--max-save-core=N
Limit the number of core files saved, to avoid filling up disks in case of a frequently crashing server. Defaults to 5, set to 0 for no limit. May also be set with the environment variable MTR_MAX_SAVE_CORE.
--max-save-datadir=N
Limit the number of data directories saved after failed tests, to avoid filling up disks in case of frequent failures. Defaults to 20, set to 0 for no limit. May also be set with the environment variable MTR_MAX_SAVE_DATADIR.
--max-test-fail=N
Stop execution after the specified number of tests have failed, to avoid using up resources (and time) in case of massive failures. retries are noe counted. Defaults to 10, set to 0 for no limit. May also be set with the environment variable MTR_MAX_TEST_FAIL.
--mem
This option is not supported on Windows and on MacOS.
Run the test suite in memory, using tmpfs or ramdisk. This can decrease test times significantly, in particular if you would otherwise be running over a remote file system. mysql-test-run.pl attempts to find a suitable location using a built-in list of standard locations for tmpfs and puts the var directory there. This option also affects placement of temporary files, which are created in var/tmp.
The default list includes /dev/shm, /run/shm and /tmp. You can also enable this option by setting the environment variable MTR_MEM[=dir_name]. If dir_name is given, it is added to the beginning of the list of locations to search, so it takes precedence over any built-in locations.
Once you have run tests with --mem within a mysql-test directory, a soflink var will have been set up to the temporary directory, and this will be re-used the next time, until the soflink is deleted. Thus, you do not have to repeat the --mem option next time.
--mysqld=value
Extra option to pass to mysqld. Only one option may be specified in value; to specify more than one, use additional –mysqld options. See Passing Options from mysql-test-run.pl to mysqld or mysqltest.
--mysqld-env=variable=value
Sets (or changes) an environment variable before starting mysqld. Varibles set in the environment from which you run mysql-test-run.pl will normally also be propagated to mysqld, but there may be cases where you want a setting just for a single run, or you may not want the setting to affect other programs. You may use additional --mysqld-env options to set more than one variable.
--mysqltest=options
Extra options to pass to mysqltest.
--ndb-connectstring=str
Pass --ndb-connectstring=str to the master MySQL server. This option also prevents mysql-test-run.pl from starting a cluster. It is assumed that there is already a cluster running to which the server can connect with the given connectstring.
--no-skip
This option forces all tests to run, ignoring any --skip commands used in the test. This ensures that all tests are run. An excluded list (excludenoskip.list) is maintained to track which tests should continue to be skipped. The --no-skip option continues to skip the tests that are named in the excluded list. In case the test or file invoking the --skip command is not present in the excluded list, the outcome of the test will be identified as [ noskip-pass ] or [ noskip-fail ] .The default value of –no-skip introduced variable is OFF, which implies users are not forced to run all tests unless the –no-skip is explicitly used.
shell> mysql-test-run.pl --suite=innodb --no-skip
--nocheck-testcases
Disable the check for test case side effects. For additional information, see the description of the --check-testcases option.
--noreorder
Do not reorder tests to reduce number of restarts, but run them in exactly the order given. If a whole suite is to be run, the tests are run in alphabetic order, though similiar combinations will be grouped together. If more than one suite is listed, the tests are run one suite at a time, in the order listed.
--notimer
Cause mysqltest not to generate a timing file. The effect of this is that the report from each test case does not include the timing in milliseconds as it normally does.
--nounit-tests
Do not run unit tests, overriding default behavior or setting of the MTR_UNIT_TESTS variable.
--nowarnings
Do not look for and report errors and warning in the server logs.
--nowarnings can also be passed in the .opt file of a test case. --only-big-tests
This option causes only big tests to run. Normal (non-big) tests are skipped. If both --big-test and --only-big-tests are given, --only-big-tests is ignored.
--opt-trace-protocol
Prints the optimizer trace of SQL statements in a test. Running any MTR test with this option executes the following SQL statement after every DML statement that it encounters:
SELECT trace FROM information_schema.optimizer_trace /* injected by --opt-trace-protocol */;
--parallel={N|auto}
Run tests using N parallel threads. By default, 1 thread is used. Use --parallel=auto to set N automatically.
Setting the MTR_PARALLEL environment variable to N has the same effect as specifying --parallel=N.
The MTR_MAX_PARALLEL environment variable, if set, specifies the maximum number of parallel workers that can be spawned when the --parallel=auto option is specified. If --parallel=auto is not specified, MTR_MAX_PARALLEL variable has no effect.
--port-base=P
Specify base of port numbers to be used; a block of 10 will be allocated. P should be divisible by 10; if it is not, it will be rounded down. If running with more than one parallel test thread, thread 2 will use the next block of 10 and so on.
If the port number is given as auto, which is also the default, mysql-test-run.pl will allocate a number unique to this host. The value may also be given with the environment variable MTR_PORT_BASE.
--port-base was added as a more logical alternative to --build-thread. If both are used, --port-base takes precedence.
--print-testcases
Do not run any tests, but print details about all tests, in the order they would have been run.
--ps-protocol
Pass the --ps-protocol option to mysqltest.
--quiet
Reuse the output buffer and maintain a single line for reporting successful tests, skipped tests and disabled tests. Failed tests and the necessary information about the failure will be printed on a separate line.
When --quiet option is enabled, MTR prints only the test name and result of the test run.
--record
Pass the --record option to mysqltest. This option requires a specific test case to be named on the command line.
--reorder
Reorder tests to minimize the number of server restarts needed. This is the default behavior. There is no guarantee that a particular set of tests will always end up in the same order.
--repeat=N
Run each test N number of times, in parallel if --parallel option value is greater than 1.
--report-features
Display the output of SHOW ENGINES and SHOW VARIABLES. This can be used to verify that binaries are built with all required features.
--report-times
At the end of the test run, write a summary of how much time was spent in various phases of execution. If you run with --parallel, the total will exceed the wall clock time passed, since it will be summed over all threads.
The times reported should only be treated as approximations, and the exact points where the time is taken may also change between releases. If the test run is aborted, including if a test fails and --force is not in use, the time report will not be produced.
--report-unstable-tests
The option can be used to distinguish unstable tests which pass on at least one retry attempt from hard failing ones. Unstable tests are reported separately in the end summary in the format: test_name(num_of_failures/num_of_attempts).
In case all failures encountered are due to unstable tests, MTR will print the below warning and exit with a zero status code.
mysql-test-run: WARNING: There are failures due to unstable test cases. However, the tests are not hard-failing.
This option has no effect unless --force is used and both --retry and --retry-failure are set to values greater than 1.
--retry=N
If a test fails, it is retried up to a maximum of N runs, but will terminate after 2 failures. Default is 3, set to 1 or 0 for no retries. This option has no effect unless --force is also used; without it, test execution will terminate after the first failure.
The --retry and --retry-failure options do not affect how many times a test repeated with --repeat may fail in total, as each repetition is considered a new test case, which may in turn be retried if it fails.
--retry-failure=N
Allow a failed and retried test to fail more than the default 2 times before giving it up. Setting it to 0 or 1 effectively turns off retries
--sanitize
Scan the server log files for warnings from various sanitizers. Use of this option assumes that MySQL was configured with -DWITH_ASAN or -DWITH_UBSAN.
The TSAN_OPTIONS environment variable can be set to specify the path name of a file containing ThreadSanitizer suppressions.
--shutdown-timeout=seconds
Max number of seconds to wait for servers to do controlled shutdown before killing them. Default is 10.
--skip-combinations
Do not apply combinations; ignore combinations file or option.
--skip-ndbcluster, --skip-ndb
Do not start NDB Cluster; skip Cluster test cases. This option only has effect if you do have NDB, if not it will have no effect as it cannot run those tests anyway.
--skip-ndbcluster-slave, --skip-ndb-slave
Do not start an NDB Cluster slave.
--skip-rpl
Skip replication test cases.
--skip-test=regex
Specify a regular expression to be applied to test case names. Cases with names that match the expression are skipped. tests to skip.
The argument for the --skip-test option allows more flexible specification of which tests to skip. If the argument contains a pattern metacharacter other than a lone period, it is interpreted as a Perl regular expression and applies to test names that match the pattern. See the description of the --do-test option for details.
--skip-test-list=file
Specify a file listing tests that should be skipped (disabled).
The file has the same format as the disabled.def file listing disabled tests. With this option, disabling can be done on a case by case basis.
--skip-*
--skip-* options not otherwise recognized by mysql-test-run.pl are passed to the master server.
--sp-protocol
Pass the --sp-protocol option to mysqltest.
--start
Initialize and start servers with the startup settings for the specified test case. You can use this option to start a server to which you can connect later. For example, after building a source distribution you can start a server and connect to it with the mysql client like this:
shell> cd mysql-test shell> ./mysql-test-run.pl --start alias & shell> ../mysql -S ./var/tmp/master.sock -h localhost -u root
If no tests are named on the command line, the server(s) will be started with settings for the first test that would have been run without the --start option.
mysql-test-run.pl will stop once the server has been started, but will terminate if the server dies. If killed, it will also shut down the server.
--start-and-exit
This is similar to --start, but mysql-test-run.pl terminates once the server has been started, leaving just the server process running.
--start-dirty
This is similar to --start, but will skip the database initialization phase and assume that database files are already available. Usually this means you must have run another test first.
--start-from=test_name
mysql-test-run.pl sorts the list of names of the test cases to be run, and then begins with test_name.
--strace-client
Create strace output for mysqltest. Will produce default strace output as mysqltest.strace. Note that this will be overwritten for each new test case, so it's most useful for running only one test.
The strace-client option is only supported on Linux.
--strace-server
Create strace output for the server. Will produce default strace output as mysqld.1.strace. Note that this will be overwritten each time the server is restarted, so it's most useful for running a single test, or if you want trace from the first test that fails.
The strace-server option is available on Linux only.
--stress=stress options
Start a server, but instead of running a test, run mysql-stress-test.pl with the supplied arguments. Arguments needed to communicate with the server will be automatically provided, the rest should be given as arguments to this option. Command line options for mysql-stress-test.pl should be separeted by a comma.
--suite(s)={suite_name|suite_list|suite_set}
Run a suite or a comma separated list of suites or those suites which fall under the specified suite set. If the option is not specified explicitly, MTR assumes that it needs to use the default suite set for the test run.
The suite set's MTR can use are case insensitive and take the below values:
default - To run the default list of suites.
all - To scan the mysql directory and run all available suites.
non-default - To scan the mysql directory for available suites and
collect only the non-default suites.The suite set 'all' is used implicitly if neither a suite nor a suite list is specified and
(i) There are tests given on the MTR command line or
(ii) Any of the following options are used: –do-suite, –do-test, –do-test-list.
Note that suite sets 'all' and 'non-default' do not include ndb test suites in case ndb cluster is not enabled.
If a suite set is specified in the suite list, MTR does not treat it as a suite set and considers it as the name of a suite to be run.
--suite-timeout=minutes
Specify the maximum test suite runtime in minutes.
--summary-report=file_name
Generate a plain text version of the test summary only and write it to the file named as the option argument. The file is suitable for sending by email.
--test-progress[={0|1}]
Print the percentage of tests completed. This option is enabled by default. The argument to --test-progress is optional. To disable it, set the value to 0.
For example, consider a test suite sample_test_suite containing 3 tests.
./mysql-test-run.pl --suite=sample_test_suite [--test-progress]
Running the suite with --test-progress option enabled will print the percentage of tests completed.
[ 33%] sample_test_suite.1st [ pass ] 1 [ 66%] sample_test_suite.2nd [ pass ] 1 [100%] sample_test_suite.3rd [ pass ] 1
--testcase-timeout=minutes
Specify the maximum test case runtime in minutes.
--timediff
Adds to each test report for a test case, the total time in seconds and milliseconds passed since the preceding test ended. This option can only be used together with --timestamp, and has no effect without it.
--timer
Cause mysqltest to generate a timing file. The default file is named ./var/log/timer.
--timestamp
Prints a timestamp before the test case name in each test report line, showing when the test ended.
--tmpdir=path
The directory where temporary file are stored. The default location is ./var/tmp. The environment variable MYSQL_TMP_DIR will be set to the path for this directory, whether it has the default value or has been set explicitly. This may be referred to in tests.
--unit-tests
Force running of unit tests, overriding default behavior or setting of the MTR_UNIT_TESTS variable.
--unit-tests-report
Extend the unit test run by also outputting the log from the test run, independently of whether it succeeded or not. This option implies --unit-tests so it is not necessary to specify both.
--user=user_name
The MySQL user name to use when connecting to the server.
--user-args
Drops all non-essential command line arguments to the mysqld server, except those supplied with --mysqld arguemnts, if any. Only works in combination with --start, --start-and-exit or --start-dirty, and only if no test name is given.
--valgrind
Run mysqltest and mysqld with valgrind. This and the following --valgrind options require that the executables have been build with valgrind support.
When the server is run with valgrind, an extra pass over the server log file(s) will be performed after all tests are run, and any report with problems that have been reported at server shutdown will be extracted and printed. The most common warnings are memory leaks. With each report will also be listed all tests that were run since previous server restart; one of these is likely to have caused the problem.
A final "pseudo" test named valgrind_report is added to the list of tests when the server is run in valgrind. This test is reported as failed if any such shutdown warnings were produced by valgrind. Pass or failure of this test is also added to the total test count reported.
--valgrind-clients
Run all clients started by .test files with valgrind. This option requires valgrind 3.9 or later.
--valgrind-mysqld
Run the mysqld server with valgrind.
--valgrind-mysqltest
Run mysqltest with valgrind.
--valgrind-option=str
Extra options to pass to valgrind.
--valgrind-path=path
Specify the path name to the valgrind executable.
--vardir=path
Specify the path where files generated during the test run are stored. The default location is ./var. The environment variable MYSQLTEST_VARDIR will be set to the path for this directory, whether it has the default value or has been set explicitly. This may be referred to in tests.
--verbose
Give more verbose output regarding test execution. Note that the output generated within each test case is not affected.
--verbose-restart
Write when and why servers are restarted between test cases.
--view-protocol
Pass the --view-protocol option to mysqltest.
--vs-config=config_val
Specify the configuration used to build MySQL (for example, –vs-config=debug –vs-config=release). This option is for Windows only.
--wait-all
If --start or --start-dirty is used, wait for all servers to exit before termination. Otherise, it will terminate if one (of several) servers is restarted.
--warnings
Search the server log for errors or warning after each test and report any suspicious ones; if any are found, the test will be marked as failed. This is the default behavior, it may be turned off with --nowarnings.
--with-ndbcluster-only, --with-ndb-only
Run only ndb tests. If no suites are explicitly given, this option also skip all non ndb suites without checking individual test names.
--xml-report=file_name
Generate an xml file containing result of the test run and write it to the file named as the option argument. It includes suite-wise and overall statistics like the number of tests run, number of failures, number of disabled and skipped tests along with information about each individual test according to its outcome.
The xml report's structure can be found below :
<testsuites ... >
<testsuite ... >
<testcase ... />
<testcase ... >
<failure ... > ... </failure>
</testcase>
</testsuite>
</testsuites> 
1.9.2