MySQL 8.4.0
Source Code Documentation
How to Report Bugs from Tests in the MySQL Test Suite

If test cases from the test suite fail, you should do the following:

  • Do not file a bug report before you have found out as much as possible about what when wrong. See the instructions at https://dev.mysql.com/doc/refman/8.0/en/bug-reports.html.

  • Make sure to include the output of mysql-test-run.pl, as well as contents of all .reject files in the mysql-test/var/log directory, or the (often much shorter) diff that mysql-test-run.pl reported.

  • Check whether an individual test in the test suite also fails when run on its own:

    shell> cd mysql-test
    shell> ./mysql-test-run.pl test_name

    If this fails, and you are compiling MySQL yourself, you should configure MySQL with --with-debug and run mysql-test-run.pl with the --debug option. If this also fails, open a bug report and upload the trace file mysql-test/var/tmp/master.trace to the report, so that we can examine it. For instructions, see How to Report Bugs or Problems. Please remember to also include a full description of your system, the version of the mysqld binary and how you compiled it.

  • Run mysql-test-run.pl with the --force option to see whether any other tests fail.

  • If you have compiled MySQL yourself, check the MySQL Reference Manual to see whether there are any platform-specific issues for your system. There might be configuration workarounds to deal with the problems that you observe. Also, consider using one of the binaries we have compiled for you at https://dev.mysql.com/downloads/. All our standard binaries should pass the test suite!

  • If you get an error such as Result length mismatch or Result content mismatch it means that the output of the test was not an exact match for the expected output. This could be a bug in MySQL or it could be that your version of mysqld produces slightly different results under some circumstances.

    This output from mysql-test-run.pl should include a diff of the expected and actual result. If unable to produce a diff, it will instead print out both in full. In addition, the actual result is put in a file in the r directory with a .result extension.

  • If a test fails completely, you should check the logs file in the mysql-test/var/log directory for hints of what went wrong.

  • If you have compiled MySQL with debugging, you can try to debug test failures by running mysql-test-run.pl with either or both of the --gdb and --debug options.

    If you have not compiled MySQL for debugging you should probably do so by specifying the -DWITH_DEBUG option when you invoke CMake.