Please think twice before you create
engine-specific variants of a test
or
let runs with ps-protocol/sp-protocol/cursor-protocol/view-protocol
happen. They might be
of low value and a permanent wasting of resources and/or
fail (protocol variants only)
Checks of the INFORMATION_SCHEMA
The storage engines to be used for information_schema tables are hardcoded. Therefore tests focussed on permissions, optimizer strategies, column data types etc. when selecting on INFORMATION_SCHEMA tables should not run with storage engine variations.
Please prepare the scripts at least for use with different storage engines.
-
That means mostly:
DO NOT use hardcoded storage engine assignments within CREATE TABLE statements.
If you assume that there is no significant impact of storage engines on your testing object,
create/run with storage engine variants and check this.
Solution 1:
Do not assign the storage engine within your CREATE TABLE statements at all. The default storage engine MyISAM will be used for your tables. Check your assumption with:
./myql-test-run.pl --mysqld="--default-storage-engine=<engine>" <test_case>
Solution 2:
Assign the storage engine to be used via $variable.
Top level script:
let $engine_type= MyISAM;
The same script or a sourced script:
eval CREATE TABLE .... ENGINE = $engine_type ...
Check your assumption by creating and running storage engine variants of the top level test.
--source include/have_falcon.inc let $engine_type= Falcon;
It is usual to run tests with and without the mysql-test-run.pl startup option "--ps-protocol". And there are also attempts of System QA to run the other protocols.
Effect of the "--ps-protocol" option: mysqltest will run as many SQL statements as possible as prepared statements.
Effect of the "--sp-protocol"/"--cursor-protocol"/"--view-protocol" options: As far as I know, mysqltest takes many statements and transforms them into a statement sequence checking the corresponding feature (stored procedures, cursors, or views).
We have a lot of tests running very similar and extremely simple SQL just for the creation of a situation to be tested, check of table content, etc. So it can be assumed that the n'th test running again simple statements does not improve the coverage.
Conclusion:
If your test contains
Simple statements: Prevent the non valuable runs for "protocols".
Storage engine variations: Prevent the protocol runs for all except one storage engine (my take would be MyISAM).
-
"Unique" and complex statements:
ps-protocol statements = all SQL
sp-protocol statements = DML
view/cursor-protocols statements = SELECTs
Do not prevent the protocol runs.
Another problem around runs with such protocols is that there are cases where we get different protocol content.
Example:
Script:
SELECT * FROM processlist ...
Protocol content if running without any "--*-protocol":
ID USER ... COMMAND ... STATE INFO 1 root ... Query ... preparing SELECT * FROM processlist ...
Protocol content if running with "--ps-protocol":
ID USER ... COMMAND ... STATE INFO 1 root ... Execute ... preparing SELECT * FROM processlist ...
So please check whether every new test gives the same result with every protocol but at least with "--ps-protocol". If not, do something of the following:
Exclude some protocol variants from execution.
Disable the the use of the protocols for problematic statements.
Write protocol variant specific tests.
Example solution (code within the top level scripts):
# The file with expected results fits only to a run without
if (`SELECT $PS_PROTOCOL + $SP_PROTOCOL
+ $CURSOR_PROTOCOL + $VIEW_PROTOCOL > 0`)
{
--skip Test requires: ps-protocol/sp-protocol/cursor-protocol/view-protocol disabled
}
--source include/<whatever>.inc
or
# The file with expected results fits only to a run with "--ps-protocol".
if (`SELECT $SP_PROTOCOL + $CURSOR_PROTOCOL + $VIEW_PROTOCOL > 0
OR $PS_PROTOCOL = 0`)
{
--skip Test requires: ps-protocol enabled, other protocols disabled
}
--source include/<whatever>.inc
