WL#3743: make unit-tests differentiation by time/disk space which they need for execution
Affects: Server-7.x
—
Status: Un-Assigned
To be able to run short variant of unit test (by default) and whole tests: i.e. add parameters to the unit-test running shell: --small (default) / --large (disk space) --fast (default) / --slow (time of execution) (Other parameters like multithread & Co) The shell will send the parameters to the test applications (which can be ignored by fast/ small test suites) and increase timeout for "slow" tests. The test applications get parameters via environment variables and command line: - command line parameters preceed - even if the test do not process command line environment variable will be processed by plan() call The test application can skip all or some of the tests or modify its behaviour in other way which correlates with required parameters (for example reduce file/page/ record size or reduce number of iterations). Simple fast tests can ignore parameters at all.
Eventually we may want to have this differentiation a. needs a lot of disk space .... doesn't b. keeps CPU busy ... doesn't c. keeps *all* CPU busy (e.g. my_atomic-t) ... doesn't d. intensive i/o ... no e. takes a lot of time ... doesn't One may want to skip a-tests (need a lot of disk space) if there's not enough disk space. b-tests can be run in parallel, c-tests cannot. e-tests can be skipped in autopush, for example. And so on. Also some tests may want to accept command-line arguments, so it should be allowed. But if a test does not use them - it should not be required to care/know about argc/argv. This is the framework that allows it: 1. There will be command-line arguments that a test can accept and use any way it wants. It's up to the test itself. 2. There will be also command-line arguments that are common to all tests - arguments that mytap library recognizes. 3. They will allow to specify what subset of tests (disk/cpu/ram/io/etc intensive) we want to exclude from a test run. 4. A test is supposed to recognize these arguments (during the parsing of its own test-specific arguments) and setup configuration variables accordingly. But mytap library may provide a helper functions for that. 5. For tests that don't parse command-line arguments there will also be MYTAP_CONFIG environment variable - which will have command-line options (similar to, say, LESS variable) 6. It'll be read and parsed automatically from plan() function that any test program should call in the beginning anyway. Thus existing tests won't need to be modified. As a preliminary step, --no-big command-line option was added to unit.pl. If used, it'll cause skip_big_tests variable _in the test program_ to be true. The test program can do the following: An example from my_atomic-t.c - decrease the number of iterations: if (skip_big_tests) CYCLES/=100; An example from mf_pagecache_single.c - skip "big" tests completely: SKIP_BIG_TESTS(4) { if (!simple_big_test()) exit(1); }
Copyright (c) 2000, 2024, Oracle Corporation and/or its affiliates. All rights reserved.