This documentation is for an older version. If you're using the most current version, select the documentation for that version with the version switch in the upper right corner of the online documentation, or by downloading a newer PDF or EPUB file.

3 Feature Testing

New features in MySQL are developed and tested in separate feature trees before they are pushed to TRUNK. Quality goals for new features are the following:

• Complete functional and nonfunctional test coverage of changed and new functionality

• No regressions

• At least more than 80% code coverage

QA involvement starts as soon as the requirements and specifications of the feature are finalized by the development team. QA reviews available documents and provides feedback on the design, usability, testability, etc. A discussion follows with the developer and changes are made as needed to ensure that the feature can be tested.

Once the specifications and requirements are acceptable, QA creates the test plan which documents all scenarios that are to be tested. This includes standalone tests, integration tests, nonfunctional tests, etc. The test plan is reviewed by developers and peer QA team members.

While the developers are writing the product code, QA engineers start working on the automated tests, test infrastructure improvements, etc. The final round of testing starts after the feature has passed code reviews. This phase can last anywhere between a few days to months, depending on the complexity of the feature, stability of the code, number of bugs found, etc.

Features get signed off when the following conditions are met:

• No known bugs in the new feature – This is very strictly enforced and even minor bugs are not allowed. We believe that bugs are easiest to fix when the code is new, and hence this can help us deliver features that are of high quality.

• No known regressions – A feature gets developed on a tree which gets tested regularly through a continuous integration testing tool. Any regressions are detected and fixed before signoff.

• Acceptable code coverage numbers – A code coverage report is generated for the changed lines of code and the minimum expected coverage is 80%. Most features have a coverage of more than 90%. Any uncovered lines of code are analyzed and, wherever possible, new tests are added to increase code coverage.

• All new tests are added to the automated regression suite.