Launchpad Entry: qa-testscase-wiki
Created: 2008-12-15
Contributors: heno, schwuk, apulido, sbeattie
Packages affected:
Summary
Migrating all test cases to testcases.qa.ubuntu.com, adding useful macros and making sure all relevant teams are using the new facility.
Rationale
We need a consistent syntax to define the test cases and a better way to track them. Right now, test cases are maintained under https://wiki.ubuntu.com/Testing/Cases but the syntax is not defined and there is no logical way to track what it is available.
Use Cases
- Mark, a netbook-remix user, wants to add a new test case for an application that already has a test case for Ubuntu desktop. He calls the already available test case from the UNR page, needing only to upload the netbook-remix specific screenshots.
- Anne, a Kubuntu user, finds a bug in Kpdf while reading a complex PDF file. She creates a LP bug but also creates a new test case to track future regressions. This file will be added also as test case for Evince in Ubuntu.
- Chris wishes to track the testing for a mobile build. He can do that in the wiki by using a macro
Design
Testcases Syntax
A standard test case template will be drawn up which will be used by all test cases. Application Template The syntax will not be automatically checked, at this point it is enough to follow the template and reedit them manually if errors are found.
Metadata
- Distro(s) - for which distro flavours is the case valid?
- Version(s) - for which distro versions is the case valid?
- Application name
- Attachments required for the test case should be stored in the wiki and linked explicitly
Testcase Number
After much discussion it has been agreed that we will use the following syntax to track the test cases number as:
- If it is a test case for an application, then the testcase number will be [a-z]{2}-[0-9]{3} (2 letters for the application and 3 numbers for the test case number)
- If it is a test case for a Launchpad bug then the testcase number will be lp-[bugnumber]
The testcase number will be checked for syntax and a warning will be raised in these cases:
- No track number has been added
- The track number is already used by other testcase.
Path Structure
We have decided to move the pages from /Distribution/Applications to just /Applications. For example, the Ekiga page will be at http://testcases.qa.ubuntu.com/Applications/Ekiga instead of http://testcases.qa.ubuntu.com/Ubuntu/Applications/Ekiga.
A Moin Moin plugin will be needed to change to the proper screenshots when called with /Ubuntu, /Kubuntu, /Xubuntu, etc.
The testcases from pulled from launchpad bugs exists under http://testcases.qa.ubuntu.com/Regressions/ with per-data type subtrees, but is open to renaming or restructuring.
Tracking results
A test tracking facility in a separate page tree but in the same wiki will make it easy to track test results linked to the cases.
Use a standard page space such as /TestRuns/<identifier> to collect results
- Write a macro to list the tests used (and optionally display the full case text) and collect the results through simple input boxes
Implementation
Admin: A new LP project needs to be created to track bugs and code for the custom changes (plugins, themes and macros).
Displaying case numbers - A Macro for collecting and displaying case numbers and titles in table form
Test results - A Macro based on PageComment2 to display case information with result entry boxes and result tables. The actual results are stored in a sub-page (as the comment macro does).
Migration
- After migration all the old test cases wiki pages need to have just an advice pointing to the new test cases wiki.
BoF agenda and discussion
merged into spec