TestcaseWikiMigration
Launchpad Entry: qa-testscase-wiki
Created: 2008-12-15
Contributors: heno, schwuk, apulido, sbeattie
Packages affected:
Summary
Migrating all test cases to testcases.qa.ubuntu.com, adding useful macros and making sure all relevant teams are using the new facility.
Rationale
We need a consistent syntax to define the test cases and a better way to track them. Right now, test cases are maintained under https://wiki.ubuntu.com/Testing/Cases but the syntax is not defined and there is no logical way to track what it is available.
Use Cases
- Mark, a netbook-remix user, wants to add a new test case for an application that already has a test case for Ubuntu desktop. He calls the already available test case from the UNR page, needing only to upload the netbook-remix specific screenshots.
- Anne, a Kubuntu user, finds a bug in Kpdf while reading a complex PDF file. She creates a LP bug but also creates a new test case to track future regressions. This file will be added also as test case for Evince in Ubuntu.
Design
Testcases Syntax
A standard test case template will be drawn up which will be used by all test cases. Template should appear in the spec. - heno
- Code will be added to the wiki theme or a plug-in to parse the page and validate that the syntax follows the correct standard.
- A new-test-case macro will be written to easily add cases with the correct syntax
Lobbing for a respect in the syntax through how-tos or tutorials is not enough; syntax should be more or less forced by the wiki itself.
plugin should then add the text to the specific application page and track the testcase number. -- how does this work? - heno
Testcase Number
After much discussion it has been agreed that we will use the following syntax to track the test cases number as:
- If it is a test case for an application, then the testcase number will be [a-z]{2}-[0-9]{3} (2 letters for the application and 3 numbers for the test case number)
- If it is a test case for a Launchpad bug then the testcase number will be lp-[bugnumber]
Path Structure
We have decided to move the pages from /Distribution/Applications to just /Applications. For example, the Ekiga page will be at http://testcases.qa.ubuntu.com/Applications/Ekiga instead of http://testcases.qa.ubuntu.com/Ubuntu/Applications/Ekiga.
A Moin Moin plugin will be needed to change to the proper screenshots when called with /Ubuntu, /Kubuntu, /Xubuntu, etc.
The testcases from pulled from launchpad bugs exists under http://testcases.qa.ubuntu.com/Regressions/ with per-data type subtrees, but is open to renaming or restructuring.
Tracking results
An improvement, once the rest of the changes are in place, would be an easy way to maintain a table of results of test runs.
Implementation
Admin: A new LP project needs to be created to track bugs and code for the custom changes (plugins, themes and macros).
Syntax validation plug-in - A Parser or Action is required for test case syntax validation We need a starting reference for this - heno
Form for adding test cases Write a plug-in or macro to easily add test cases based on the NewPage or PageComment2 macros.
Macro for collecting and displaying case numbers and titles in table form We need a starting reference for this - heno
Migration
- After migration all the old test cases wiki pages need to have just an advice pointing to the new test cases wiki.
BoF agenda and discussion
Testcases in wiki.ubuntu.com: https://wiki.ubuntu.com/Testing/Cases
New testcases specific wiki at http://testcases.qa.ubuntu.com
- Syntax of testcases - Way to add attached documents or additional files to the test case.
How to track test cases run?
- - Chris maintains a table with results and then use a python script to parse it and convert it if necessary. - [Chris] We need to use testcases.qa.ubuntu.com to track results of tests. Maybe with macros. (in a ISO tracker manner)
Meta Data for case suite creation - Meta codes for each case:
- - Distro(s) - Version(s)
Code format - ff-001 = Ubuntu Jaunty, Firefox, Case 1