TestcaseWikiMigration
Launchpad Entry: qa-testscase-wiki
Created: 2008-12-15
Contributors: heno, schwuk, apulido, sbeattie
Packages affected:
Summary
Migrating all test cases to testcases.qa.ubuntu.com, adding useful macros and making sure all relevant teams are using the new facility.
Rationale
We need a consistent syntax to define the test cases and a better way to track them. Right now, test cases are maintained under https://wiki.ubuntu.com/Testing/Cases but the syntax is not defined and there is no logic way to track what it is available.
Use Cases
Mark, a netbook-remix user, wants to add a new test case for an application that it is already available for Ubuntu desktop. He uses the already available test case for Ubuntu, needing only to upload the netbook-remix specific screenshots.
Anne, a Kubuntu user, finds a bug in Kpdf while reading a complex PDF file. She creates a LP bug but also creates a new test case to track future regressions. This file will be added also as test case for Evince in Ubuntu.
Design
Testcases Syntax
Lobbing for a respect in the syntax through how-tos or tutorials is not enough; syntax should be more or less forced by the wiki itself.
Possible solutions
- Create a new form to add new test cases, that allows to add steps and screenshot
- A test case template to be copied and edited when adding a new test case
In both cases a plugin should then add the text to the specific application page and track the testcase number.
Testcase Number
After much discussion it has been agread that we will use the following syntax to track the test cases number as:
- If it is a test case for an application, then the testcase number will be [a-z]{2}-[0-9]{3} (# 2 letters for the application and 3 numbers for the test case number)
- If it is a test case for a Launchpad bug then the testcase number will be lp-[bugnumber]
Path Structure
We have decided to move the pages from /Distribution/Application to just application. Therefore, in example, Ekiga page will be at http://testcases.qa.ubuntu.com/Applications/Ekiga instead of http://testcases.qa.ubuntu.com/Ubuntu/Applications/Ekiga.
A Moin Moin plugin will be needed to change to the proper screenshots when called with /Ubuntu, /Kubuntu, /Xubuntu, etc.
Tracking results
An improvement, once the rest of the changes are in place, would be an easyway to maintain a table of results of test runs.
Implementation
- A new LP project needs to be created to track bugs and code for the custom changes (plugins, themes and macros).
UI Changes
Should cover changes required to the UI, or specific UI that is required to implement this
Code Changes
Code changes should include an overview of what needs to change, and in some cases even the specific details.
Migration
- After migration all the old test cases wiki pages need to have just an advice pointing to the new test cases wiki.
Test/Demo Plan
It's important that we are able to test new features, and demonstrate them to users. Use this section to describe a short plan that anybody can follow that demonstrates the feature is working. This can then be used during testing, and to show off after release. Please add an entry to http://testcases.qa.ubuntu.com/Coverage/NewFeatures for tracking test coverage.
This need not be added or completed until the specification is nearing beta.
Unresolved issues
This should highlight any issues that should be addressed in further specifications, and not problems with the specification itself; since any specification with problems cannot be approved.
BoF agenda and discussion
Testcases in wiki.ubuntu.com: https://wiki.ubuntu.com/Testing/Cases
New testcases specific wiki at http://testcases.qa.ubuntu.com
- Syntax of testcases - Way to add attached documents or additional files to the test case.
- https://testcases.qa.ubuntu/Distro/Applications/Application seems a natural way to order the wiki - We could have a macro that pulls the correct screenshots for a test cases depending on the distributuion:
- i.e. We cave testcases.qa.ubuntu.com/Applications/Gedit, but when called from /Xubuntu/Applications/Gedit it will contain Xubuntu screenshots if available.
- Syntax of the track number - Right now is going u-ff-001 (u for ubuntu, ff for the app (in this case firefox), and 3 number for the test case numbers) - Change that to ff-001 (2 letters for the application and 3 numbers for the test case number)
- How to use macros to encourage people to contribute in a easier
OEM/UNR Testcase format example: https://wiki.ubuntu.com/Testing/Cases/Ubuntu-Netbook-Remix
How to track test cases run?
- - Chris maintains a table with results and then use a python script to parse it and convert it if necessary. - [Chris] We need to use testcases.qa.ubuntu.com to track results of tests. Maybe with macros. (in a ISO tracker manner) - ISO tracker and testcases.qa.ubuntu.com communication
Are testcases wiki tasks handled in LP? Do we have a project to file bugs?
- - Dave to set up a LP project to file bugs and have the code in bzr
Meta Data for case suite creation - Meta codes for each case:
- - Distro(s) - Version(s)
Code format - ff-001 = Ubuntu Jaunty, Firefox, Case 1