TestcaseWikiMigration
5510
Comment:
|
5124
|
Deletions are marked like this. | Additions are marked like this. |
Line 35: | Line 35: |
* If it is a test case for an application, then the testcase number will be [a-z]{2}-[0-9]{3} (# 2 letters for the application and 3 numbers for the test case number) | * If it is a test case for an application, then the testcase number will be [a-z]{2}-[0-9]{3} (2 letters for the application and 3 numbers for the test case number) |
Line 43: | Line 43: |
The testcases from pulled from launchpad bugs exists under http://testcases.qa.ubuntu.com/Regressions/ with per-data type subtrees, but is open to renaming or restructuring. |
|
Line 54: | Line 56: |
Should cover changes required to the UI, or specific UI that is required to implement this | It would be really useful to have a way to add new test cases easily. A new form with the necessary information as text boxes would be the best approach. |
Line 58: | Line 60: |
Code changes should include an overview of what needs to change, and in some cases even the specific details. | TBA |
Line 64: | Line 66: |
== Test/Demo Plan == It's important that we are able to test new features, and demonstrate them to users. Use this section to describe a short plan that anybody can follow that demonstrates the feature is working. This can then be used during testing, and to show off after release. Please add an entry to http://testcases.qa.ubuntu.com/Coverage/NewFeatures for tracking test coverage. This need not be added or completed until the specification is nearing beta. |
|
Line 72: | Line 68: |
This should highlight any issues that should be addressed in further specifications, and not problems with the specification itself; since any specification with problems cannot be approved. | In the discussion we talked a bit about integration and communication with the ISO tracker, but this was a light talk and nothing was committed. |
Launchpad Entry: qa-testscase-wiki
Created: 2008-12-15
Contributors: heno, schwuk, apulido, sbeattie
Packages affected:
Summary
Migrating all test cases to testcases.qa.ubuntu.com, adding useful macros and making sure all relevant teams are using the new facility.
Rationale
We need a consistent syntax to define the test cases and a better way to track them. Right now, test cases are maintained under https://wiki.ubuntu.com/Testing/Cases but the syntax is not defined and there is no logic way to track what it is available.
Use Cases
Mark, a netbook-remix user, wants to add a new test case for an application that it is already available for Ubuntu desktop. He uses the already available test case for Ubuntu, needing only to upload the netbook-remix specific screenshots.
Anne, a Kubuntu user, finds a bug in Kpdf while reading a complex PDF file. She creates a LP bug but also creates a new test case to track future regressions. This file will be added also as test case for Evince in Ubuntu.
Design
Testcases Syntax
Lobbing for a respect in the syntax through how-tos or tutorials is not enough; syntax should be more or less forced by the wiki itself.
Possible solutions
- Create a new form to add new test cases, that allows to add steps and screenshot
- A test case template to be copied and edited when adding a new test case
In both cases a plugin should then add the text to the specific application page and track the testcase number.
Testcase Number
After much discussion it has been agread that we will use the following syntax to track the test cases number as:
- If it is a test case for an application, then the testcase number will be [a-z]{2}-[0-9]{3} (2 letters for the application and 3 numbers for the test case number)
- If it is a test case for a Launchpad bug then the testcase number will be lp-[bugnumber]
Path Structure
We have decided to move the pages from /Distribution/Application to just application. Therefore, in example, Ekiga page will be at http://testcases.qa.ubuntu.com/Applications/Ekiga instead of http://testcases.qa.ubuntu.com/Ubuntu/Applications/Ekiga.
A Moin Moin plugin will be needed to change to the proper screenshots when called with /Ubuntu, /Kubuntu, /Xubuntu, etc.
The testcases from pulled from launchpad bugs exists under http://testcases.qa.ubuntu.com/Regressions/ with per-data type subtrees, but is open to renaming or restructuring.
Tracking results
An improvement, once the rest of the changes are in place, would be an easyway to maintain a table of results of test runs.
Implementation
- A new LP project needs to be created to track bugs and code for the custom changes (plugins, themes and macros).
UI Changes
It would be really useful to have a way to add new test cases easily. A new form with the necessary information as text boxes would be the best approach.
Code Changes
TBA
Migration
- After migration all the old test cases wiki pages need to have just an advice pointing to the new test cases wiki.
Unresolved issues
In the discussion we talked a bit about integration and communication with the ISO tracker, but this was a light talk and nothing was committed.
BoF agenda and discussion
Testcases in wiki.ubuntu.com: https://wiki.ubuntu.com/Testing/Cases
New testcases specific wiki at http://testcases.qa.ubuntu.com
- Syntax of testcases - Way to add attached documents or additional files to the test case.
- https://testcases.qa.ubuntu/Distro/Applications/Application seems a natural way to order the wiki - We could have a macro that pulls the correct screenshots for a test cases depending on the distributuion:
- i.e. We cave testcases.qa.ubuntu.com/Applications/Gedit, but when called from /Xubuntu/Applications/Gedit it will contain Xubuntu screenshots if available.
- Syntax of the track number - Right now is going u-ff-001 (u for ubuntu, ff for the app (in this case firefox), and 3 number for the test case numbers) - Change that to ff-001 (2 letters for the application and 3 numbers for the test case number)
- How to use macros to encourage people to contribute in a easier
OEM/UNR Testcase format example: https://wiki.ubuntu.com/Testing/Cases/Ubuntu-Netbook-Remix
How to track test cases run?
- - Chris maintains a table with results and then use a python script to parse it and convert it if necessary. - [Chris] We need to use testcases.qa.ubuntu.com to track results of tests. Maybe with macros. (in a ISO tracker manner) - ISO tracker and testcases.qa.ubuntu.com communication
Are testcases wiki tasks handled in LP? Do we have a project to file bugs?
- - Dave to set up a LP project to file bugs and have the code in bzr
Meta Data for case suite creation - Meta codes for each case:
- - Distro(s) - Version(s)
Code format - ff-001 = Ubuntu Jaunty, Firefox, Case 1
QATeam/Specs/TestcaseWikiMigration (last edited 2009-01-13 10:54:46 by cpc4-oxfd8-0-0-cust39)