TestcaseWikiMigration
5124
Comment:
|
5304
|
Deletions are marked like this. | Additions are marked like this. |
Line 12: | Line 12: |
We need a consistent syntax to define the test cases and a better way to track them. Right now, test cases are maintained under https://wiki.ubuntu.com/Testing/Cases but the syntax is not defined and there is no logic way to track what it is available. | We need a consistent syntax to define the test cases and a better way to track them. Right now, test cases are maintained under https://wiki.ubuntu.com/Testing/Cases but the syntax is not defined and there is no logical way to track what it is available. |
Line 16: | Line 16: |
Mark, a netbook-remix user, wants to add a new test case for an application that it is already available for Ubuntu desktop. He uses the already available test case for Ubuntu, needing only to upload the netbook-remix specific screenshots. Anne, a Kubuntu user, finds a bug in Kpdf while reading a complex PDF file. She creates a LP bug but also creates a new test case to track future regressions. This file will be added also as test case for Evince in Ubuntu. |
* Mark, a netbook-remix user, wants to add a new test case for an application that already has a test case for Ubuntu desktop. He calls the already available test case from the UNR page, needing only to upload the netbook-remix specific screenshots. * Anne, a Kubuntu user, finds a bug in Kpdf while reading a complex PDF file. She creates a LP bug but also creates a new test case to track future regressions. This file will be added also as test case for Evince in Ubuntu. |
Line 24: | Line 23: |
A standard test case template will be drawn up which will be used by all test cases. * Code will be added to the wiki theme or a plug-in to parse the page and validate that the syntax follows the correct standard. * A new-test-case macro will be written to easily add cases with the correct syntax |
|
Line 25: | Line 28: |
Possible solutions * Create a new form to add new test cases, that allows to add steps and screenshot * A test case template to be copied and edited when adding a new test case In both cases a plugin should then add the text to the specific application page and track the testcase number. |
* plugin should then add the text to the specific application page and track the testcase number. '''-- how does this work? - heno''' |
Line 34: | Line 32: |
After much discussion it has been agread that we will use the following syntax to track the test cases number as: | After much discussion it has been agreed that we will use the following syntax to track the test cases number as: |
Line 40: | Line 38: |
We have decided to move the pages from /Distribution/Application to just application. Therefore, in example, Ekiga page will be at http://testcases.qa.ubuntu.com/Applications/Ekiga instead of http://testcases.qa.ubuntu.com/Ubuntu/Applications/Ekiga. | We have decided to move the pages from {{{/Distribution/Applications}}} to just {{{/Applications}}}. For example, the Ekiga page will be at {{{http://testcases.qa.ubuntu.com/Applications/Ekiga}}} instead of {{{http://testcases.qa.ubuntu.com/Ubuntu/Applications/Ekiga}}}. |
Line 48: | Line 46: |
An improvement, once the rest of the changes are in place, would be an easyway to maintain a table of results of test runs. | An improvement, once the rest of the changes are in place, would be an easy way to maintain a table of results of test runs. |
Launchpad Entry: qa-testscase-wiki
Created: 2008-12-15
Contributors: heno, schwuk, apulido, sbeattie
Packages affected:
Summary
Migrating all test cases to testcases.qa.ubuntu.com, adding useful macros and making sure all relevant teams are using the new facility.
Rationale
We need a consistent syntax to define the test cases and a better way to track them. Right now, test cases are maintained under https://wiki.ubuntu.com/Testing/Cases but the syntax is not defined and there is no logical way to track what it is available.
Use Cases
- Mark, a netbook-remix user, wants to add a new test case for an application that already has a test case for Ubuntu desktop. He calls the already available test case from the UNR page, needing only to upload the netbook-remix specific screenshots.
- Anne, a Kubuntu user, finds a bug in Kpdf while reading a complex PDF file. She creates a LP bug but also creates a new test case to track future regressions. This file will be added also as test case for Evince in Ubuntu.
Design
Testcases Syntax
A standard test case template will be drawn up which will be used by all test cases.
- Code will be added to the wiki theme or a plug-in to parse the page and validate that the syntax follows the correct standard.
- A new-test-case macro will be written to easily add cases with the correct syntax
Lobbing for a respect in the syntax through how-tos or tutorials is not enough; syntax should be more or less forced by the wiki itself.
plugin should then add the text to the specific application page and track the testcase number. -- how does this work? - heno
Testcase Number
After much discussion it has been agreed that we will use the following syntax to track the test cases number as:
- If it is a test case for an application, then the testcase number will be [a-z]{2}-[0-9]{3} (2 letters for the application and 3 numbers for the test case number)
- If it is a test case for a Launchpad bug then the testcase number will be lp-[bugnumber]
Path Structure
We have decided to move the pages from /Distribution/Applications to just /Applications. For example, the Ekiga page will be at http://testcases.qa.ubuntu.com/Applications/Ekiga instead of http://testcases.qa.ubuntu.com/Ubuntu/Applications/Ekiga.
A Moin Moin plugin will be needed to change to the proper screenshots when called with /Ubuntu, /Kubuntu, /Xubuntu, etc.
The testcases from pulled from launchpad bugs exists under http://testcases.qa.ubuntu.com/Regressions/ with per-data type subtrees, but is open to renaming or restructuring.
Tracking results
An improvement, once the rest of the changes are in place, would be an easy way to maintain a table of results of test runs.
Implementation
- A new LP project needs to be created to track bugs and code for the custom changes (plugins, themes and macros).
UI Changes
It would be really useful to have a way to add new test cases easily. A new form with the necessary information as text boxes would be the best approach.
Code Changes
TBA
Migration
- After migration all the old test cases wiki pages need to have just an advice pointing to the new test cases wiki.
Unresolved issues
In the discussion we talked a bit about integration and communication with the ISO tracker, but this was a light talk and nothing was committed.
BoF agenda and discussion
Testcases in wiki.ubuntu.com: https://wiki.ubuntu.com/Testing/Cases
New testcases specific wiki at http://testcases.qa.ubuntu.com
- Syntax of testcases - Way to add attached documents or additional files to the test case.
- https://testcases.qa.ubuntu/Distro/Applications/Application seems a natural way to order the wiki - We could have a macro that pulls the correct screenshots for a test cases depending on the distributuion:
- i.e. We cave testcases.qa.ubuntu.com/Applications/Gedit, but when called from /Xubuntu/Applications/Gedit it will contain Xubuntu screenshots if available.
- Syntax of the track number - Right now is going u-ff-001 (u for ubuntu, ff for the app (in this case firefox), and 3 number for the test case numbers) - Change that to ff-001 (2 letters for the application and 3 numbers for the test case number)
- How to use macros to encourage people to contribute in a easier
OEM/UNR Testcase format example: https://wiki.ubuntu.com/Testing/Cases/Ubuntu-Netbook-Remix
How to track test cases run?
- - Chris maintains a table with results and then use a python script to parse it and convert it if necessary. - [Chris] We need to use testcases.qa.ubuntu.com to track results of tests. Maybe with macros. (in a ISO tracker manner) - ISO tracker and testcases.qa.ubuntu.com communication
Are testcases wiki tasks handled in LP? Do we have a project to file bugs?
- - Dave to set up a LP project to file bugs and have the code in bzr
Meta Data for case suite creation - Meta codes for each case:
- - Distro(s) - Version(s)
Code format - ff-001 = Ubuntu Jaunty, Firefox, Case 1
QATeam/Specs/TestcaseWikiMigration (last edited 2009-01-13 10:54:46 by cpc4-oxfd8-0-0-cust39)