TestcaseWikiMigration

Differences between revisions 6 and 7
Revision 6 as of 2008-12-18 17:03:55
Size: 5124
Editor: 208-151-246-43
Comment:
Revision 7 as of 2009-01-06 12:34:44
Size: 5304
Editor: cpc4-oxfd8-0-0-cust39
Comment:
Deletions are marked like this. Additions are marked like this.
Line 12: Line 12:
We need a consistent syntax to define the test cases and a better way to track them. Right now, test cases are maintained under https://wiki.ubuntu.com/Testing/Cases but the syntax is not defined and there is no logic way to track what it is available. We need a consistent syntax to define the test cases and a better way to track them. Right now, test cases are maintained under https://wiki.ubuntu.com/Testing/Cases but the syntax is not defined and there is no logical way to track what it is available.
Line 16: Line 16:
Mark, a netbook-remix user, wants to add a new test case for an application that it is already available for Ubuntu desktop. He uses the already available test case for Ubuntu, needing only to upload the netbook-remix specific screenshots.

Anne, a Kubuntu user, finds a bug in Kpdf while reading a complex PDF file. She creates a LP bug but also creates a new test case to track future regressions. This file will be added also as test case for Evince in Ubuntu.
 * Mark, a netbook-remix user, wants to add a new test case for an application that already has a test case for Ubuntu desktop. He calls the already available test case from the UNR page, needing only to upload the netbook-remix specific screenshots.
 * Anne, a Kubuntu user, finds a bug in Kpdf while reading a complex PDF file. She creates a LP bug but also creates a new test case to track future regressions. This file will be added also as test case for Evince in Ubuntu.
Line 24: Line 23:
A standard test case template will be drawn up which will be used by all test cases.
 
 * Code will be added to the wiki theme or a plug-in to parse the page and validate that the syntax follows the correct standard.
 * A new-test-case macro will be written to easily add cases with the correct syntax
Line 25: Line 28:

Possible solutions
 * Create a new form to add new test cases, that allows to add steps and screenshot
 * A test case template to be copied and edited when adding a new test case
 
In both cases a plugin should then add the text to the specific application page and track the testcase number.
 * plugin should then add the text to the specific application page and track the testcase number. '''-- how does this work? - heno'''
Line 34: Line 32:
After much discussion it has been agread that we will use the following syntax to track the test cases number as: After much discussion it has been agreed that we will use the following syntax to track the test cases number as:
Line 40: Line 38:
We have decided to move the pages from /Distribution/Application to just application. Therefore, in example, Ekiga page will be at http://testcases.qa.ubuntu.com/Applications/Ekiga instead of http://testcases.qa.ubuntu.com/Ubuntu/Applications/Ekiga. We have decided to move the pages from {{{/Distribution/Applications}}} to just {{{/Applications}}}. For example, the Ekiga page will be at {{{http://testcases.qa.ubuntu.com/Applications/Ekiga}}} instead of {{{http://testcases.qa.ubuntu.com/Ubuntu/Applications/Ekiga}}}.
Line 48: Line 46:
An improvement, once the rest of the changes are in place, would be an easyway to maintain a table of results of test runs. An improvement, once the rest of the changes are in place, would be an easy way to maintain a table of results of test runs.
  • Launchpad Entry: qa-testscase-wiki

  • Created: 2008-12-15

  • Contributors: heno, schwuk, apulido, sbeattie

  • Packages affected:

Summary

Migrating all test cases to testcases.qa.ubuntu.com, adding useful macros and making sure all relevant teams are using the new facility.

Rationale

We need a consistent syntax to define the test cases and a better way to track them. Right now, test cases are maintained under https://wiki.ubuntu.com/Testing/Cases but the syntax is not defined and there is no logical way to track what it is available.

Use Cases

  • Mark, a netbook-remix user, wants to add a new test case for an application that already has a test case for Ubuntu desktop. He calls the already available test case from the UNR page, needing only to upload the netbook-remix specific screenshots.
  • Anne, a Kubuntu user, finds a bug in Kpdf while reading a complex PDF file. She creates a LP bug but also creates a new test case to track future regressions. This file will be added also as test case for Evince in Ubuntu.

Design

Testcases Syntax

A standard test case template will be drawn up which will be used by all test cases.

  • Code will be added to the wiki theme or a plug-in to parse the page and validate that the syntax follows the correct standard.
  • A new-test-case macro will be written to easily add cases with the correct syntax

Lobbing for a respect in the syntax through how-tos or tutorials is not enough; syntax should be more or less forced by the wiki itself.

  • plugin should then add the text to the specific application page and track the testcase number. -- how does this work? - heno

Testcase Number

After much discussion it has been agreed that we will use the following syntax to track the test cases number as:

  • If it is a test case for an application, then the testcase number will be [a-z]{2}-[0-9]{3} (2 letters for the application and 3 numbers for the test case number)
  • If it is a test case for a Launchpad bug then the testcase number will be lp-[bugnumber]

Path Structure

We have decided to move the pages from /Distribution/Applications to just /Applications. For example, the Ekiga page will be at http://testcases.qa.ubuntu.com/Applications/Ekiga instead of http://testcases.qa.ubuntu.com/Ubuntu/Applications/Ekiga.

A Moin Moin plugin will be needed to change to the proper screenshots when called with /Ubuntu, /Kubuntu, /Xubuntu, etc.

The testcases from pulled from launchpad bugs exists under http://testcases.qa.ubuntu.com/Regressions/ with per-data type subtrees, but is open to renaming or restructuring.

Tracking results

An improvement, once the rest of the changes are in place, would be an easy way to maintain a table of results of test runs.

Implementation

  • A new LP project needs to be created to track bugs and code for the custom changes (plugins, themes and macros).

UI Changes

It would be really useful to have a way to add new test cases easily. A new form with the necessary information as text boxes would be the best approach.

Code Changes

TBA

Migration

  • After migration all the old test cases wiki pages need to have just an advice pointing to the new test cases wiki.

Unresolved issues

In the discussion we talked a bit about integration and communication with the ISO tracker, but this was a light talk and nothing was committed.

BoF agenda and discussion

Testcases in wiki.ubuntu.com: https://wiki.ubuntu.com/Testing/Cases

New testcases specific wiki at http://testcases.qa.ubuntu.com

  • Syntax of testcases - Way to add attached documents or additional files to the test case.

    - https://testcases.qa.ubuntu/Distro/Applications/Application seems a natural way to order the wiki - We could have a macro that pulls the correct screenshots for a test cases depending on the distributuion:

    • i.e. We cave testcases.qa.ubuntu.com/Applications/Gedit, but when called from /Xubuntu/Applications/Gedit it will contain Xubuntu screenshots if available.
  • Syntax of the track number - Right now is going u-ff-001 (u for ubuntu, ff for the app (in this case firefox), and 3 number for the test case numbers) - Change that to ff-001 (2 letters for the application and 3 numbers for the test case number)
  • How to use macros to encourage people to contribute in a easier

OEM/UNR Testcase format example: https://wiki.ubuntu.com/Testing/Cases/Ubuntu-Netbook-Remix

How to track test cases run?

  • - Chris maintains a table with results and then use a python script to parse it and convert it if necessary. - [Chris] We need to use testcases.qa.ubuntu.com to track results of tests. Maybe with macros. (in a ISO tracker manner) - ISO tracker and testcases.qa.ubuntu.com communication

Are testcases wiki tasks handled in LP? Do we have a project to file bugs?

  • - Dave to set up a LP project to file bugs and have the code in bzr

    http://moinmo.in/

Meta Data for case suite creation - Meta codes for each case:

  • - Distro(s) - Version(s)

Code format - ff-001 = Ubuntu Jaunty, Firefox, Case 1


CategorySpec

QATeam/Specs/TestcaseWikiMigration (last edited 2009-01-13 10:54:46 by cpc4-oxfd8-0-0-cust39)