TestcaseWikiMigration

Differences between revisions 8 and 9
⇤ ← Revision 8 as of 2009-01-06 13:17:10 →
Size: 4470
Editor: cpc4-oxfd8-0-0-cust39
Comment: cleaned up
← Revision 9 as of 2009-01-06 13:44:03 → ⇥
Size: 4582
Editor: cpc4-oxfd8-0-0-cust39
Comment: merged in the rest of the BoF notes
Deletions are marked like this. Additions are marked like this.
Line 18: Line 18:
 * Chris wishes to track the testing for a mobile build. He can do that in the wiki by using a macro
Line 24: Line 25:
 
Line 27: Line 28:
Lobbing for a respect in the syntax through how-tos or tutorials is not enough; syntax should be more or less forced by the wiki itself.
 * plugin should then add the text to the specific application page and track the testcase number. '''-- how does this work? - heno'''		  * A plugin should then add the text to the specific application page and track the testcase number. '''-- how does this work? - heno'''

==== Metadata ====

  * Distro(s) - for which distro flavours is the case valid?
  * Version(s) - for which distro versions is the case valid?
  * Application name
  * Attachments required for the test case should be stored in the wiki and linked explicitly
Line 46: Line 53:
An improvement, once the rest of the changes are in place, would be an easy way to maintain a table of results of test runs. A test tracking facility in a separate page tree but in the same wiki will make it easy to track test results linked to the cases.

 * Use a standard page space such as {{{/TestRuns/<identifier>}}} to collect results
 * Write a macro to list the tests used (and optionally display the full case text) and collect the results through simple input boxes
Line 54: Line 64:
 * '''Macro for collecting and displaying case numbers and titles in table form''' '''We need a starting reference for this - heno'''  * '''Displaying case numbers''' - A ''Macro'' for collecting and displaying case numbers and titles in table form '''We need a starting reference for this - heno'''
 * '''Test results''' - A '''Macro''' based on [[http://moinmo.in/MacroMarket/PageComment2|PageComment2]] to display case information with result entry boxes and result tables. The actual results are stored in a sub-page (as the comment macro does).
Line 62: Line 73:
Testcases in wiki.ubuntu.com: https://wiki.ubuntu.com/Testing/Cases

New testcases specific wiki at http://testcases.qa.ubuntu.com

 * Syntax of testcases
 - Way to add attached documents or additional files to the test case.

How to track test cases run?
 - Chris maintains a table with results and then use a python script to parse it and convert it if necessary.
 - [Chris] We need to use testcases.qa.ubuntu.com to track results of tests. Maybe with macros. (in a ISO tracker manner)
 

Meta Data for case suite creation - Meta codes for each case:
 - Distro(s)
 - Version(s)
 

Code format - ff-001 = Ubuntu Jaunty, Firefox, Case 1		 ''merged into spec''

  • Launchpad Entry: qa-testscase-wiki

  • Created: 2008-12-15

  • Contributors: heno, schwuk, apulido, sbeattie

  • Packages affected:

Summary

Migrating all test cases to testcases.qa.ubuntu.com, adding useful macros and making sure all relevant teams are using the new facility.

Rationale

We need a consistent syntax to define the test cases and a better way to track them. Right now, test cases are maintained under https://wiki.ubuntu.com/Testing/Cases but the syntax is not defined and there is no logical way to track what it is available.

Use Cases

  • Mark, a netbook-remix user, wants to add a new test case for an application that already has a test case for Ubuntu desktop. He calls the already available test case from the UNR page, needing only to upload the netbook-remix specific screenshots.
  • Anne, a Kubuntu user, finds a bug in Kpdf while reading a complex PDF file. She creates a LP bug but also creates a new test case to track future regressions. This file will be added also as test case for Evince in Ubuntu.
  • Chris wishes to track the testing for a mobile build. He can do that in the wiki by using a macro

Design

Testcases Syntax

A standard test case template will be drawn up which will be used by all test cases. Template should appear in the spec. - heno

  • Code will be added to the wiki theme or a plug-in to parse the page and validate that the syntax follows the correct standard.
  • A new-test-case macro will be written to easily add cases with the correct syntax

  • A plugin should then add the text to the specific application page and track the testcase number. -- how does this work? - heno

Metadata

  • Distro(s) - for which distro flavours is the case valid?
  • Version(s) - for which distro versions is the case valid?
  • Application name
  • Attachments required for the test case should be stored in the wiki and linked explicitly

Testcase Number

After much discussion it has been agreed that we will use the following syntax to track the test cases number as:

  • If it is a test case for an application, then the testcase number will be [a-z]{2}-[0-9]{3} (2 letters for the application and 3 numbers for the test case number)
  • If it is a test case for a Launchpad bug then the testcase number will be lp-[bugnumber]

Path Structure

We have decided to move the pages from /Distribution/Applications to just /Applications. For example, the Ekiga page will be at http://testcases.qa.ubuntu.com/Applications/Ekiga instead of http://testcases.qa.ubuntu.com/Ubuntu/Applications/Ekiga.

A Moin Moin plugin will be needed to change to the proper screenshots when called with /Ubuntu, /Kubuntu, /Xubuntu, etc.

The testcases from pulled from launchpad bugs exists under http://testcases.qa.ubuntu.com/Regressions/ with per-data type subtrees, but is open to renaming or restructuring.

Tracking results

A test tracking facility in a separate page tree but in the same wiki will make it easy to track test results linked to the cases.

  • Use a standard page space such as /TestRuns/<identifier> to collect results

  • Write a macro to list the tests used (and optionally display the full case text) and collect the results through simple input boxes

Implementation

  • Admin: A new LP project needs to be created to track bugs and code for the custom changes (plugins, themes and macros).

  • Syntax validation plug-in - A Parser or Action is required for test case syntax validation We need a starting reference for this - heno

  • Form for adding test cases Write a plug-in or macro to easily add test cases based on the NewPage or PageComment2 macros.

  • Displaying case numbers - A Macro for collecting and displaying case numbers and titles in table form We need a starting reference for this - heno

  • Test results - A Macro based on PageComment2 to display case information with result entry boxes and result tables. The actual results are stored in a sub-page (as the comment macro does).

Migration

  • After migration all the old test cases wiki pages need to have just an advice pointing to the new test cases wiki.

BoF agenda and discussion

merged into spec

CategorySpec

QATeam/Specs/TestcaseWikiMigration (last edited 2009-01-13 10:54:46 by cpc4-oxfd8-0-0-cust39)