TestcaseWikiMigration

Differences between revisions 7 and 8
Revision 7 as of 2009-01-06 12:34:44
Size: 5304
Editor: cpc4-oxfd8-0-0-cust39
Comment:
Revision 8 as of 2009-01-06 13:17:10
Size: 4470
Editor: cpc4-oxfd8-0-0-cust39
Comment: cleaned up
Deletions are marked like this. Additions are marked like this.
Line 23: Line 23:
A standard test case template will be drawn up which will be used by all test cases. A standard test case template will be drawn up which will be used by all test cases. '''Template should appear in the spec. - heno'''
Line 50: Line 50:
 * A new LP project needs to be created to track bugs and code for the custom changes (plugins, themes and macros).  '''Admin:''' A new LP project needs to be created to track bugs and code for the custom changes (plugins, themes and macros).
Line 52: Line 52:
=== UI Changes ===

It would be really useful to have a way to add new test cases easily. A new form with the necessary information as text boxes would be the best approach.

=== Code Changes ===

TBA
 * '''Syntax validation plug-in''' - A ''Parser'' or ''Action'' is required for test case syntax validation '''We need a starting reference for this - heno'''
 * '''Form for adding test cases''' Write a plug-in or macro to easily add test cases based on the [[http://moinmo.in/MacroMarket/NewPage|NewPage]] or [[http://moinmo.in/MacroMarket/PageComment2|PageComment2]] macros.
 * '''Macro for collecting and displaying case numbers and titles in table form''' '''We need a starting reference for this - heno'''
Line 64: Line 60:
== Unresolved issues ==

In the discussion we talked a bit about integration and communication with the ISO tracker, but this was a light talk and nothing was committed.
Line 75: Line 68:
 - https://testcases.qa.ubuntu/Distro/Applications/Application seems a natural way to order the wiki
 - We could have a macro that pulls the correct screenshots for a test cases depending on the distributuion:
  i.e. We cave testcases.qa.ubuntu.com/Applications/Gedit, but when called from /Xubuntu/Applications/Gedit it will contain Xubuntu screenshots if available.
 
 * Syntax of the track number
 - Right now is going u-ff-001 (u for ubuntu, ff for the app (in this case firefox), and 3 number for the test case numbers)
 - Change that to ff-001 (2 letters for the application and 3 numbers for the test case number)

 * How to use macros to encourage people to contribute in a easier

OEM/UNR Testcase format example:
https://wiki.ubuntu.com/Testing/Cases/Ubuntu-Netbook-Remix
Line 91: Line 72:
 - ISO tracker and testcases.qa.ubuntu.com communication
Line 93: Line 73:
  Are testcases wiki tasks handled in LP? Do we have a project to file bugs?
 - Dave to set up a LP project to file bugs and have the code in bzr
 
 http://moinmo.in/
  • Launchpad Entry: qa-testscase-wiki

  • Created: 2008-12-15

  • Contributors: heno, schwuk, apulido, sbeattie

  • Packages affected:

Summary

Migrating all test cases to testcases.qa.ubuntu.com, adding useful macros and making sure all relevant teams are using the new facility.

Rationale

We need a consistent syntax to define the test cases and a better way to track them. Right now, test cases are maintained under https://wiki.ubuntu.com/Testing/Cases but the syntax is not defined and there is no logical way to track what it is available.

Use Cases

  • Mark, a netbook-remix user, wants to add a new test case for an application that already has a test case for Ubuntu desktop. He calls the already available test case from the UNR page, needing only to upload the netbook-remix specific screenshots.
  • Anne, a Kubuntu user, finds a bug in Kpdf while reading a complex PDF file. She creates a LP bug but also creates a new test case to track future regressions. This file will be added also as test case for Evince in Ubuntu.

Design

Testcases Syntax

A standard test case template will be drawn up which will be used by all test cases. Template should appear in the spec. - heno

  • Code will be added to the wiki theme or a plug-in to parse the page and validate that the syntax follows the correct standard.
  • A new-test-case macro will be written to easily add cases with the correct syntax

Lobbing for a respect in the syntax through how-tos or tutorials is not enough; syntax should be more or less forced by the wiki itself.

  • plugin should then add the text to the specific application page and track the testcase number. -- how does this work? - heno

Testcase Number

After much discussion it has been agreed that we will use the following syntax to track the test cases number as:

  • If it is a test case for an application, then the testcase number will be [a-z]{2}-[0-9]{3} (2 letters for the application and 3 numbers for the test case number)
  • If it is a test case for a Launchpad bug then the testcase number will be lp-[bugnumber]

Path Structure

We have decided to move the pages from /Distribution/Applications to just /Applications. For example, the Ekiga page will be at http://testcases.qa.ubuntu.com/Applications/Ekiga instead of http://testcases.qa.ubuntu.com/Ubuntu/Applications/Ekiga.

A Moin Moin plugin will be needed to change to the proper screenshots when called with /Ubuntu, /Kubuntu, /Xubuntu, etc.

The testcases from pulled from launchpad bugs exists under http://testcases.qa.ubuntu.com/Regressions/ with per-data type subtrees, but is open to renaming or restructuring.

Tracking results

An improvement, once the rest of the changes are in place, would be an easy way to maintain a table of results of test runs.

Implementation

  • Admin: A new LP project needs to be created to track bugs and code for the custom changes (plugins, themes and macros).

  • Syntax validation plug-in - A Parser or Action is required for test case syntax validation We need a starting reference for this - heno

  • Form for adding test cases Write a plug-in or macro to easily add test cases based on the NewPage or PageComment2 macros.

  • Macro for collecting and displaying case numbers and titles in table form We need a starting reference for this - heno

Migration

  • After migration all the old test cases wiki pages need to have just an advice pointing to the new test cases wiki.

BoF agenda and discussion

Testcases in wiki.ubuntu.com: https://wiki.ubuntu.com/Testing/Cases

New testcases specific wiki at http://testcases.qa.ubuntu.com

  • Syntax of testcases - Way to add attached documents or additional files to the test case.

How to track test cases run?

  • - Chris maintains a table with results and then use a python script to parse it and convert it if necessary. - [Chris] We need to use testcases.qa.ubuntu.com to track results of tests. Maybe with macros. (in a ISO tracker manner)

Meta Data for case suite creation - Meta codes for each case:

  • - Distro(s) - Version(s)

Code format - ff-001 = Ubuntu Jaunty, Firefox, Case 1


CategorySpec

QATeam/Specs/TestcaseWikiMigration (last edited 2009-01-13 10:54:46 by cpc4-oxfd8-0-0-cust39)