TestcaseWikiMigration

Revision 2 as of 2008-12-18 15:20:48

Clear message
  • Launchpad Entry: qa-testscase-wiki

  • Created: 2008-12-15

  • Contributors: heno, schwuk, apulido, sbeattie

  • Packages affected:

Summary

Migrating all test cases to testcases.qa.ubuntu.com, adding useful macros and making sure all relevant teams are using the new facility.

Rationale

We need a consistent syntax to define the test cases and a better way to track them. Right now, test cases are maintained under https://wiki.ubuntu.com/Testing/Cases but the syntax is not defined and there is no logic way to track what it is available.

Use Cases

Mark, a netbook-remix user, wants to add a new test case for an application that it is already available for Ubuntu desktop. He uses the already available test case for Ubuntu, needing only to upload the netbook-remix specific screenshots.

Anne, a Kubuntu user, finds a bug in Kpdf while reading a complex PDF file. She creates a LP bug but also creates a new test case to track future regressions. This file will be added also as test case for Evince in Ubuntu.

Design

Testcases Syntax

Lobbing for a respect in the syntax through how-tos or tutorials is not enough; syntax should be more or less forced by the wiki itself.

Possible solutions

  • Create a new form to add new test cases, that allows to add steps and screenshot
  • A test case template to be copied and edited when adding a new test case

In both cases a plugin should then add the text to the specific application page and track the testcase number.

Testcase number

After much discussion it has been agread that we will use the following syntax to track the test cases number as:

  • If it is a test case for an application, then the testcase number will be [a-z]{2}-[0-9]{3} (# 2 letters for the application and 3 numbers for the test case number)
  • If it is a test case for a Launchpad bug then the testcase number will be lp-[bugnumber]

Implementation

This section should describe a plan of action (the "how") to implement the changes discussed. Could include subsections like:

UI Changes

Should cover changes required to the UI, or specific UI that is required to implement this

Code Changes

Code changes should include an overview of what needs to change, and in some cases even the specific details.

Migration

Include:

  • data migration, if any
  • redirects from old URLs to new ones, if any
  • how users will be pointed to the new way of doing things, if necessary.

Test/Demo Plan

It's important that we are able to test new features, and demonstrate them to users. Use this section to describe a short plan that anybody can follow that demonstrates the feature is working. This can then be used during testing, and to show off after release. Please add an entry to http://testcases.qa.ubuntu.com/Coverage/NewFeatures for tracking test coverage.

This need not be added or completed until the specification is nearing beta.

Unresolved issues

This should highlight any issues that should be addressed in further specifications, and not problems with the specification itself; since any specification with problems cannot be approved.

BoF agenda and discussion

Testcases in wiki.ubuntu.com: https://wiki.ubuntu.com/Testing/Cases

New testcases specific wiki at http://testcases.qa.ubuntu.com

  • Syntax of testcases - Way to add attached documents or additional files to the test case.

    - https://testcases.qa.ubuntu/Distro/Applications/Application seems a natural way to order the wiki - We could have a macro that pulls the correct screenshots for a test cases depending on the distributuion:

    • i.e. We cave testcases.qa.ubuntu.com/Applications/Gedit, but when called from /Xubuntu/Applications/Gedit it will contain Xubuntu screenshots if available.
  • Syntax of the track number - Right now is going u-ff-001 (u for ubuntu, ff for the app (in this case firefox), and 3 number for the test case numbers) - Change that to ff-001 (2 letters for the application and 3 numbers for the test case number)
  • How to use macros to encourage people to contribute in a easier

OEM/UNR Testcase format example: https://wiki.ubuntu.com/Testing/Cases/Ubuntu-Netbook-Remix

How to track test cases run?

  • - Chris maintains a table with results and then use a python script to parse it and convert it if necessary. - [Chris] We need to use testcases.qa.ubuntu.com to track results of tests. Maybe with macros. (in a ISO tracker manner) - ISO tracker and testcases.qa.ubuntu.com communication

Are testcases wiki tasks handled in LP? Do we have a project to file bugs?

  • - Dave to set up a LP project to file bugs and have the code in bzr

    http://moinmo.in/

Meta Data for case suite creation - Meta codes for each case:

  • - Distro(s) - Version(s)

Code format - ff-001 = Ubuntu Jaunty, Firefox, Case 1


CategorySpec