QATracker Video Screencast

A video screencast of the qatracker and how to use it can be found here.

What is it?

The QATracker is the master repository for all our testing within ubuntu QA. It holds our testcases, records our results, and helps coordinate our testing events. There are several instance of the qatracker to help separate out results.

QAtracker Instances

http://iso.qa.ubuntu.com/ is used to report results for image testing

http://packages.qa.ubuntu.com/ is used to report results for package testing

http://laptop.qa.ubuntu.com/ is used for laptop/hardware testing

How does it work?

The QATracker works by presenting a list of milestones (or events) for testing. Each milestone (or event) contains products (or packages) which we want to test. Within each product (or package) is a series of testcases. Finally, for each testcase a tester is able to report a result of pass or failure, along with bugs encountered and comments. For more details, see the How can I use it? section below.

Milestones

A milestone is a named testing event. It contains a set of products (hardware, packages, or isos) that have been specified for testing.

STATUS

A milestone can have a Testing, Released, or Archived status. A milestone starts with a testing status, allowing new results to be posted. Once the testing is complete, it is set to released status. Finally, the results are changed from released to archived once the development cycle is over and the next has begun. The released and archived statuses are read only and no results can be submitted for them.

Builds / Respins

Each product has a build or version number attached to it. When a new version is added, the previous version is archived, including the results. During iso testing when this happens it is dubbed a 'respin'. The 'respin' of the iso is generally to fix bugs encountered during the testing. After a respin it is necessary to retest all the mandatory testcases in order to ensure the new version of the iso is now compliant.

SUMMARY SCREEN

The picture above shows an example of product summaries. In the summary, you can see how many "mandatory", "run-once" and "optional" tests a product has, along with information how many of them have been run and how many of them have been failed (a red number in parenthesis).

In addition, you can see how many non-critical (green bug icon) and critical bugs (red bug icon) have been found. Mousing over a bug loads a bug summary. Clicking the bug icon will load the corresponding bug on Launchpad.

Finally, the first column provides you a link to the downloads and the last column shows the version/build in question.

ARCHIVED AND SUPERSEDED RESULTS

I submitted a result, but now it appears to be gone?

Don't worry! The result is still present. Click the "See removed and superseded builds too" link above the product listing. A page showing the older versions, along with results will load.

What does a testcase look like?

The format is documented here. On the qatracker, the testcase is an expanding link on the results page.

Click to expand the testcase and view it.

What testcases should I run?

Testcases can be marked as mandatory, run-once or optional. Mandatory testcases should always be run, for the item being tested in order to ensure it's successful. Run-once testcases should be executed at least one time, but are not required to be re-run if a new version of the item is released (for example, a re-spin to fix bugs found in an iso). Finally, optional testcases are just that, optional. They don't need to be run in order for the item to be considered "tested", but are still important. If possible, running an optional test can help ensure there are no regressions.

How can I use it?

If you’ve never used the QATracker before, don’t worry. First you’ll need a ubuntu SSO account. You might already have one of these from ubuntu one, or by signing up for launchpad. If not, follow the new account guide for launchpad and sign up to get a launchpad (ubuntu sso) account.

Got your ubuntu sso account? Great, check out the walkthrough’s below to help you submit your first result! Although the underlying website is the same, the process for testing is slightly different depending on the type of testing your wishing to help with.

Rather watch a video?

Are you a visual learner and would rather see a video version of these walkthroughs? Check out this youtube channel.

Cadence Testing

Interested in helping out during a cadence testing week? Cadence Testing Walkthrough

Image testing

Interested in helping test out our images’s to help make sure they install and work properly on a variety of hardware? Iso Testing Walkthrough

Call for testing

Interested in helping test a specific package or piece of software via a call for testing? Call for Testing Walkthrough