CallforTesting

Revision 8 as of 2012-07-25 21:43:54

Clear message

Summary

A call for testing is meant to call attention to a particular piece of software for testing over a short time-frame in order to confirm and resolve and bugs and ready for general consumption. The QA community can install and test your software for bugs and usability and provide bug reports and feedback using the qatracker.

What do I get?

As a developer, you will get consolidated feedback on how many people tested, the bugs reported, and any comments they may have left while testing. In addition, you will be able to see this for each of your versions of the software. This allows you to spot regressions and confirm fixes between versions.

What does it look like?

Have a look at the Kernel testing as an example. Let's point out the different pieces:

Quick summary of all the versions of the software, number of testers and bugs found (rollover the bugs to get additional details)

Testcases for the software. Notice the link to the installation instructions and bug reporting instructions. Also notice also the changelog showing the changes in this version at the top

Detailed results for the specifc version selected. Notice the usernames of the tester, bugs, the pass/fail indications and comments

A list of bugs, and more information about bug status found for each piece of software under test

Finally, here's information on how the users will use the site to submit results.

What do I need to do?

Coordinate with Nicholas Skaggs to setup a tracker for your software. You'll need to supply the following

  • Testcases
    • The testcases can be simple smoke tests (like, "Does the application "work"?"), but ultimately we would like to add more in-depth testcases for specific funcationality. The testcase format should be as follows -- (the QA Community coordinator can help you with this):
      • Perform X action
      • Expect Y result
  • Create a ppa with builds for each of your target releases.
  • Create instructions for installing (something like add the ppa, and install X package)
  • Create instructions for uninstalling (usually just ppa purge works)
  • Create instructions for reporting a bug. It's important to note what package you want the bug report against, and if multiple packages, which one to report for which reason. You may choose to use ubuntu-bug, or provide a pre-formmatted link to 'report a bug' on launchpad.
  • Give the timeframe for testing, if applicable

Then what happens?

With those pre-requisites completed, the QA community coordinator will verify the application, tests and the ppa. A new milestone for your package will be created and and the link will be provided back to you. A wide call for testing will go out to the greater ubuntu community, and users will utilize the tracker to report results.

Results

As results come in, the tracker will update to reflect the number of contributors, what tests are passing or failing, and what bugs are reported. Everything is public and availible via the tracker. See the 'What does it look like?' section above.