This page will walk through participating in a cadence testing week. It will demonstrate how to use a live session from a development image or an installed development system to run through testcases found on the QAtracker and submit a result.
NOTE: This covers testing of packages during the cadence week. If you wish to help test images during a cadence week, you can follow this tutorial instead.
Video Screencast of Tutorial
A video version of this tutorial can be seen here.
- Launchpad/Ubuntu SSO account
- Updated daily image of ubuntu (development) or up to date installed version of ubuntu (development)
- Instructions for obtaining below
Setup your testing environment
Using an installed version of ubuntu (development)
If you already running the development version of ubuntu, login to your system and make sure your up to date by running
sudo apt-get update && sudo apt-get dist-upgrade
You are now set to perform the tests.
Grabbing a live image
If your not running a development version of ubuntu, head over to the cdimage repository and download the latest image matching your hardware. For most new hardware, this will be the 64-bit PC (AMD64) desktop image.
Burning the live image
Now, in order for you to test you will use the live image in a 'live session' environment by booting up a pc or virtual machine using the image. Once booted, select 'Try ubuntu without installing' to boot into a live session.
If you wish to use a virtual machine to run the live session, create a new virtual machine with the image as the cd-rom and boot the machine.
Logging in to the package tracker
Open a browser and navigate to http://packages.qa.ubuntu.com/
Click the login link located on the lefthand side of the page
- Enter your ubuntu sso account information
Selecting the milestone
Once logged into the package tracker, you should see a list of milestones. The cadence week you wish to help test for should be listed, as shown below. Select the cadence milestone you wish to test. NOTE: Only milestones with a status of 'Testing' can have test results submitted. If the status is set to released, no new results can be contributed.
Select the milestone that matches your cadence week with a status of 'Testing'
You should know see a list of packages ready for testing. Select the package you wish to test.
This page shows the testcases to perform for the selected package.
Reporting a bug
Additionally, there is a link to the bug reporting instructions. This page will detail how to report a bug and/or provide a link to help report bugs found properly. If a bug is found while testing, refer to that link for instructions on reporting.
Executing the testcase
Select a testcase and a page containing instructions to follow to execute the test will appear. Most of the testcases follow a simple format - perform an action, and then expect a result. A step in this format is shown below.
Alternatively the testcase may appear as the form of yes/no questions, where the expected result is a generally a yes. This is called a smoke test.
For more information on the testcase formats, have a look at testcase formats page.
Go through each step of the testcase and follow the instructions. Perform the action, and ensure the expected result occurs, or that you are able to positively answer each question. If something different occurs, note it as a bug, but continue the testcase.
Preparing to submit a result
While performing the test, if an expected result does not occur, note it as a bug. Follow the bug-reporting instructions for the product you are testing, and report it once the test has ended. A bug can be attached to a 'passed' or 'failed' result, and testcases can have multiple bugs.
If a bug was encountered, check the “Bugs” section above the submit form to see if the bug is a duplicate of one already reported against the testcase.
The list contains previously encountered bugs found during testing this testcase. Roll the mouse over the bug icon to see the information, or click to be taken to the report on launchpad. If the bug is found to be a duplicate, note the number of the bug and use it during your report. If the bug was not a duplicate and a new bug needs to be reported, follow the instructions found under the 'Link to bug reporting instructions' for the iso.
Determining the result
A testcase can have unexpected results, but still pass. Generally a passed result can be issued whenever each step in the testcase is able to be executed (even if the results don't match the expected result) and the final result matches the intent of the testcase. If in doubt of whether or not the bug in question represents a testcase failure, report the bug and file the testcase as 'failed' with a comment regarding the confusion. Ask for help on the mailing list for confirmation from someone else on the team.
Submit a result
An example of a failure result submission. Note that a bug number is required to submit a 'failed' result.
An example of a passed submission.
Congratulations, the result has been submitted! After submitting the result should appear in the list below the testcase.
If editing is needed, select the 'pencil' icon in the row of your submission to edit the testcase or delete it.
Further testcases can now executed and reported against. Explore other testcases and submit results. If unable to understand or execute a testcase, seek help on the mailing list. Happy Testing!