Ubuntu Wiki

Preparations

What you need

• A laptop to be tested in a live environment or in an installed system. (A virtual machine cannot be used because we need to test real hardware.)

• A moderate level of Linux experience so you are able to find the relevant log files and produce useful bug reports: ReportingBugs

• Some available time just after a milestone release to perform tests and file reports.

Setting up

• If you do not already have one, set up a Launchpad account so you can report bugs. If you have signed up to edit the Ubuntu wiki you will already have a Launchpad account.

• Now you have to have an account in the tracking system, which lets you report your test results. We are going to use it to generate data (it will be publicly available).

  1. Go to laptop.qa.ubuntu.com.

  2. Click on "Log In" and "Create New Account"

     Note: If you have already registered for one of the other QA sites (e.g. ISO Tracker) you don't need to do it for the laptop tracker also, unless you are going to test more than a laptop. In this case you need a new registration for each additional laptop you own. A nice suggestion could be to name each new tracker username as Username-Laptopmake, where Laptopmake = Asus, Acer, Toshiba etc.

• Set up your profile providing all these required following info (mandatory):

  1. Laptop make and model

     • Fill in with the manufacturer name and the model of your laptop, as outputted by the commands reported capitalizing the first letter only(see below). In some special cases the output can be wrong or not as expected, so check it and, if needed, put the correct value manually adjusting it as these examples:

       • Dell Inc. → Dell

       • Matsushita Electric Industrial Co.,Ltd. → Panasonic

       • TOSHIBA → Toshiba

       • ASUSTeK Computer Inc. → Asus

     

  2. Laptop hardware info

     Fill in with the output of lspci -v command.

     Fill in with the output of lsusb command.

     Fill in with the output of lshw -short command. (You'll get a warning that you should be a super user or the list could be incomplete. Don't worry about that) .

• Launchpad username

  • Fill in with your Launchpad username

• Select the ISO images you expect to test and look over the relevant test cases, making sure you have the test setup you need.
• You can sign up for individual test cases on any ISO. This helps everyone to see which tests are being covered and sends you a notification email when a new milestone is available to be tested, provided you have opted in to email notification (user settings).

Downloading images

Milestone ISO images are found at cdimage.ubuntu.com.

The images currently being tested are listed on laptop.qa.ubuntu.com with a direct download link and the appropriate rsync command for your convenience (click for the download page). We advise that you download the images you expect to test ahead of time and use rsync to update them, as this will reduce the turn around time on testing. Rsync only downloads the parts of the image that have changed and typically takes about 10% of the full download time with reasonably fresh images. See instructions on using rsync.

Testing

On the tracker you will be presented with a list of available milestones to test, the stable ones like Lucid Lynx 10.04.2 and those of the current development release of Ubuntu.

Every milestone we will be starting a new testing cycle. Everybody is free to submit their results in the weeks that follow the official release of a milestone (although we encourage you to run them as soon as possible, so developers can have enough time to fix the bugs).

You have to choose the milestone and the architecture (32 or 64 bit) you want to test and each one has a set of test cases associated with it. Click on the tracker to see details of the test case.

To subscribe to a specific ISO and tests, select the architecture from the list. Then select the tests you wish to subscribe to and then hit subscribe. You will receive an email announcing there are tests available, provided you also opt-in to email notification on your profile.

Reporting results

The test results are tracked on the laptop testing tracker website at laptop.qa.ubuntu.com. Once you are registered, you can log in to report your test results. The website is divided into several sections, and results are reported based on the following criteria:

• Ubuntu architecture (e.g., Ubuntu i386 or amd64)
• Category of hardware (e.g., Audio, Video, Input/Output etc.)
• Piece of hardware to test (e.g., Speakers, Keyboard, Hotkeys, etc.)

Starting a test

Laptop test cases normally last for at least half an hour. To avoid duplicating efforts, it is good to let others know that you have started a test case.

To mark a test as "Started" select the result "Started" and submit the result. It will show up in the list with a "clock" icon.

Adding test reports

When you have conducted a test you can report the results in the Laptop tracker with a general comment and bug numbers if applicable.

You can mark a test as Passed or Failed. In the Laptop test tracker we mark as failures those experiences that prevented us to do what we wanted to achieve in that test case. I.e. If we want that Audio works, and you can't hear from speakers, that's a failure. If you hear slight noise from speaker, the test didn't fail as such. We would mark that as success, but will link the non-critical bugs that we find.

Filing bugs

As with all testing it is important to file bugs. If you find a bug, please, search if it has already been reported, and if it hasn't, report it yourself. You should also refer to the bug filing/debugging guide for that specific piece of hardware to make sure you are aware of known issues and have attached the relevant log files.

To mark a test as Failed it is mandatory to link the test report with at least one bug in Launchpad.

Results summary

You can see a summary of results clicking on the link Laptops summary present in Action left-hand menu and are reported based on the following criteria:

• Laptop manufacturer (Acer, Dell, Asus etc.)
• Laptop model
• Release (Lucid, Natty etc.)
• Architecture (i386, amd64)
• Category of hardware (e.g., Audio, Video, Input/Output etc.)
• Piece of hardware (e.g., Speakers, Keyboard, Hotkeys, etc.)