Testing
Contents |
Upgrade Testing
Lucid is an LTS release meaning that many Ubuntu users will take the opportunity to upgrade their systems to Ubuntu 10.04. Testing upgrades is not easy, as the differences in hardware configurations and state of the systems in the moment of upgrade, can raise corner cases that it is impossible to cover.
We are going to take the opportunity of the Ubuntu Global Jam to ask everyone participating in a Jam to upgrade their Hardy and/or Karmic systems to Lucid and report back any issues they may encounter.
Requirements
A Upgrade Jam is very similar to any kind of jam, but it's a bit special. The bottleneck is the bandwidth, as many of the attendees will be upgrading, so it is almost crucial to have a Local Ubuntu Mirror. A good how-to for setting up an ubuntu mirror can be found here: https://help.ubuntu.com/community/Debmirror.
You also want to provide some snacks and drinks.
Asking people to prepare for the Jam
- People wanting to participate in the Upgrade Jam need to have a Launchpad account.
We will be reporting our results in the ISO tracker, so an ISO tracker account is also necessary. Ask people to read the ISO procedures and to create an account.
Running the Jam
Point people to use the local mirror set up, instead of the common Ubuntu archive. Instructions on how to set up a client are also at https://help.ubuntu.com/community/Debmirror
Upgrade your system to Lucid by running sudo update-manager --d
After finishing the upgrade check that your system still works correctly. https://wiki.ubuntu.com/Testing/UpgradeChecklist contains a simple checklist of things that worth a try.
File any bugs you may find. Tag them with the tags hardy2lucid or karmic2lucid. Check the Debugging Update Manager wiki page for a nice guide on how to report your upgrade bugs.
After the upgrade it would be nice if you could run System Testing (under System->Administration->System Testing) to gather information about your system. Submit the data to Launchpad.
- Report "Pass or Failed in the corresponding ISO test case". See below for details on this part.
Reporting to the ISO tracker
The ISO tracker is used to test the ISOs and/or upgrades during milestone testing. We are going to use it as well for the Ubuntu Global Jam.
We are going to create a fake milestone called UbuntuGlobalJam and this is the one that we are going to use during the jam. The information about how to use the ISO tracker is in the testing wiki pages. All upgrade tests in the tracker are found within the Upgrade category.
When reporting in the ISO tracker you can report as "Passed", "Failed" and you can add the numbers of the bugs reported. Also, there is some space to submit a comment. It would be nice if you could put the URL of your System Testing submission (https://www.launchpad.net/~<your_lp_id>/+hwdb-submissions)
General Testing
Testing is split into 2 main parts.
- The testing of the installer. This covers Ubiquity (the live cd installer), Debian-installer (the alternate installer) and wubi.
- The testing of applications. This is done by everyone who installs the latest developer version of *buntu.
Most people test these in a virtual environment although by the time beta comes along a lot of user will now be testing this on real hardware too.
I will break down how to test the above in the sections below and describe the usage of the iso test tracker for those that decide to participate.
Requirements
A computer.
A Launchpad Account.
A current CD or ISO.
A network connection.
Check if your computer has a cpu with added virtualization extensions. If the cpu line comes back with vmx or svm then you do. Which means you be able to use kvm instead of qemu.
egrep '(vmx|svm)' /proc/cpuinfo
Finally a vm KVM/Qemu/Virtualbox, unless installing on hardware.
Asking people to prepare for the Jam
Grabbing a copy of the latest iso.
Check All the above to ensure you can use vm.
Install your chosen vm, unless running on hardware.
Material
Important links:
The Iso Tracker.
Using VirtualBox (older version but works pretty much the same now).
Towards the bottom half of the page gives you the general idea for a kvm command.
Links to the testcases.
The Example-content package if not running on Ubuntu.
Holding the Session
Introduction Session
Application/System Testing
The easy bit
Goto System->Administration->System Testing. Run through the application filling in the yes no questions at the end you'll be asked for your LaunchPad id and password. This is because the information about you machine is stored in your Launchpad account at https://edge.launchpad.net/~yourusername/+hwdb-submissions. This is especially useful if you hit hardware regression as you can add you HW id into your bug report.
The longer bit
Okay so the system testing is out of the way now it's time to start on the main application testing.
First off if your not running Ubuntu install the example-content package. This gives you example files to test applications for media, documents and pictures.
Secondly start running through the applications one at a time till you're happy they are working. For the most part this will involve opening the application, running the application, closing the application. For examples on what to test goto http://testcases.qa.ubuntu.com/Applications/ this currently covers most of the default installed applications.
Install Testing
TEST TRACKER
First things first lets split the team between hardware installed/installing and vm users.
Next for those new to the process lets set up a tracker account. Goto http://iso.qa.ubuntu.com click on Log In then on Create New Account fill in your details and then click on ISO tracker.
Now you have access to the tracker you can select the type of install you wish to perform. Start with the iso type you have and then the partitioning type.
On The Tracker
Now you have selected your install type you can goto the Additional instructions are available for this testcase, click here to read them. this will then tell you the steps to run through to install using your selected method.
Once you complete the install please mark if the test passed or failed and any bugs you have found by adding the bug number to the boxes provided.
This helps the developers and release team fix issue for the next release.
Once the install is complete please move on to the Application/System testing.
Workflow
Tools & Helpful Information
Get help on #ubuntu-testing.