Procedures

Revision 11 as of 2008-01-30 18:42:31

Clear message

Getting Ready

What you need

  • A spare computer or hard drive for test installs. (A virtual machine can be used but we would prefer to see more tests performed on a variety of physical hardware.)
  • A moderate level of Linux experience so you are able to find the relevant log files and produce useful bug reports: ReportingBugs

  • Some available time before a milestone release to perform tests and file reports.
  • A [https://launchpad.net Launchpad] account so you can sign up for testing and report bugs. If you have signed up to edit the Ubuntu wiki you will already have a Launchpad account.

Downloading images

Download the latest daily build for testing from one of these locations:

Updates to these builds will be announced on #ubuntu-devel, #ubuntu-testing and on email (for those who have signed up to testing).

Check the md5sum! To check the md5sum of your image, open a terminal and execute:

md5sum imagename.iso

Where "imagename.iso" is the name of the iso image that you have downloaded and want to verify. The reference value can be found in a file named MD5SUMS in the ISO image download directory.

Using Rsync

Once you've downloaded a daily image you can get updated versions of that image more quickly using rsync (typically 10% of the full download time). This is because the images will likely not have changed dramatically over a few days, and they are built in such a way as to make running a rsync feasible (yay!). Read instructions on using rsync at: https://help.ubuntu.com/community/RsyncCdImage . You may want to set up a simple shell script to perform rsyncs of the images you plan to test.

Again please: Check the md5sum!. In fact, when using rsync that is also a handy way of checking that you actually have the latest daily image.

Testing

In doing specific testing of the CD/DVD images it is important to focus on those aspects that are typically not used by those who simply run the latest unstable version on their system through daily updates (Such testing is of course also extremely valuable). Key points in image testing include image integrity (md5sums), Live CD and installer functionality.

See the overview of [:Testing/InstallMethods:Install Methods].

Filing bugs

As with all testing it is important to [https://help.ubuntu.com/community/ReportingBugs file bugs]. If you find a bug, please, search if it has already been reported, and if it hasn't, report it yourself. You should also refer to the [https://wiki.ubuntu.com/DebuggingProcedures bug filing/debugging guide] for that specific package (if available) to make sure you are aware of known issues and have attached the relevant log files.

Daily images and Alpha releases

The daily images between milestone releases will typically only be tested by developers, testers and enthusiasts, while the milestones (Alphas in the case of the Hardy Heron) are more widely announced and will be tested by a larger audience.

The most important goal with these images is simply to find as many bugs as possible and report them with enough detail that they can be fixed. Finding the bugs ahead of the milestone crunch (in random dailies) is helpful as it gives us more time to fix them. Of course finding and reporting bugs in the released milestones (Alphas) is also useful for improving the overall quality of the distribution.

As there is less time pressure on testing these images than the pre-release ones, we would encourage more in-depth testing on a range of hardware to flush out the deeper hidden bugs. If possible, please use the ["Testing/Cases"] and perform installs using a variety of methods.

pre-Alpha releases

The purpose of the Alpha milestones is to encourage some wider public testing of the CD builds. Before we can recommend the images for public testing we must at least ensure that they have all built correctly (are not in themselves corrupt) and are at least known to boot on some machines. Some form of installation should also be tested. We do not require that these images be tested for any possible minor defect or that all possible install methods work, since that is rather the goal of the wider image testing itself. Find the show stopper bugs and get them fixed using the ["Testing/Cases"] with at least one install method.

When an important error is found, the current frozen images are rejected and new ones are built after the fix has been applied. It is therefore important to make sure that you are testing the most recent images.

If the bug is specific to one flavour, only those will be re-built and the other flavour builds remain valid. So if we find a serious bug in the Ubuntu version we will rebuild those images, which will take several hours in total. That would be a good opportunity to test K/X/Edubuntu while you wait Smile :) Likewise, if we find a bug that exists across all flavors, but only on i386, we will re-build i386 and all amd64 and PPC images remain valid.

Beta, RC and Final

For Beta, RC and Final we have extended testing periods where we try to find as many bugs as we can. With the Beta we want to put our best foot forward across the distribution and want to fix as many bugs as we have the manpower for, large or small. For RC and the final release on the other hand, we will only fix the remaining release critical bugs. This is because seemingly small changes can have unintended consequences and break other things. Minor changes that are merely a good idea to fix will be deferred to downloadable updates.

The procedure of rejecting know bad images and continuing testing of unaffected ones also applies here as with the Alphas.

Reporting results

Once you have completed your testing of a particular image, you can [:Testing/Community/ReportingResults:report your results].