When do we test?

An ISO is tested just before it is released, always on a Thursday. These are the possible ISO releases during a development period (releases marked as opt-in are not obligatory for flavors):

Testing ISO How To

See Testing/ISO for more thorough information regarding ISO testing.

Launchpad Account

In order to report results, you need a launchpad account. So, if you don't have one, head to http://launchpad.net, and create one!

Where do we do testing?

iso.qa.ubuntu.com - this is where we find our test cases and where we report our results from testing the ISO. You need a launchpad account in order to login there.

The milestones (Alpha, Beta, Release Candidate, Final Release)

If there is something to test, it will be listed under something equivalent to Milestones for Wily Series, and it's status will be marked Testing. We don't test the Daily Milestone, only actual releases, such as Final Beta, or Release Candidate.

The Builds at the Testing Tracker Page

To see the actual ISOs being tested, click on the milestone you want to test to get to the Testing Tracker page.

Ubuntu Studio ISOs are listen under Product(Ubuntu Studio). You can see some information about the tests there, already. How many mandatory tests there are, and how many of them are completed. Also, the date for the last build, under Version.

What if all tests are completed?

Even if all tests are completed, you should still test if you are able. This way we get more machines involved with more possible bugs that can be reported.

Upgrade Tests

Ubuntu Studio ISOs are also listed under product(Upgrade), which is where you test upgrading from the previous release to the current development release. These tests are currently less of a priority to us, but you may also test these if you like.

Where are the testing instructions?

Click on one of the Ubuntu Studio products, such as Ubuntu Studio amd64. This will take you to the testing page for the Ubuntu Studio amd64 ISO.

You'll see the link Link to the download information, which is where you go to download the ISOs. You'll also see a table with test cases. You'll want to at least perform the mandatory test cases, but before we do that, we need to download the ISO.

Downloading the ISO

Click on Link to the download information. You'll see a list of download types and links to them. We prefer using zsync.

zsync checks if the ISO has previously been downloaded, and if it has, it only downloads what was changed. You may need to download the same ISO a couple of times during testing, because of rebuilds, so this is where zsync becomes handy.

To download using zsync, you need to use the terminal. Create a folder for your ISOs, such as ~/iso, head to that dir, and then use zsync to download. Following commands do just that (replace the url to the ISO with the one you want to download, lines beginning with # are comments):

  # To make sure you are in home dir, cd to home root (~)
  cd ~ 
  # Create the iso dir with mkdir
  mkdir iso
  # change directory to iso
  cd iso/
  # zsync the file
  zsync http://cdimage.ubuntu.com/ubuntustudio/dvd/20150924/wily-dvd-amd64.iso.zsync

Every time you need to re-download the ISO, just open a terminal, cd into the iso dir, and zsync the file again.

Check your ISO for errors - important!

Sometimes every bit is not correct in the file you downloaded, and that can result in installation failures. So, to make sure there's nothing wrong with the ISO, you need to do a checksum. On the download page, there's a link to a MD5 checksum. Click on it, and you'll see something like this:

  d9508ed2fbc20713f126e00530cbdc7e *wily-dvd-amd64.iso
  01f20ed4850a56002735140eb04ec3e3 *wily-dvd-i386.iso

The above lines are the checksums to the two particular builds of Ubuntu Studio ISOs. You're checksums will be different.

To get the checksum of the ISO you downloaded, use a terminal:

  # change directory to where the ISOs are located
  cd iso/
  # get the md5 checksum
  md5sum wily-dvd-amd64.iso

Then compare the result you got with the checksum on the download page. If they are not identical, there is a problem with your ISO, and you'll need to re-download.

Create an Installer

Now that you have the ISO downloaded, you'll need to install it somehow. Either on a virtual machine, such as VirtualBox, or a real machine - which is much preferred. If you use a virtual machine, you don't need to create an installer. You can install directly from the ISO.

There are many ways to create an installer. Burning a DVD is the simplest, but those are particularly error prone when it comes to OS installers. If one bit is wrong, it may result in an installation failure. USB installers are much preferred.

There are numerous ways to create USB installers. The fastest is using the command line tool dd. Mount your USB stick, then make sure you know which drive letter it has (something like /dev/sdb, or /dev/sdc - most likely your system is installed on /dev/sda) - otherwise you risk overwriting the wrong drive, and that may end in tears!

To find out what letter your USB drive has, you can run this:

  sudo fdisk -l

You should see be able to tell which drive has which letter by the size of the disk.

Once you know which drive letter your USB stick has, perform this command:

  sudo dd bs=512 if=/path/to/yakkety-dvd-amd64.iso of=/dev/sdX

If your usb stick has the drive letter sdb, and the iso is located in /home/<username>/iso/, then the command would be (for the yakkety development release of Ubuntu Studio):

  sudo dd bs=512 if=/home/<username>/iso/yakkety-dvd-amd64.iso of=/dev/sdb

It takes a few minutes. Once you see your prompt again, the process has finished, and you can try using the USB installer.

Performing a test

To perform a test, head back to testcases page, and click on the test you'd like to perform, such as Install(entire disk). Just follow the instructions, such as here:

If you find bugs

If you found any bugs during your test, begin by checking the bottom section of the testing page for bugs to look out for, and if someone already reported your bug. In that case, you may want to head to the bug link, read about it, see if you can add something to it, and mark it affecting you as well. Later, you will want to add the bug into your test results.

But, if no one had reported the bug, you need to report a new one. You need a launchpad account to report bugs, so make sure you have that first. Then, from the release that you are testing (if possible), run the command (replace <packagename> with the name of the package you want to report against):

  ubuntu-bug <packagename>

How to know which package to report the bug against

Not easy knowing which package the bug is in at all times, so if you need help figuring that out, please post to the ubuntu-studio-devel mail list, and you'll get assistance. Reporting bugs in itself can be quite a task for beginners, so patience is required.

There's a general instruction on the http://iso.qa.ubuntu.com page for which package to report bugs against in different situations, and it goes like this:

For problems encountered ...
... during the installation itself:                         ubuntu-bug ubiquity
... while booting the ISO image:                            ubuntu-bug syslinux
... while booting the system after installation:            ubuntu-bug grub2
... with the graphical display after installation:          ubuntu-bug xorg
... with rescue mode:                                       ubuntu-bug friendly-recovery
... with an application:                                    ubuntu-bug PACKAGE_NAME

Someone already reported this bug

If the bug you wanted to report had already been reported, make sure to save the bug number, because you will be using it for the test result later.

Submitting a test result

Once you are done following the test instructions, and possibly reporting bugs, you should submit a test result. This is done at the bottom of test page. You can only do this if you are logged in!

Mark the test failed only if you were not able to complete the test. Otherwise, mark it passed.

Add any bugs you encountered in the bug fields. Like this:

   1487819, 1466191

Critical bugs are bugs that makes the test fail. Other bugs should be put in the Bugs field.

You may read more about hardware profile at http://iso.qa.ubuntu.com, if you would like find out more about that.

Add comments, if you wish. Then press submit.

And, that's it!