This is a place where I draft documents for inclusion in the wiki

Proposed Changes to ISO Testing Tutorial 14/04/2021

ISO-Testing Tutorial - QA Testing

Introduction

You have chosen the "QA Testing" alternative to test the Ubuntu development branch ISO images. If this is not right, click here to return to the previous page.

In this section you will learn how to:

• Access the Ubuntu QA Team ISO Tracker
• Select ISO images, download and install them

• Perform Test-Cases

• Report results

Step 1: Create a Launchpad account

If you don't have a Launchpad Account, click here to create one.

Step 2: Hardware Profile

When we run the SYSTEM TESTING utility or any of the other Checkbox type testing suites a hardware profile is generated at the end of the test. A copy of it is stored in our Launchpad account.

Log in to your Launchpad account and add this line to the url of your page and press enter.

/+hwdb-submissions

The page that you go to will list all the hardware profiles that have been generated when you ran the System Testing and Checkbox test suites.

Use that same page url when reporting test results to QA. So, copy that url into a text document where is can be handily retrieved when reporting your results.

When we run a test case we can report to the QA team that the result was either a PASS or a FAIL.

• To Report a PASS result to QA we need:
  • A Single Sign On identification.
  • A hardware profile.
• To report a FAIL result to QA we need:
  • A Single Sign On identity.
  • A hardware profile.
  • A number for the bug or bugs that caused the test case to fail.

SINGLE SIGN ON IDENTIFICATION

We get a SSO ID when we register at Launchpad. The email address and the password that we gave when registering at Launchpad are used by the Single Sign ON process to uniquely identify us.

Step 3: Login and use the QA ISO Tracker

• Access the QA ISO Tracker here and use your Launchpad account and password to login.

• Click on the milestone that has the status of Testing

• Click on the Ubuntu product you will download and test (example: Ubuntu, Lubuntu, Ubuntu Studio, Mythbuntu, etc).
• A list of "test cases" for the selected product will be displayed.

• Select a test-case and the tracker will show you a Link to the download information and a link to the test-case description.

• These ISO images come with an identification code based upon the date - for example 20120421. The ISO image available on the 2012/04/21 and which is being tested during that 24 hour period.

Step 4: Small FAQ

What is an ISO?

The ISO is an "image" file. It can be burned to a CD, DVD or USB portable drive (like a pen drive, USB Stick, etc).

Where are the ISOs?

• When you select an Ubuntu product inside the QA ISO Tracker, you will see a Link to download information.

• Click on this link to see download options you can use.

What are "Standard" and "Alternate" ISOs?

• "Standard" ISOs will use a graphical user interface (GUI) when installing and install a standard Ubuntu desktop operating system.

• "Alternate" ISOs use a different installer and allow for some different install tasks, such as installing to systems with very low ram (less than 384MB, implementing RAID and LVM, among other.

• This tutorial is about QA testing of ISO images so we should download the ISO from QA ISO Testing Tracker page. These ISO images come with an identification code based upon the date - for example 20120421. The ISO image available on the 2012/04/21 and which is being tested during that 24 hour period and against which we test.

Should I pick the "standard" or the "alternate"?

• If you know and use the features mentioned in the previous topic (RAID, LVM) or have previously used the Alternate ISO to overcome some particular issue in your setup, choose the Alternate ISO.

• If you don't use or are not familiar with the features mentioned and have never needed to use the alternate ISO, choose the Standard ISO.

• Despite that, if you are more confident in trying something new, you are encouraged to try the alternate ISO if your test-case requires that, even if you haven't ever used it.

Which ISO "architecture" should I download?

Your system can probably run a 64-Bit version of Ubuntu if you have a modern computer. If you are not sure, just download the 32-Bit version.

Where are the ISOs?

As you click on Link to the download information, you'll be presented with some options for downloading the ISO.

Step 5: Downloading Ubuntu development branch ISO

There are many possible ways to get it. It's just an ordinary download from the web. Let's explore two main methods of download:

• Browser: Simply point your browser to the ISO download site (QA download page) and download the desired ISO to any folder in your current operating system. ~/Downloads is a good choice. If you are using Windows or other operating system, save the file to the usual download location.

• ZSync: ZSync allows you to quickly update a local ISO file to a version that is available Online. We use it to keep up with daily changes (and not have to re-download the entire ISO everyday). But it's smart to use it even if you don't have an outdated ISO because ZSync can continue a download from where it stopped (in case you loose your Internet connection during the download). ZSync will only be available if you're using Ubuntu (or other Linux distribution).

Example 1: Downloading Ubuntu Precise Pangolin Beta 2 64-Bit standard ISO to your Downloads folder with ZSync:

• 1.Click on the Dash
  2.Open a Gnome-Terminal
  3.Type in (or paste) the following into the Gnome-Terminal window:
  

Tip: To paste into the terminal, use the keys Ctrl+Shift+V or right click anywhere inside the terminal and select "Paste"

   1 cd $HOME/Downloads
   2 zsync http://cdimage.ubuntu.com/daily-live/current/precise-desktop-amd64.iso.zsync 

Example 2: Downloading Ubuntu Precise Pangolin Beta 2 32-Bit standard ISO to your Downloads folder with ZSync:

• 1.Click on the Dash
  2.Open a Gnome-Terminal
  3.Type in (or paste) the following into the Gnome-Terminal window:
  

Tip: To paste into the terminal, use the keys Ctrl+Shift+V or right click anywhere inside the terminal and select "Paste"

   1 cd $HOME/Downloads
   2 zsync http://cdimage.ubuntu.com/daily-live/current/precise-desktop-i386.iso.zsync 

ZSync will show the download status and completion percentage on the terminal. It will exit when it reaches 100%. After that, if you browse to your Downloads folder, you'll see the ISO there.

ZSync: Command not found
If you see this message, it means you don't have ZSync installed in your system. Don't worry, it's a very small and fast download. Just type in (or paste) the following into the Gnome-Terminal window:
Tip: To paste into the terminal, use the keys Ctrl+Shift+V or right click anywhere inside the terminal and select "Paste"

   1 sudo apt-get update && sudo apt-get install zsync 

Enter your password in Gnome-Terminal and wait a few seconds for the installation to finish. Now use one of the two examples above to download a 32-Bit or 64-Bit Ubuntu ISO using your recently installed ZSync.

Step 6: Check your Test-Case!

• Your test case may ask you to install from a CD-ROM/DVD-ROM, from a USB Stick, from Ubuntu, from Windows, using a graphical or console installer.
• Verify the appropriate media and proceed to the adequate section in the next step.

• Some test cases apply to a range of ISO images. This breakdown will give you more information about test cases Test case breakdown.

Step 7: Burning the ISO

Burning the ISO to a CD-ROM or DVD-ROM

From Ubuntu

• 1.Right-click on the downloaded ISO file. A context-menu should appear.
  2.In this context-menu, you should see either a default or a 3rd part application option to burn the ISO to a CD-ROM/DVD-ROM
  3.It's a fairly simple process, but if you need detailed instructions, read them here.
  

From Windows

• 1.Right-click on the downloaded ISO file. A context-menu should appear.
  2.In this context-menu, you should see either a default or a 3rd part windows program option to burn the ISO to a CD-ROM/DVD-ROM
  3.If you don't have this option as a default on Windows and also don't have a 3rd part application to burn CD-ROM/DVD-ROM installed, download and install InfraRecorder. It's a simple application, but if you need instructions, read them here.
  

Burning the ISO to a USB Removable Drive

From Ubuntu

• 1.Click on the Dash and open Startup Disk Creator'
  2.Click on "Other" and select the Ubuntu ISO you have downloaded previously.
  3.Select your PenDrive and create your Ubuntu ISO USB stick
  

From Windows

1. Insert a USB stick with at least 2GB in your PC.
   

2. Download and install PenDrive Linux
   

3. Run the program. On "Step 1", select "Local ISO". On "Step 2", select the Ubuntu ISO you have downloaded previously. On "Step 3", select your PenDrive unit and click on "Create". Your ISO will be recorded to the PenDrive.
   

Mounting the ISO to a Virtual Machine Hard Disk or virtual CD-ROM drive

If your test case allows for this possibility, consider using VirtualBox as the virtualization software. If you already have vmware-player or other virtualization software installed and know how to use it, presumably you don't need help mounting the Ubuntu ISO as one of it's virtual drives. If that is the case, please proceed directly to Step 3 (Installing the Ubuntu ISO).

• If you have VirtualBox installed and know how to mount Ubuntu ISO on it, proceed directly to the next step in this tutorial.

• If you would like to read instructions on how to install VirtualBox and mount Ubuntu ISO as one of its virtual drives, please click here.

Step 8: Check your Test-Case (Yes. Again!)

• Your test-case will describe what exactly you are supposed to do and how you should install the previously burned ISO CD/DVD or USB (or a previously setup Virtual Machine). Test case breakdown

Step 9: Installing Ubuntu Development Branch

This sections provides some help on the most common ways to install Ubuntu. Please refer to your test-case and choose the most appropriate method.

From Ubuntu

"Graphical" Install (Ubiquity)

1. Simply insert your recently burned CD-ROM/DVD-ROM or USB Stick and you should automatically see the installer in a few moments
   

2. At this point you are already testing.
   

3. Remember to take notes of any interesting error, bug, misbehavior you see.
   

Console (text-mode, "Non-graphical") Install

1. Simply insert your recently burned CD-ROM/DVD-ROM or USB Stick to your PC and restart it.
   

2. If your PC still boots from the hard disk, loading your previous Ubuntu (or other operating system) and not the beta, it means you have to instruct it to boot from the CD-ROM/DVD-ROM drive or USB unit. If that is the case, restart the PC and, during POST (the usually black screen that appears right after you power on the PC), try to read what Key should be pressed to select the boot order or boot device. If this option is not present directly, you should press the key to enter the PC BIOS Setup. You can get detailed instruction on setting up the PC to boot from a CD/DVD here and, for booting from USB, check here.

3. At this point you are already testing.
   

4. Remember to take notes of any interesting error, bug, misbehavior you see.
   

From Windows

"Graphical" Install (Wubi)

1. Simply insert your recently burned CD-ROM/DVD-ROM or USB Stick and you will be prompted by Windows to execute it.
   

2. At this point you are already testing.
   

3. Remember to take notes of any interesting error, bug, misbehavior you see.
   

Console (text-mode, "Non-graphical") Install

1. Simply insert your recently burned CD-ROM/DVD-ROM or USB Stick to your PC and restart it.
   

2. If your PC still boots from the hard disk, loading your previous Ubuntu (or other operating system) and not the beta, it means you have to instruct it to boot from the CD-ROM/DVD-ROM drive or USB unit. If that is the case, restart the PC and, during POST (the usually black screen that appears right after you power on the PC), try to read what Key should be pressed to select the boot order or boot device. If this option is not present directly, you should press the key to enter the PC BIOS Setup. You can get detailed instruction on setting up the PC to boot from a CD/DVD here and, for booting from USB, check here.

3. At this point you are already testing.
   

4. Remember to take notes of any interesting error, bug, misbehavior you see.
   

On a VirtualBox Virtual Machine (Windows or Ubuntu "Host")

Follow the step by step procedures outlined here.

Step 10:Reporting Bugs

1. The first thing to do is create a new Launchpad account if you don't already have one.
   

2. By the end of the test-case you will have a link for reporting any bug you eventually find.

3. If you are unsure or don't know exactly how to explain the bug, consider reading the procedures outlined here or ask for support at the UbuntuForums.

April 28th 12.04 (on 26th) released

This is a mock up of an Activities Planner for testing. I have taken the offical QQ draft release schedule and I am adapting it into an Activities Planner

ACTIVITIES PLANNER - TESTING

Period: QQ - May to October 2012

Start date is the release date of Precise Pangolin on 26th April 2012. The Week start on a Thursday.

Aim: To identify testing opportunities that fit in with the development release process

Combined Testing Activities Planner & Draft QQ Release Schedule

each cell has a place holder with the same colour as the background. This hides the text. Replace the place holder with your text and remove the style=color format and change the background color or remove it Unused cells have have #F1F1DD as the background color; Test dates yet to be held should have red background. Test dates completed should have green background.

Coordinated

Each cell has a place holder with the same colour as the background. This hides the text. Replace the place holder with your text and remove the style=color format for the text and change the background color or remove it Unused cells have have #F1F1DD as the background color; Test dates yet to be held should have red background. Test dates completed should have green background.

Regular

each cell has a place holder with the same colour as the background. This hides the text. Replace the place holder with your text and remove the style=color format and change the background color or remove it Unused cells have have #F1F1DD as the background color; Test dates yet to be held should have red background. Test dates completed should have green background.

Color ranges being tested here. I want paler colors than these.

Ideas

• Week 2 May 10
  • Testing the tool chain. But how?
• Week 3
• Week 4
• Week 5 May 31
  • June 02 - 06 Live session and install tests of pre-alpha 1 ISO
• Week 6 June 7 or a later week
  • June 07 - August 09 Identifying and testing packages subject to Partner Upload Deadline
  • June 07 - August 23 Identifying and testing packages subject to a Feature Freeze Deadline
  • June 07 - August 30 Identifying and testing packages subject to User Interface Freeze
• week 8 June 21
  • Identifying and testing Debian packages post Debian Import Freeze
• Week 8
  • June 23 - 27 Live session and install tests of pre-alpha 2 ISO
• Week 13
  • July 28 - August 01 Live session and install tests of pre-alpha 3 ISO

* Week 18

• September 01 - 05 Live session and install tests of pre-beta 1 ISO

• Week 21
  • September 22 - 26 Live session and install tests of pre-beta 2 ISO
• Week 23 October 04
  • Identifying and testing kernel packages post kernel freeze
• Week 24 October 11
  • Identifying and testing packages relative to post release candidate
  • October 13 - 17 Live session and install tests of pre-final release 12:10 ISO

Links to the ISO testing test cases

Live Session: Start ID: DC-DLS-001; Finish ID: DC-DLS-002; Persistence ID: dis-003 Test cases DC-DLS-001, 002 & dis-003

Install (entire disk): ID: TC-UWI-001 test case TC-UWI-001

Free software Only - Entire disk ID: ufo-001 test case ufo-001

Install auto-resize: ID: uri-001 test case uri-001

Install Desktop (manual partitioning): ID: umi-001 test case umi-001

Wubi Desktop Install: Windows side ID: wdi-001; Linux side ID: wdi-002; CD Boot Helper ID: wdi-003; Wubi Stand Alone Desktop Install ID: wdi-004 http://testcases.qa.ubuntu.com/Install/DesktopWubi

Install (manual partitioning): ID: uoi-001; Desktop Ready for End User ID: uoi-002; End User Creation ID: uoi-003 test case uoi-001, 002 & 003

Install (oem set up)- Entire disk: ID: uoi-001 test case uoi-001

Note: The same test cases are used for Installation Broken Network, Non-English Installation and Non-English Live Session & Installation. They are all Entire disk installations.

Full Network support ID: nen-001; No Network ID: nen-002; Live Session Install Full Network Support ID: nen-003; Installation Full Network support & CJK Input ID: nen-004; Live Session Installation No Network Support ID: nen-005 test case nen-001, 002, 003, 004, 005

Install Screen Reader: ID: uai-001 test case uai-001 Note: Sound is muted by default in a Ubuntu live session and in a first install of Ubuntu.

Migration Assistant auto-resize install: ID: uma-001 test case uma-001

VMWare Easy Install: ID: vmw-001 test case vmw-001