Testing

Differences between revisions 2 and 25 (spanning 23 versions)
Revision 2 as of 2009-09-11 18:56:23
Size: 4144
Editor: 94-171-216-200
Comment:
Revision 25 as of 2013-02-14 22:36:58
Size: 6915
Editor: nskaggs
Comment:
Deletions are marked like this. Additions are marked like this.
Line 2: Line 2:
<<TableOfContents>> ||<tablestyle="float:right; font-size: 0.9em; width:40%; background:#F1F1ED; margin: 0 0 1em 1em;" style="padding:0.5em;"><<TableOfContents>>||
Line 4: Line 4:
= Introduction = = Preparing for the Session =
You are encouraged to familiarize yourself with each type of testing in advance so you have a basic understanding of what to expect. In addition, it's encouraged for you to read the following pages:

https://wiki.ubuntu.com/Testing/QATracker

https://wiki.ubuntu.com/Testing/Activities

https://wiki.ubuntu.com/QATeam/ContributingTestcases


For each of the different types of testing a ubuntu global jam milestone should be setup on each tracker with the testcases ready to and and results ready to be reported against. See the tests on [[https://wiki.ubuntu.com/QATeam/Cadence/Raring/Week7UbuntuGlobalJam|this milestone page]]

Note that unless folks are willing to do installations (via VM or real hardware) they will be testing using the live session of the image they brought with them (by booting it up into a VM or on real hardware).

= Asking people to prepare for the Jam =
Have participants choose what ways they would like to contribute. In general for contributing test results, they should plan to bring a copy of the latest daily image, or provide an up-to-date installation of the development version of ubuntu.
 * Download a [[http://cdimage.ubuntu.com/daily-live/current/|copy of the latest image]]
 * Ensure they have a [[https://help.launchpad.net/YourAccount/NewAccount|Launchpad/Ubuntu SSO account]]

==== Important links ====
 * [[https://wiki.ubuntu.com/QATeam/Cadence/Raring/Week7UbuntuGlobalJam|Global Jam testcases]]
 * [[http://qa.ubuntu.com/getting-involved/|Getting Involved in quality]]
 * [[http://iso.qa.ubuntu.com/ | The Iso Tracker.]]
 * [[http://packages.qa.ubuntu.com/ | The Package Tracker.]]
 * [[http://laptop.qa.ubuntu.com/ | The Hardware Tracker.]]
 * [[Testing/VirtualBox | Using VirtualBox]]

==== Getting Help ====
Get help on [[http://webchat.freenode.net/?randomnick=1&channels=ubuntu-quality|#ubuntu-quality]] on freenode.


== Holding the Session ==
Provide an overview of each type of testing and let folks know they can choose to do any or all of the different types during the jam. [[https://wiki.ubuntu.com/Jams/Testing#Testing_Descriptions|See the testing descriptions]] on the second half of this page; feel free to provide the descriptions and links to folks who attend the session.

Milestones will be setup on all three trackers, a summary of what all can be tested can be found on [[https://wiki.ubuntu.com/QATeam/Cadence/Raring/Week7UbuntuGlobalJam|this milestone page]].

If folks are interested in contributing testcases instead of results, direct them to the [[https://wiki.ubuntu.com/QATeam/ContributingTestcases|resources found on the QATeam wiki]]. In addition, refer them to the applicable launchpad project as needed.

[[https://launchpad.net/ubuntu-manual-tests|Ubuntu Manual Tests]]

[[https://launchpad.net/ubuntu-autopilot-tests/|Ubuntu Autopilot Tests]]

= Testing Descriptions =
Line 8: Line 50:
 1. The testing of the installer. This covers Ubiquity (the live cd installer), Debian-installer (the alternate installer) and wubi.
 1. The testing of applications. This is done by everyone who installs the latest developer version of *buntu.
 1. Contributing Results
 1. Contributing Tests
Line 11: Line 53:
Most people test these in a virtual environment although by the time beta comes along a lot of user will now be testing this on real hardware too. The following descriptions can be used to help people decide which area they wish to contribute to.
Line 13: Line 55:
I will break down how to test the above in the sections below and describe the usage of the iso test tracker for those that decide to participate. == Contributing Results ==
Line 15: Line 57:
= Requirements = Most users will likely be interested in contributing test results for ubuntu during the jam. There are 3 different types of results contributions that can be made.
Line 17: Line 59:
A computer.<<BR>>
A Launchpad Account.<<BR>>
A current CD or ISO.<<BR>>
A network connection.<<BR>>
Check if your computer has a cpu with added virtualization:
{{{
egrep '(vmx|svm)' /proc/cpuinfo
}}}
Finally a vm KVM/Qemu/Virtualbox, unless installing on hardware.
 1. Image Testing
 1. Application Testing
 1. Hardware Testing
Line 27: Line 63:
== Asking people to prepare for the Jam == === Image Testing ===
Line 29: Line 65:
Grabbing a copy of the latest iso.<<BR>>
Check All the above to ensure you can use vm.<<BR>>
Install your chosen vm, unless running on hardware.
Image or iso testing consists of downloading a copy of the latest daily ISO images, burning them to CDs/USB keys (or loading them into VM's) and running through an installation with them.
Line 33: Line 67:
=== Material === What's tested?:
Iso image
ubiquity / debian installer
Line 35: Line 71:
Important links:<<BR>>
[[http://iso.qa.ubuntu.com/ | The Iso Tracker.]]<<BR>>
[[https://wiki.ubuntu.com/Testing/VirtualBox | Using VirtualBox (older version but works pretty much the same now).]]<<BR>>
[[https://wiki.ubuntu.com/Testing/Automation/KVM | Towards the bottom half of the page gives you the general idea for a kvm command.]]<<BR>>
[[http://testcases.qa.ubuntu.com/ | Links to the testcases.]]<<BR>>
The Example-content package if not running on Ubuntu.
Requirements:
 * Launchpad/Ubuntu SSO account
 * Copy of the iso/image you wish to test
Line 42: Line 75:
= Holding the Session =
== Introduction Session ==
Tutorials:
[[https://wiki.ubuntu.com/Testing/ISO/Walkthrough|ISO Testing walkthrough]]
Line 45: Line 78:
=== Install Testing ===

'''TEST TRACKER'''

First things first lets split the team between hardware installed/installing and vm users.<<BR>>
Next for those new to the process lets set up a tracker account. Goto http://iso.qa.ubuntu.com click on '''Log In''' then on '''Create New Account''' fill in your details and then click on '''ISO tracker'''.<<BR>>
Now you have access to the tracker you can select the type of install you wish to perform. Start with the iso type you have and then the partitioning type.

'''On The Tracker'''

Now you have selected your install type you can goto the '''Additional instructions are available for this testcase, click here to read them.''' this will then tell you the steps to run through to install using your selected method.<<BR>>
Once you complete the install please mark if the test passed or failed and any bugs you have found by adding the bug number to the boxes provided.<<BR>>
This helps the developers and release team fix issue for the next release.<<BR>>
Once the install is complete please move on to the '''Application/System testing'''.

=== Application/System Testing ===

'''The easy bit'''

Goto System->Administration->System Testing. Run through the application filling in the yes no questions at the end you'll be asked for your LaunchPad id and password. This is because the information about you machine is stored in your Launchpad account at '''https://edge.launchpad.net/~yourusername/+hwdb-submissions'''. This is especially useful if you hit hardware regression as you can add you HW id into your bug report.

'''The longer bit'''

Okay so the system testing is out of the way now it's time to start on the main application testing.<<BR>>
First off if your not running Ubuntu install the example-content package. This gives you example files to test applications for media, documents and pictures.<<BR>>
Secondly start running through the applications one at a time till you're happy they are working. For the most part this will involve opening the application, running the application, closing the application. For examples on what to test goto http://testcases.qa.ubuntu.com/Applications/ this currently covers most of the default installed applications.
Testcases to execute:
[[https://wiki.ubuntu.com/QATeam/Cadence/Raring/Week7UbuntuGlobalJam|Global Jam Testcases]]
Line 73: Line 82:
== Workflow == === Application Testing ===
Line 75: Line 84:
== Tools & Helpful Information == Application testing is the manual testing of specific things (test cases) in applications.
Line 77: Line 86:
Get help on [[http://webchat.freenode.net/?randomnick=1&channels=ubuntu-testing|#ubuntu-testing]]. What's tested?:
Specific ubuntu packages
Line 79: Line 89:
Requirements:
 * Launchpad/Ubuntu SSO account
 * Development version of ubuntu (installed or via live session from daily image)
Line 80: Line 93:
Tutorials:
[[https://wiki.ubuntu.com/Testing/Cadence/Walkthrough|Cadence Testing Walkthrough]]

''Note, the ubuntu global milestone is incorporated with the normal cadence testing; hence the walkthrough is valid for the event''

Testcases to execute:
[[https://wiki.ubuntu.com/QATeam/Cadence/Raring/Week7UbuntuGlobalJam|Global Jam Testcases]]

=== Hardware Testing ===

Hardware Testing is about the manual testing of specific things (test cases) mainly related to laptop hardware. Smoke tests are used to determine hardware compatibility for the system being tested.

What's tested?:
Hardware compatibility with ubuntu

Requirements:
 * Launchpad/Ubuntu SSO account
 * Development version of ubuntu (installed or via live session from daily image)
 * Laptop/Hardware for testing

Tutorials:
[[https://wiki.ubuntu.com/Testing/Laptop/Procedures|Laptop Testing Walkthrough]]

Testcases to execute:
[[https://wiki.ubuntu.com/QATeam/Cadence/Raring/Week7UbuntuGlobalJam|Global Jam Testcases]]

== Contributing Testcases ==

For contributing testcase, there are 3 different types that can be contributed.

 1. Autopilot
 1. Autopkg
 1. Manual

=== Autopilot ===
Autopilot is a testing tool capable of introspecting applications and generating keyboard and mouse events. Autopilot tests are utilized for functional testing, including GUI testing and simulating end user interaction.

[[https://wiki.ubuntu.com/QATeam/ContributingTestcases/Autopilot|Autopilot Tutorial]]

[[https://bugs.launchpad.net/ubuntu-autopilot-tests/+bugs?field.tag=todo|Testcases to contribute]]

[[https://launchpad.net/ubuntu-autopilot-tests/|Ubuntu Autopilot Tests Project]]

=== Autopkg ===
Autopkg tests are run at build time automatically by the buildbots for the package. The goal of these tests is to provide system and integration testing to guarantee basic functionality.

[[http://developer.ubuntu.com/packaging/html/auto-pkg-test.html|Autopkg Tutorial]]

[[https://wiki.ubuntu.com/QATeam/RequiredTests|Testcases to contribute]]

=== Manual ===
Manual Testcases are simply sets of instructions designed to be followed and reported against by real people.

[[https://wiki.ubuntu.com/QATeam/ContributingTestcases/Manual|Manual Testcase Tutorial]]

[[https://bugs.launchpad.net/ubuntu-manual-tests/+bugs?field.tag=todo|Testcases to contribute]]

[[https://launchpad.net/ubuntu-manual-tests|Ubuntu Manual Tests Project]]

Preparing for the Session

You are encouraged to familiarize yourself with each type of testing in advance so you have a basic understanding of what to expect. In addition, it's encouraged for you to read the following pages:

https://wiki.ubuntu.com/Testing/QATracker

https://wiki.ubuntu.com/Testing/Activities

https://wiki.ubuntu.com/QATeam/ContributingTestcases

For each of the different types of testing a ubuntu global jam milestone should be setup on each tracker with the testcases ready to and and results ready to be reported against. See the tests on this milestone page

Note that unless folks are willing to do installations (via VM or real hardware) they will be testing using the live session of the image they brought with them (by booting it up into a VM or on real hardware).

Asking people to prepare for the Jam

Have participants choose what ways they would like to contribute. In general for contributing test results, they should plan to bring a copy of the latest daily image, or provide an up-to-date installation of the development version of ubuntu.

Getting Help

Get help on #ubuntu-quality on freenode.

Holding the Session

Provide an overview of each type of testing and let folks know they can choose to do any or all of the different types during the jam. See the testing descriptions on the second half of this page; feel free to provide the descriptions and links to folks who attend the session.

Milestones will be setup on all three trackers, a summary of what all can be tested can be found on this milestone page.

If folks are interested in contributing testcases instead of results, direct them to the resources found on the QATeam wiki. In addition, refer them to the applicable launchpad project as needed.

Ubuntu Manual Tests

Ubuntu Autopilot Tests

Testing Descriptions

Testing is split into 2 main parts.

  1. Contributing Results
  2. Contributing Tests

The following descriptions can be used to help people decide which area they wish to contribute to.

Contributing Results

Most users will likely be interested in contributing test results for ubuntu during the jam. There are 3 different types of results contributions that can be made.

  1. Image Testing
  2. Application Testing
  3. Hardware Testing

Image Testing

Image or iso testing consists of downloading a copy of the latest daily ISO images, burning them to CDs/USB keys (or loading them into VM's) and running through an installation with them.

What's tested?: Iso image ubiquity / debian installer

Requirements:

  • Launchpad/Ubuntu SSO account
  • Copy of the iso/image you wish to test

Tutorials: ISO Testing walkthrough

Testcases to execute: Global Jam Testcases

Application Testing

Application testing is the manual testing of specific things (test cases) in applications.

What's tested?: Specific ubuntu packages

Requirements:

  • Launchpad/Ubuntu SSO account
  • Development version of ubuntu (installed or via live session from daily image)

Tutorials: Cadence Testing Walkthrough

Note, the ubuntu global milestone is incorporated with the normal cadence testing; hence the walkthrough is valid for the event

Testcases to execute: Global Jam Testcases

Hardware Testing

Hardware Testing is about the manual testing of specific things (test cases) mainly related to laptop hardware. Smoke tests are used to determine hardware compatibility for the system being tested.

What's tested?: Hardware compatibility with ubuntu

Requirements:

  • Launchpad/Ubuntu SSO account
  • Development version of ubuntu (installed or via live session from daily image)
  • Laptop/Hardware for testing

Tutorials: Laptop Testing Walkthrough

Testcases to execute: Global Jam Testcases

Contributing Testcases

For contributing testcase, there are 3 different types that can be contributed.

  1. Autopilot
  2. Autopkg
  3. Manual

Autopilot

Autopilot is a testing tool capable of introspecting applications and generating keyboard and mouse events. Autopilot tests are utilized for functional testing, including GUI testing and simulating end user interaction.

Autopilot Tutorial

Testcases to contribute

Ubuntu Autopilot Tests Project

Autopkg

Autopkg tests are run at build time automatically by the buildbots for the package. The goal of these tests is to provide system and integration testing to guarantee basic functionality.

Autopkg Tutorial

Testcases to contribute

Manual

Manual Testcases are simply sets of instructions designed to be followed and reported against by real people.

Manual Testcase Tutorial

Testcases to contribute

Ubuntu Manual Tests Project


CategoryJam

Jams/Testing (last edited 2015-07-28 21:25:13 by nskaggs)