Testing

Differences between revisions 33 and 34
Revision 33 as of 2015-01-22 15:36:15
Size: 6573
Editor: dholbach
Comment:
Revision 34 as of 2015-01-22 22:04:30
Size: 3727
Editor: nskaggs
Comment:
Deletions are marked like this. Additions are marked like this.
Line 5: Line 5:
You are encouraged to familiarize yourself with each type of testing in advance so you have a basic understanding of what to expect. In addition, it's encouraged for you to read the following pages: You are encouraged to familiarize yourself with performing the testing in advance so you have a basic understanding of what to expect. This is best done by performing a test yourself before the day of the jam.
Line 7: Line 7:
 * https://wiki.ubuntu.com/Testing/QATracker
 * https://wiki.ubuntu.com/Testing/Activities
 * https://wiki.ubuntu.com/QATeam/ContributingTestcases


For each of the different types of testing a milestone is setup on each tracker with the testcases ready to and and results ready to be reported against.

Note that unless folks are willing to do installations (via VM or real hardware) they will be testing using the live session of the image they brought with them (by booting it up into a VM or on real hardware).
It is recommended you perform testing on the development version of ubuntu during the jam. To do this, you will be booting a live session of the development version of ubuntu and testing applications. Additionally you may perform an installation of the development version on real hardware or in a virtual machine.
Line 17: Line 10:
Have participants choose what ways they would like to contribute. In general for contributing test results, they should plan to bring a copy of the latest daily image, or provide an up-to-date installation of the development version of ubuntu. Have participants come prepared with a laptop and usb key (2 gb or greater) that can be wiped clean.
Line 19: Line 13:
 * Download a [[http://cdimage.ubuntu.com/daily-live/current/|copy of the latest image]]


==== Important links ====
 * [[http://iso.qa.ubuntu.com/ | The Iso Tracker.]]
 * [[http://packages.qa.ubuntu.com/ | The Package Tracker.]]
 * [[http://laptop.qa.ubuntu.com/ | The Hardware Tracker.]]
 * [[Testing/VirtualBox | Using VirtualBox]]

==== Getting Help ====
Get help on [[http://webchat.freenode.net/?randomnick=1&channels=ubuntu-quality|#ubuntu-quality]] on freenode.
 * Ask them to download a [[http://cdimage.ubuntu.com/daily-live/current/|copy of the latest image]]
Line 33: Line 16:
Let folks know they will be helping test there hardware and look for bugs in the development version of ubuntu.
Line 34: Line 18:

Provide an overview of each type of testing and let folks know they can choose to do any or all of the different types during the jam. [[https://wiki.ubuntu.com/Jams/Testing#Testing_Descriptions|See the testing descriptions]] on the second half of this page; feel free to provide the descriptions and links to folks who attend the session.


Milestones should be setup on all three trackers.

If folks are interested in contributing testcases instead of results, direct them to the [[https://wiki.ubuntu.com/QATeam/ContributingTestcases|resources found on the QATeam wiki]]. In addition, refer them to the applicable launchpad project as needed.

[[https://launchpad.net/ubuntu-manual-tests|Ubuntu Manual Tests]]

[[https://launchpad.net/ubuntu-autopilot-tests/|Ubuntu Autopilot Tests]]

= Testing Descriptions =

Testing is split into 2 main parts.

 1. Contributing Results
 1. Contributing Tests

The following descriptions can be used to help people decide which area they wish to contribute to.
== Setup ==
[[https://wiki.ubuntu.com/QATeam/DevelopmentSetup|Follow the guide]] to ensure each participant has valid a bootable usb stick. Should this cause issues, follow the guide for virtualbox and have them test in a virtual machine.
Line 56: Line 22:
If possible participants should try to test both the installation and the desktop / applications. If participants are unwilling or unable to test the installation, they can focus on testing the desktop instead inside a live session.
Line 57: Line 24:
Most users will likely be interested in contributing test results for ubuntu during the jam. There are 3 different types of results contributions that can be made. == Booting the image ==
Boot the image and be sure to select the option allowing you to try ubuntu/xubuntu/kubuntu/lubuntu, etc. This will boot into a live session of the default desktop.
Line 59: Line 27:
 1. Image Testing
 1. Application Testing
 1. Hardware Testing
Once booted into a live session, you can perform application testing.

=== Application Testing ===

Application testing is the manual testing of specific things (test cases) in applications.

What's tested?:
Specific packages
Testsuites should exist for each flavor

Tutorials:
[[https://wiki.ubuntu.com/Testing/Cadence/Walkthrough|Application Testing Walkthrough]]

Testcases to execute:
[[http://packages.qa.ubuntu.com/|Application Testcases]]

== Getting more advanced ==
Once you've completed testing the default desktop applications, it's time to try an install.
Line 65: Line 48:
Image or iso testing consists of downloading a copy of the latest daily ISO images, burning them to CDs/USB keys (or loading them into VM's) and running through an installation with them. Image or iso testing consists of running through an installation of ubuntu.
Line 70: Line 53:
Line 78: Line 60:

=== Application Testing ===

Application testing is the manual testing of specific things (test cases) in applications.

What's tested?:
Specific ubuntu packages

Tutorials:
[[https://wiki.ubuntu.com/Testing/Cadence/Walkthrough|Application Testing Walkthrough]]

''Note, the ubuntu global milestone is incorporated with the normal cadence testing; hence the walkthrough is valid for the event''

Testcases to execute:
[[http://packages.qa.ubuntu.com/|Application Testcases]]

=== Hardware Testing ===

Hardware Testing is about the manual testing of specific things (test cases) mainly related to laptop hardware. Smoke tests are used to determine hardware compatibility for the system being tested.

What's tested?:
Hardware compatibility with ubuntu

Requirements:

 * Laptop/Hardware for testing

Tutorials:
[[https://wiki.ubuntu.com/Testing/Laptop/Procedures|Laptop Testing Walkthrough]]

Testcases to execute:
[[laptop.qa.ubuntu.com|HardwareTestcases]]

== Contributing Testcases ==

For contributing testcase, there are 3 different types that can be contributed.

 1. Autopilot
 1. Autopkg
 1. Manual

=== Autopilot ===
Autopilot is a testing tool capable of introspecting applications and generating keyboard and mouse events. Autopilot tests are utilized for functional testing, including GUI testing and simulating end user interaction.

[[https://wiki.ubuntu.com/QATeam/ContributingTestcases/Autopilot|Autopilot Tutorial]]

[[https://bugs.launchpad.net/ubuntu-autopilot-tests/+bugs?field.tag=todo|Testcases to contribute]]

[[https://launchpad.net/ubuntu-autopilot-tests/|Ubuntu Autopilot Tests Project]]

=== Autopkg ===
Autopkg tests are run at build time automatically by the buildbots for the package. The goal of these tests is to provide system and integration testing to guarantee basic functionality.

[[http://packaging.ubuntu.com/html/auto-pkg-test.html|Autopkg Tutorial]]

[[https://wiki.ubuntu.com/QATeam/RequiredTests|Testcases to contribute]]

=== Manual ===
Manual Testcases are simply sets of instructions designed to be followed and reported against by real people.

[[https://wiki.ubuntu.com/QATeam/ContributingTestcases/Manual|Manual Testcase Tutorial]]

[[https://bugs.launchpad.net/ubuntu-manual-tests/+bugs?field.tag=todo|Testcases to contribute]]

[[https://launchpad.net/ubuntu-manual-tests|Ubuntu Manual Tests Project]]
Line 147: Line 63:
If you want to file bugs, best use the [[https://bugs.launchpad.net/ubuntu/+source/unity8-desktop-session|unity8-desktop-session]] package. Sometimes the bug will be in some other package, like `unity8` or `mir` or something else. Bugs will be reassigned though. If you want to file bugs, best use the [[https://bugs.launchpad.net/ubuntu/+source/unity8-desktop-session|unity8-desktop-session]] package. The unity8 team will reassign your bug if necessary, please file any issues you find!
Line 150: Line 66:


==== Important links ====
 * [[QATeam/DevelopmentSetup | Testing Setup]]
 * [[http://iso.qa.ubuntu.com/ | The Iso Tracker.]]
 * [[http://packages.qa.ubuntu.com/ | The Package Tracker.]]

==== Getting Help ====
Get help on [[http://webchat.freenode.net/?randomnick=1&channels=ubuntu-quality|#ubuntu-quality]] on freenode.

Preparing for the Session

You are encouraged to familiarize yourself with performing the testing in advance so you have a basic understanding of what to expect. This is best done by performing a test yourself before the day of the jam.

It is recommended you perform testing on the development version of ubuntu during the jam. To do this, you will be booting a live session of the development version of ubuntu and testing applications. Additionally you may perform an installation of the development version on real hardware or in a virtual machine.

Asking people to prepare for the Jam

Have participants come prepared with a laptop and usb key (2 gb or greater) that can be wiped clean.

Holding the Session

Let folks know they will be helping test there hardware and look for bugs in the development version of ubuntu.

Setup

Follow the guide to ensure each participant has valid a bootable usb stick. Should this cause issues, follow the guide for virtualbox and have them test in a virtual machine.

Contributing Results

If possible participants should try to test both the installation and the desktop / applications. If participants are unwilling or unable to test the installation, they can focus on testing the desktop instead inside a live session.

Booting the image

Boot the image and be sure to select the option allowing you to try ubuntu/xubuntu/kubuntu/lubuntu, etc. This will boot into a live session of the default desktop.

Once booted into a live session, you can perform application testing.

Application Testing

Application testing is the manual testing of specific things (test cases) in applications.

What's tested?: Specific packages Testsuites should exist for each flavor

Tutorials: Application Testing Walkthrough

Testcases to execute: Application Testcases

Getting more advanced

Once you've completed testing the default desktop applications, it's time to try an install.

Image Testing

Image or iso testing consists of running through an installation of ubuntu.

What's tested?: Iso image ubiquity / debian installer

Tutorials: ISO Testing walkthrough

Testcases to execute: Image Testcases

Other forms of testing

If you're adventurous and want to check out what the future of Ubuntu is going to hold, see Unity8Desktop for more information and help with testing Unity8.

If you want to file bugs, best use the unity8-desktop-session package. The unity8 team will reassign your bug if necessary, please file any issues you find!

A priority list can be seen here.

Getting Help

Get help on #ubuntu-quality on freenode.


CategoryJam

Jams/Testing (last edited 2015-07-28 21:25:13 by nskaggs)