Testing
|
Size: 5274
Comment:
|
Size: 3821
Comment:
|
| Deletions are marked like this. | Additions are marked like this. |
| Line 4: | Line 4: |
| Testing is split into 2 main parts. | Testing is split into 3 main parts. |
| Line 6: | Line 6: |
| 1. The testing of the installer. This covers Ubiquity (the live cd installer), Debian-installer (the alternate installer) and wubi. 1. The testing of applications. This is done by everyone who installs the latest developer version of *buntu. |
1. Image Testing. 1. Application Testing 1. Hardware Testing |
| Line 9: | Line 10: |
| Most people test these in a virtual environment although by the time beta comes along a lot of user will now be testing this on real hardware too. | = Testing Descriptions = The following descriptions can be used to help people decide how they want to participate. |
| Line 11: | Line 13: |
| I will break down how to test the above in the sections below and describe the usage of the iso test tracker for those that decide to participate. | == Image Testing == Image or iso testing consists of downloading a copy of the latest daily ISO images, burning them to CDs/USB keys (or loading them into VM's) and testing them. This brings to light many issues that might have been missed by other early adopters and developers, especially in the CD builds and installers. == Application Testing == Application testing is the manual testing of specific things (test cases) in applications. Regression tests are specific tests for potential breakages from one release to another (they're also relevant for SRU testing, above). == Hardware Testing == Hardware Testing is about the manual testing of specific things (test cases) mainly related to laptops hardware. The goal is to get Ubuntu to work great on as many different makes and models of laptops as possible and this can be done knowing which hardware works straight off the install CD and which hardware needs configuring or is poorly supported. |
| Line 17: | Line 29: |
| A current CD or ISO.<<BR>> | An image of the current daily development version of ubuntu<<BR>> |
| Line 19: | Line 31: |
| Check if your computer has a cpu with added virtualization extensions. If the cpu line comes back with vmx or svm then you do. Which means you be able to use kvm instead of qemu. {{{ egrep '(vmx|svm)' /proc/cpuinfo }}} Finally a vm KVM/Qemu/Virtualbox, unless installing on hardware. |
Finally a virtual machine, such as KVM/Qemu/Virtualbox, unless installing on hardware. |
| Line 27: | Line 35: |
| Grabbing a copy of the latest iso.<<BR>> Check All the above to ensure you can use vm.<<BR>> Install your chosen vm, unless running on hardware. |
Download a [[http://cdimage.ubuntu.com/daily-live/current/|copy of the latest iso]]<<BR>> |
| Line 35: | Line 41: |
| [[http://packages.qa.ubuntu.com/ | The Package Tracker.]]<<BR>> [[http://laptop.qa.ubuntu.com/ | The Hardware Tracker.]]<<BR>> |
|
| Line 37: | Line 45: |
| [[http://testcases.qa.ubuntu.com/ | Links to the testcases.]]<<BR>> The Example-content package if not running on Ubuntu. |
|
| Line 41: | Line 47: |
| You are encouraged to perform each type of testing in advance so you have a basic understanding of what to expect. In addition, it's encouraged for you to read the following pages: https://wiki.ubuntu.com/Testing/QATracker https://wiki.ubuntu.com/Testing/Activities For each of the different types of testing a ubuntu global jam milestone should be setup on each tracker with the testcases ready to and and results ready to be reported against. Note that unless folks are willing to do installations (via VM or real hardware) they will be testing using the live session of the image they brought with them (by booting it up into a VM or on real hardware) |
|
| Line 42: | Line 58: |
| Provide an overview of each type of testing and let folks know they can choose to do any or all of the different types during the jam. | |
| Line 43: | Line 60: |
| === Application/System Testing === '''The easy bit''' Launch the "System Testing" tool: Click on the Ubuntu logo or press the Super key, enter 'checkbox' in the search field then click on the icon 'System Testing' Run through the application filling in the yes no questions at the end you'll be asked for your LaunchPad id and password. This is because the information about you machine is stored in your Launchpad account at '''https://launchpad.net/people/+me/+hwdb-submissions'''. This is especially useful if you hit hardware regression as you can add you HW id into your bug report. [[https://wiki.ubuntu.com/Jams/Testing?action=AttachFile&do=get&target=st.png | {{https://wiki.ubuntu.com/Jams/Testing?action=AttachFile&do=get&target=st.png}}]] [[https://wiki.ubuntu.com/Jams/Testing?action=AttachFile&do=get&target=stk.png | {{https://wiki.ubuntu.com/Jams/Testing?action=AttachFile&do=get&target=stk.png}}]] [[https://wiki.ubuntu.com/Jams/Testing?action=AttachFile&do=get&target=ffhw.png|{{https://wiki.ubuntu.com/Jams/Testing?action=AttachFile&do=get&target=ffhw-tn.png}}]] '''The longer bit''' Okay so the system testing is out of the way now it's time to start on the main application testing.<<BR>> First off if your not running Ubuntu install the example-content package. This gives you example files to test applications for media, documents and pictures.<<BR>> Secondly start running through the applications one at a time till you're happy they are working. For the most part this will involve opening the application, running the application, closing the application. For examples on what to test goto http://testcases.qa.ubuntu.com/Applications/ this currently covers most of the default installed applications. |
=== Application Testing === NOTE: Ubuntu global jam is a milestone, just like cadence testing. Thus the tutorial can be followed. [[https://wiki.ubuntu.com/Testing/Cadence/Walkthrough|https://wiki.ubuntu.com/Testing/Cadence/Walkthrough]] |
| Line 62: | Line 65: |
| Follow the guide here: [[https://wiki.ubuntu.com/Testing/ISO/Walkthrough|https://wiki.ubuntu.com/Testing/ISO/Walkthrough]] |
|
| Line 63: | Line 68: |
| '''TEST TRACKER''' First things first lets split the team between hardware installed/installing and vm users.<<BR>> Next for those new to the process lets set up a tracker account. Goto http://iso.qa.ubuntu.com click on '''Log In''' then on '''Create New Account''' fill in your details and then click on '''ISO tracker'''.<<BR>> Now you have access to the tracker you can select the type of install you wish to perform. Start with the iso type you have and then the partitioning type. [[https://wiki.ubuntu.com/Jams/Testing?action=AttachFile&do=get&target=fft.png|{{https://wiki.ubuntu.com/Jams/Testing?action=AttachFile&do=get&target=fft-tn.png}}]] '''On The Tracker''' Now you have selected your install type you can goto the '''Additional instructions are available for this testcase, click here to read them.''' this will then tell you the steps to run through to install using your selected method.<<BR>> Once you complete the install please mark if the test passed or failed and any bugs you have found by adding the bug number to the boxes provided.<<BR>> This helps the developers and release team fix issue for the next release.<<BR>> Once the install is complete please move on to the '''Application/System testing'''. [[https://wiki.ubuntu.com/Jams/Testing?action=AttachFile&do=get&target=fft1.png|{{https://wiki.ubuntu.com/Jams/Testing?action=AttachFile&do=get&target=fft1-tn.png}}]] == Workflow == |
=== Hardware Testing === Boot your hardware up into a Live session, then follow: [[https://wiki.ubuntu.com/Testing/Laptop/Procedures|https://wiki.ubuntu.com/Testing/Laptop/Procedures]] |
| Line 84: | Line 74: |
| Get help on [[http://webchat.freenode.net/?randomnick=1&channels=ubuntu-testing|#ubuntu-testing]]. |
Get help on [[http://webchat.freenode.net/?randomnick=1&channels=ubuntu-quality|#ubuntu-quality]]. |
Testing is split into 3 main parts.
- Image Testing.
- Application Testing
- Hardware Testing
Testing Descriptions
The following descriptions can be used to help people decide how they want to participate.
Image Testing
Image or iso testing consists of downloading a copy of the latest daily ISO images, burning them to CDs/USB keys (or loading them into VM's) and testing them. This brings to light many issues that might have been missed by other early adopters and developers, especially in the CD builds and installers.
Application Testing
Application testing is the manual testing of specific things (test cases) in applications. Regression tests are specific tests for potential breakages from one release to another (they're also relevant for SRU testing, above).
Hardware Testing
Hardware Testing is about the manual testing of specific things (test cases) mainly related to laptops hardware. The goal is to get Ubuntu to work great on as many different makes and models of laptops as possible and this can be done knowing which hardware works straight off the install CD and which hardware needs configuring or is poorly supported.
Requirements
A computer.
A Launchpad Account.
An image of the current daily development version of ubuntu
A network connection.
Finally a virtual machine, such as KVM/Qemu/Virtualbox, unless installing on hardware.
Asking people to prepare for the Jam
Download a copy of the latest iso
Material
Important links:
The Iso Tracker.
The Package Tracker.
The Hardware Tracker.
Using VirtualBox (older version but works pretty much the same now).
Towards the bottom half of the page gives you the general idea for a kvm command.
Holding the Session
You are encouraged to perform each type of testing in advance so you have a basic understanding of what to expect. In addition, it's encouraged for you to read the following pages:
https://wiki.ubuntu.com/Testing/QATracker
https://wiki.ubuntu.com/Testing/Activities
For each of the different types of testing a ubuntu global jam milestone should be setup on each tracker with the testcases ready to and and results ready to be reported against.
Note that unless folks are willing to do installations (via VM or real hardware) they will be testing using the live session of the image they brought with them (by booting it up into a VM or on real hardware)
Introduction Session
Provide an overview of each type of testing and let folks know they can choose to do any or all of the different types during the jam.
Application Testing
NOTE: Ubuntu global jam is a milestone, just like cadence testing. Thus the tutorial can be followed. https://wiki.ubuntu.com/Testing/Cadence/Walkthrough
Install Testing
Follow the guide here: https://wiki.ubuntu.com/Testing/ISO/Walkthrough
Hardware Testing
Boot your hardware up into a Live session, then follow: https://wiki.ubuntu.com/Testing/Laptop/Procedures
Tools & Helpful Information
Get help on #ubuntu-quality.
Jams/Testing (last edited 2015-07-28 21:25:13 by 1)