TestingProcessesUbuntuPersonal

Differences between revisions 8 and 9
Revision 8 as of 2015-06-29 09:11:25
Size: 3480
Editor: davmor2
Comment:
Revision 9 as of 2015-06-29 12:07:13
Size: 3844
Editor: davmor2
Comment:
Deletions are marked like this. Additions are marked like this.
Line 17: Line 17:
 * Flash the phone with the latest development image.  * Flash the phone with the latest development image (instructions [[https://wiki.ubuntu.com/QATeam/ChannelsToFlash#Image_Testing_Channels|here]] )
Line 44: Line 44:
== Device Tarball Testing == == Tarball Testing ==
Line 46: Line 46:
Tarball entries are found [[https://docs.google.com/spreadsheets/d/1LiPHq39XvPaTQoE8vmi-d#!/bin/bash
qSVMTGGLfCh7bFY01BcNNk/edit#gid=50|here]] <<BR>>
Device tarballs will require downloading locally and the following script: <<BR>>
{{{
adb shell mkdir -p /etc/system-image
adb shell touch /etc/system-image/skip-gpg-verification
adb shell system-image-upgrader /cache/recovery/ubuntu_command
adb reboot
}}}
Images are scheduled [[https://docs.google.com/spreadsheets/d/1LiPHq39XvPaTQoE8vmi-dqSVMTGGLfCh7bFY01BcNNk/edit#gid=50|here]]
Line 56: Line 48:
For Custom Tarballs Read: https://wiki.ubuntu.com/QATeam/ChannelsToFlash#Testing_Custom_Tarballs <<BR>>
For Device Tarballs Read: https://wiki.ubuntu.com/QATeam/ChannelsToFlash#Testing_Device_Tarballs
Line 57: Line 51:
See Sanity Testing Below for device tarballs
For Custom Tarballs look at practitest (cases need adding here)
Line 58: Line 54:
== Custom Tarball Testing ==
=== Prechecks ===
Tarball entries are found [[https://docs.google.com/spreadsheets/d/1LiPHq39XvPaTQoE8vmi-dqSVMTGGLfCh7bFY01BcNNk/edit#gid=50|here]]
=== Installation ===
=== Test Plan Locations ===
=== Reporting New Cases ===
All new cases must be added to the "To Review" Queue in practitest
Line 68: Line 59:
 * Follow the instructions [[https://wiki.ubuntu.com/QATeam/ChannelsToFlash#Image_Testing_Channels|here]] to get the latest image install
 * Then run the sanity suite
Line 69: Line 62:
Sanity test are located in practitest currently
Line 75: Line 69:
 * Follow the instructions [[https://wiki.ubuntu.com/QATeam/ChannelsToFlash#Image_Testing_Channels|here]] to get the latest image install
 * Then run the regression suite
Line 76: Line 72:
Regression tests are located in Practitest Currently

WIP

There are a lot of new terms, tools and procedures we want to use for Testing on Ubuntu Personal. This page is here to document those processes.

Silo Testing

Prechecks

  • All merge proposals must have been reviewed and approved.
  • All automated tests must pass.
  • New features must come with automated tests.
  • New strings must be marked for translation.

Installation

Install phablet-tools package in your desktop machine.

  • Flash the phone with the latest development image (instructions here )

  • citrain device-upgrade <silo number> <password>

  • Overlay PPA
    • In case of Vivid + Overlay PPA, the silo packages need to be pinned to a higher priority. Otherwise, silo packages won't be installed.
    • Create this file: sudo vi /etc/apt/preferences.d/silo.pref

    • Add these lines (silo number must be 0-padded, e.g. landing-001):

Package: *
Pin: release o=LP-PPA-ci-train-ppa-service-landing-<silo number>
Pin-Priority: 1002
  • Save the file.
  • Update the package list: sudo apt-get update

  • Upgrade the packages: sudo apt-get upgrade

Test Plan Locations

Reporting New Cases

Any relevant silo bug should be added to the Regression Test Suite as draft.

More Info

More info can be found here

Tarball Testing

Prechecks

Images are scheduled here

Installation

For Custom Tarballs Read: https://wiki.ubuntu.com/QATeam/ChannelsToFlash#Testing_Custom_Tarballs
For Device Tarballs Read: https://wiki.ubuntu.com/QATeam/ChannelsToFlash#Testing_Device_Tarballs

Test Plan Locations

See Sanity Testing Below for device tarballs For Custom Tarballs look at practitest (cases need adding here)

Reporting New Cases

All new cases must be added to the "To Review" Queue in practitest

Sanity Testing

Prechecks

All silos are landed and all tarballs are landed.

Installation

  • Follow the instructions here to get the latest image install

  • Then run the sanity suite

Test Plan Locations

Sanity test are located in practitest currently

Reporting New Cases

Sanity is a deliberately selected small subset of tests targeting software interactions with hardware. Any additional tests felt required should be discussed with the entire team via the mailing list or standups.

Regression Testing

Prechecks

Sanity Must of been run to ensure the image is at least testable

Installation

  • Follow the instructions here to get the latest image install

  • Then run the regression suite

Test Plan Locations

Regression tests are located in Practitest Currently

Reporting New Cases

New Cases should not be found while testing regression, however you may want make a note here(add a link later) to remind you to look at something during exploratory testing.

Exploratory Testing

Prechecks

Installation

Recording Test and Session

Reporting New Cases

If there is automation required here it should be reported as a bug for the development team.
If there are new manual cases they should be put into draft for review in practitest.

Automated Testing

Prechecks

Installation

Test Plan Locations

Reporting Of Failures

QATeam/TestingProcessesUbuntuPersonal (last edited 2015-12-03 13:56:01 by davmor2)