QANewStarterNotes

These are notes taken during the first months of work in the QA team. Feel free to comment, correct or add your own.

Public

Development release

You can change the release to devel in /etc/apt/sources.list in order to have a rolling development release (similar to debian's sid). Use with care! Smile :)

Bootstrapping the development environment

This is my list of pkgs to begin with: 

$ sudo add-apt-repository -y ppa:elopio/uci
$ sudo add-apt-repository -y ppa:thomir/trv
$ sudo apt-get update
$ sudo apt-get install -y -q bzr git autopkgtest phablet-tools python-virtualenv unrar ubuntu-sdk python3-flake8 qbzr subunit python3-uci-config python3-uci-tests trv sbuild ubuntu-dev-tools

Autopilot

The source code

The docs

More docs

Source code repositories

Autopkgtest

Autopkgtest in Launchpad (useful for bugs, VERY responsive and nice :))

Ubuntu Sanity Tests

Ubuntu OTA Tests

Ubuntu Power Tests

Unity8

Android tools

Ubuntu Touch Cli Tools

Phablet Tools

Ubuntu UI Toolkit

Ubuntu System Settings

Ubuntu System Image

Unity Greeter

Webbrowser app

Address Book app

QAKit

Packaging and building

Debian

Ubuntu

PPAs

Cross Building

Testbeds

First steps with the device

Set up developer mode (Apps -> System Settings -> About this phone -> Developer mode)

Make sure that the developer mode is kept when the screen is turned off: 

    phablet-shell

    sudo touch /userdata/.adb_onlock # root password is 0000

Phone management commands

Get a list of connected devices with serial numbers: adb devices

The following commands are can be used when a single device is connected, if not the serial numbers obtained before can be used.

Rebooting: adb reboot

Opening a shell: phablet-shell (the default root password is 0000)

Network provision: phablet-network -n /etc/NetworkManager/system-connections/<connection-file>

Make the image writable: phablet-config writable-image (or from a device shell, sudo mount -o remount,rw /)

Using the emulator

Documentation

Executing the sanity suite (or part of it) in the testbed

Method 1: readonly image

In order to get quick feedback from tests run in the device is usually better to execute the desired tests from inside the testbed instead of launching them from the host (and preventing then all the provisioning steps taken by adt-run).

For getting an environment as close as possible to the one created by the sanity suite we will run adt-run with a --shell option, which at the end of the test keeps a shell open. This shell is not very useful and the logged user is root, but all the temporary directories used for the test run are not removed. With this shell open we access the device from another shell, navigate to the temp directories and then execute the tests.

These are the required steps:

Flash the device: 

    $ ubuntu-device-flash touch --developer-mode --password 0000 --channel="ubuntu-touch/devel-proposed" --wipe

Provision the network in the device: 

    $ phablet-network -n /etc/NetworkManager/system-connections/<connection-file>

Modify the sanity configuration to execute only one test, this way we can get the first device shell earlier. To accomplish this, edit ~/.config/ubuntu-sanity-tests.conf and modify the tests_to_run line: 

    tests_to_run = ubuntu_sanity_tests.tests.test_add_contact

From sanity's source root in the host execute adt-run: 

    $ adt-run --timeout-short=600 -B --unbuilt-tree=$PWD --output-dir ~/workspace/ust --copy ~/.config/ubuntu-sanity-tests.conf:/home/phablet/.config/ubuntu-sanity-tests.conf --setup-commands 'mount -o remount,rw /' --setup-commands 'apt-add-repository -y ppa:canonical-platform-qa/selenium' --setup-commands 'apt-get --no-list-cleanup update -o Dir::Etc::SourceList=/dev/null' --setup-commands 'sync; sleep 2; mount -o remount,ro /' --shell --- ssh -s adb -- -p 0000

After finishing the execution, there's an open shell with a prompt like: 

    root@ubuntu-phablet:/tmp/adt-run.l5qRDX/build.pvJ/real-tree#

We will use that /tmp/adt-run.l5qRDX/build.pvJ/real-tree path later, the source code is in that directory and in /tmp/adt-run.l5qRDX is all that is required to execute the tests. Open another shell to the device from the host: 

    $ phablet-shell

And we can execute the tests in the device from the temporary directory with: 

    $ cd /tmp/adt-run.l5qRDX/build.pvJ/real-tree
    $ PYTHONPATH=.:../../deps/usr/lib/python3/dist-packages/ ADT_ARTIFACTS=~/ust-out/ debian/tests/sanity

Method 2: writable image

In this case we are going to work with the image in writable mode, but with the dependencies installed preventing the upgrade of existing packages. The environment is not so similar to the one setup by adt-run, but the process is perhaps simpler, although a complete run on a recently flashed device after the changes are pulled is always required, to verify that all will work in the real environment. It doesn't hurt to do this also with the previous approach, but the results are more likely to be the same.

With the device connected and from the source code branch, these would be the steps to take:

Flash the device: 

    $ ubuntu-device-flash touch --developer-mode --password 0000 --channel="ubuntu-touch/devel-proposed" --wipe

Push the sanity suite code to the device: 

    $ adb push . /home/phablet/ust

Provision the network in the device: 

    $ phablet-network -n /etc/NetworkManager/system-connections/<connection-file>

Log into the device 

    $ phablet-shell

From this point on all the commands should be executed from the device shell.

Put the image in writable mode: 

    $ sudo mount -o remount,rw /

Install the required dependencies: 

    $ sudo apt-add-repository -y ppa:canonical-platform-qa/selenium
    $ sudo apt-get install -y grep-dctrl
    $ grep-dctrl Depends -s Depends -n ./debian/tests/control | sed ':a;N;$!ba;s/\n/ /g' | sed 's/,\s*/ /g' | sed -r 's/ \(>=([a-z0-9\+\-].?)*\) / /g' | sed -r 's/ \| ([a-z0-9\.\-])* / /' | xargs sudo apt-get install -y --no-upgrade

Execute the tests: 

    $ cd /home/phablet/ust 
    $ PYTHONPATH=. ADT_ARTIFACTS=~/ust-out/ debian/tests/sanity

*IMPORTANT* Remember to execute the full suite from the host after making your changes to verify that all goes fine running the tests through the adt-run wrapper with the image in readonly mode, from the host: 

    $ python3 -m ubuntu_sanity_tests.command_line

Executing generic autopilot suites in the testbed

For example, we want to execute the address-book-app autopilot tests in the device. From the project source we push the files to the device: 

    adb push . /home/phablet/address-book-app

Open a shell in the device: 

    phablet-shell

Make the image writable 

    sudo mount -o remount,rw /

Install the dependencies. This is done installing the related package, if the dependencies are changed in source then the new dependencies should be installed manually: 

    sudo apt-get install address-book-app-autopilot python3-evdev

Execute the tests, in this case before actually executing the tests you should have the phone unlocked: 

    cd /home/phablet/address-book-app/tests/autopilot && autopilot3 run address_book_app

Landing

Team page

Process - CI Train

DBus

Documentation

Python tutorial

Private

IRC Channels

Canonical

##qa

#barajas

#canonical

#phablet

#qa

#unity

Freenode

#ubuntu-autopilot

#ubuntu-ci-eng

#ubuntu-quality

#ubuntu-touch

IRC Highlighting

Some keywords to start with:

* ubuntu-qa

* project-team

* projects-team

* qa-team

Paste utilities

There are http://paste.ubuntu.com and http://paste.canonical.com, the latter for sensible data

Setting up the VPN

This is the wiki page, always is useful to contact IS through rt@admin.canonical.com

QANewStarterNotes (last edited 2015-05-14 09:44:03 by fgimenez)