ui-toolkit

Differences between revisions 9 and 10
⇤ ← Revision 9 as of 2014-09-09 05:51:27 →
Size: 2270
Editor: a88-114-196-56
Comment:
← Revision 10 as of 2014-09-11 06:29:43 → ⇥
Size: 3722
Editor: 74-95-113-201-Colorado
Comment: added process notes after getting the new setup to work
Deletions are marked like this. Additions are marked like this.
Line 69: Line 69:

== Suggested process ==

  * First, flash the latest image.
  * Boot into the image, go through the wizard, get rid of the welcome tutorial, and disable automatic updates.
  * adb shell mount -o remount,rw /
  * Set up click tests: phablet-click-test-setup (or, for RTM: phablet-click-test-setup --distribution=ubuntu-rtm --series=14.09 )
  * apt-get install python3-lxml
  * Install the silo. (add the PPA, disable all other package sources, apt-get update, apt-get dist-upgrade, disable the PPA, re-enable other package sources)
  * adb shell reboot
  * Grab the UITK source: bzr branch lp:ubuntu-ui-toolkit
  * cd ubuntu-ui-toolkit/tests/
  * mkdir results
  * Modify uitk_test_plan.sh as needed. Might need to change the variables at the top (particularly where it stores results), perhaps more depending on your specific needs.
  * time ./uitk_test_plan.sh -r
  * Save the overall output into results/summary.txt (or similar).
  * mv results results.silo

If anything failed (and it probably did), reflash without the silo then re-test the components which failed. Requires editing uitk_test_plan.sh again, to remove components which don't need re-testing. Then compare the results against the first run. If the silo has the same or fewer failures as the base image, it passes.

Note that several AP tests like to produce a different number of failures on each test run. This can greatly complicate the above process.

Test Plan

Install latest (utopic-proposed) image on phone. Install freshly build MPs that are needed for landing Ensure that all AP tests pass on the device what are marked OK on the CI dash.

  • - p reminders-app-autopilot reminders
  • -p dialer-app-autopilot dialer_app
  • -p messaging-app-autopilot messaging_app
  • -p ubuntu-system-settings-autopilot ubuntu_system_settings
  • -p ubuntu-html5-ui-toolkit-autopilot ubuntu_html5_ui_toolkit
  • -p unity-webapps-qml-autopilot unity_webapps_qml
  • filemanager
  • dropping_letters_app
  • sudoku_app
  • -p ubuntu-ui-toolkit-autopilot ubuntuuitoolkit
  • -p webbrowser-app-autopilot webbrowser_app
  • -n unity8
  • shorts_app
  • calendar_app
  • music_app
  • -p mediaplayer-app-autopilot mediaplayer_app
  • ubuntu_terminal_app
  • ubuntu_clock_app
  • ubuntu_calculator_app
  • ubuntu_weather_app
  • -p address-book-app-autopilot address_book_app
  • online_accounts_ui
  • gallery_app
  • -p camera-app-autopilot camera_app

Local Validation

$ bzr branch lp:ubuntu-ui-toolkit $ cd ubuntu-ui-toolkit/tests

The uitk_test_plan.sh is the script to execute the test plan

Usage

uitk_test_plan.sh -s [serial number] -m -c

  • -r : Reset after each tests.
  • -s: Use the device with the given serial number. The default is to use the first available adb device
  • -c : Comission the device with the PPA enabled
  • -n : Do not run the test set. Useful just to comission the device.
  • -o : Output directory.
  • -p : Source PPA for the UITK. Use -p archive to test stock image.
  • -f : Filter for the test suite.

Example

$./uitk_test_plan.sh -c -p ppa:ci-train-ppa-service/landing-010 -f music

  • This will flash the image from the ubuntu-touch/utopic-propose, set up the click test nd install the UITK from the Silo10 PPA
  • The script dumps the test results in the local directory and tries max 3 times if an app test is not OK.

Suggested process

  • First, flash the latest image.
  • Boot into the image, go through the wizard, get rid of the welcome tutorial, and disable automatic updates.
  • adb shell mount -o remount,rw /
  • Set up click tests: phablet-click-test-setup (or, for RTM: phablet-click-test-setup --distribution=ubuntu-rtm --series=14.09 )
  • apt-get install python3-lxml
  • Install the silo. (add the PPA, disable all other package sources, apt-get update, apt-get dist-upgrade, disable the PPA, re-enable other package sources)
  • adb shell reboot
  • Grab the UITK source: bzr branch lp:ubuntu-ui-toolkit
  • cd ubuntu-ui-toolkit/tests/
  • mkdir results
  • Modify uitk_test_plan.sh as needed. Might need to change the variables at the top (particularly where it stores results), perhaps more depending on your specific needs.
  • time ./uitk_test_plan.sh -r
  • Save the overall output into results/summary.txt (or similar).
  • mv results results.silo

If anything failed (and it probably did), reflash without the silo then re-test the components which failed. Requires editing uitk_test_plan.sh again, to remove components which don't need re-testing. Then compare the results against the first run. If the silo has the same or fewer failures as the base image, it passes.

Note that several AP tests like to produce a different number of failures on each test run. This can greatly complicate the above process.

Process/Merges/TestPlan/ui-toolkit (last edited 2014-09-12 12:52:07 by nemein)