Testing a ubuntu touch application

Unit Testing

Unit testing is a requirement for proper test-driven development. To learn how to utilize the tools Qt and the SDK make available for this, see this page.

Functional testing

For functional testing you can utilize the autopilot tool. Originally developed for the testing unity, autopilot has become a useful tool for functionally testing many different types of applications; including QML! See this page for more information on how to get started using autopilot.

Community Exploratory testing

Interested in helping prevent regressions and ensure new features land bugfree everyday? Join us in doing manual exploratory testing of each new stable image.

Testing your Ubuntu Touch Code before submission

To test your code before submitting use phablet-tools.

(instructions as of April 2014)

Preparation

These commands are to be run on the host computer (not the phone itself):

install the latest image:

ubuntu-device-flash --channel=trusty-proposed

NOTE: please check in which channel your last green image actually is. also check the exact version, use "ubuntu-device-flash -h" for the necessary flashing options.

Disable the intro:

phablet-config edges-intro --disable

Make image writable and reboot; only do this for testing debs, if only testing click packages this can be skipped:

phablet-config writable-image --ppa ppa:ci-train-ppa-service/landing-###

For click apps, this will download the tests for all your preinstalled apps you can use --click for a specific package name:

phablet-click-test-setup

Enable autopilot to reach the confined click apps, running this can take a few to several minutes depending on the amount of click apps installed:

phablet-config autopilot --dbus-probe enable

Run this in another terminal window and leave it running to make sure display stays on for the tests:

adb shell powerd-cli display on bright

Running Deb tests

Make sure you unlock the screen on your device prior to running these commands on the host:

phablet-test-run -p address-book-app-autopilot address_book_app
phablet-test-run -p camera-app-autopilot camera_app
phablet-test-run -p cordova-ubuntu-autopilot cordova_ubuntu
phablet-test-run -p dialer-app-autopilot dialer_app
phablet-test-run -p gallery-app-autopilot gallery_app
phablet-test-run -p friends-app-autopilot friends_app
phablet-test-run -p messaging-app-autopilot messaging_app
phablet-test-run -p ubuntu-keyboard-autopilot ubuntu_keyboard
phablet-test-run -p ubuntu-system-settings-autopilot ubuntu_system_settings
phablet-test-run -p ubuntu-ui-toolkit-autopilot ubuntuuitoolkit
phablet-test-run -p ubuntu-html5-ui-toolkit-autopilot ubuntu_html5_ui_toolkit
phablet-test-run -p unity-webapps-qml-autopilot unity_webapps_qml
phablet-test-run -p webbrowser-app-autopilot webbrowser_app

Running Unity8 tests on Mir

# 1. reboot the device
# 2. wait for the device mount to settle (nautilus pops up, at that point adb connection breaks)
phablet-test-run -n -p unity8-autopilot unity8

Running Click tests

# allows autopilot to snoop on dbus
phablet-config autopilot --dbus-probe enable

Note: This is not a comprehensive list of preinstalled click packages. For that list see http://people.canonical.com/~ubuntu-archive/click_packages/click_list

# pulls tests                                                 # runs tests
phablet-click-test-setup --click com.ubuntu.calculator;       phablet-test-run ubuntu_calculator_app
phablet-click-test-setup --click com.ubuntu.calendar;         phablet-test-run calendar_app
phablet-click-test-setup --click com.ubuntu.clock;            phablet-test-run ubuntu_clock_app
phablet-click-test-setup --click com.ubuntu.dropping-letters; phablet-test-run dropping_letters_app
phablet-click-test-setup --click com.ubuntu.filemanager;      phablet-test-run ubuntu_filemanager_app
phablet-click-test-setup --click com.ubuntu.music;            phablet-test-run music_app
phablet-click-test-setup --click com.ubuntu.notes;            phablet-test-run notes_app
phablet-click-test-setup --click com.ubuntu.terminal;         phablet-test-run ubuntu_terminal_app
phablet-click-test-setup --click com.ubuntu.weather;          phablet-test-run ubuntu_weather_app
phablet-click-test-setup --click com.ubuntu.shorts;           phablet-test-run ubuntu_rssreader_app

# (on the device)
# clean up the downloaded tests (see notes below)
rm -rf /home/phablet/autopilot

IMPORTANT NOTE: remember that after doing a phablet-click-test-setup, you are fetching some autopilot tests to the ~/autopilot directory on the device. The tests in this directory have priority over all other directories. This means that if a given click package test pulls in, for instance, unity8 autopilot tests (which most of them do), you need to remove the ~/autopilot directory to use the latest autopilot tests installed on the system.

Running Security tests

# (on the local machine)
bzr branch lp:ubuntu-test-cases/touch
cd touch/tests/security
./setup.sh
adb push run_test.sh /tmp/
adb push qrt_tests /tmp/qrt_tests

# (on the device)
cd /tmp/qrt_tests
# run the tests
../run_test.sh apparmor-easyprof-ubuntu
../run_test.sh check-packages
../run_test.sh click-apparmor
../run_test.sh check-ufw-kernel-requirements
../run_test.sh check-apparmor
../run_test.sh check-ufw

Start Applications through adb

In almost all cases it is a good idea to also validate your work by using your phone by hand after adding your change. A convenient way for developers to do app testing is by starting them through adb. To make that convenient we are providing a cli tool, called upstart-app-launch.

Syntax:

upstart-app-launch <DESKTOP-FILENAME>

Examples:

su - phablet

upstart-app-launch com.ubuntu.music
upstart-app-launch dialer-app

Receipts for different Developer Roles

For application developers, we highly recommend to run at least your own application autopilot before submitting code.

# install your autopilots (example if in a package); on target:
apt-get install your-app-autopilot

# unlock screen by hand and run on target through adb:
phablet-test-run your_app

For indicator developers, we highly recommend to run at least your indicator tests, but also the unity8 test suite; also if your indicator collaborates with any application, also run their tests

# install unity8 autopilot and two apps; on target:
apt-get install unity8-autopilot webbrowser-app-autopilot camera-app-autopilot

# unlock screen by hand and run on target through adb:
phablet-test-run webbrowser_app

# unlock screen by hand and run on target through adb:
phablet-test-run camera_app

# run unity8 ( remember the -n)
phablet-test-run -n unity8

For core developers, we highly recommend to run unity8, three random apps and whatever your own test suites are.