Your first Autopilot test

This page will walk through writing and contributing an autopilot testcase to the ubuntu core applications project.

Video Version

There is a video version of this tutorial that covers much more depth than what is listed here. Highly Recommended you view. NOTE: The video is quite old and uses things like the emulators, which have been replaced by the UITK helpers. The methodology is still the same

Requirements

• Autopilot
  • sudo apt-add-repository ppa:autopilot/1.5

  • sudo apt-get update && sudo apt-get install python3-autopilot python3-autopilot-vis

• bzr
  • sudo apt-get install bzr
• Launchpad/Ubuntu SSO account

  • Sign up if needed

  • Proper setup for code contribution

• Ubuntu SDK

  • Check out http://developer.ubuntu.com/start/ubuntu-sdk/installing-the-sdk/ to install

• SDK autopilot helpers
  • sudo apt-get install ubuntu-ui-toolkit-autopilot
• cmake and build tools
  • sudo apt-get install cmake build-essential
• You might encounter applications that need additional dependencies specific to an application. After checking out the application source, check the debian/control file for additional dependencies needed for building and running applications.

Things you need to know

Should you get stuck / need help

Don't forget there is a wonderful quality community get help from if you get stuck! Here's a list of resources to help you connect with for help.

* Mailing lists:

• ubuntu-touch

• ubuntu-quality

* IRC channels:

• #ubuntu-touch

• #ubuntu-autopilot

• #ubuntu-quality

Use these resources to help you!

developer.ubuntu.com

This site, while intended at application developers, contains useful information for you as a test author. Here's a few handy links you might want to review:

http://developer.ubuntu.com/en/start/quality/ https://developer.ubuntu.com/api/autopilot/python/1.5.0/tutorial-getting_started/

Launchpad

Launchpad holds the source code for all ubuntu core apps in separate project branches. See the list below:

Core Apps source branches

bzr branch lp:ubuntu-calendar-app
bzr branch lp:ubuntu-clock-app
bzr branch lp:ubuntu-weather-app
bzr branch lp:ubuntu-calculator-app
bzr branch lp:ubuntu-rssreader-app
bzr branch lp:ubuntu-filemanager-app
bzr branch lp:ubuntu-docviewer-app
bzr branch lp:ubuntu-terminal-app
bzr branch lp:music-app
bzr branch lp:dropping-letters
bzr branch lp:reminders-app
bzr branch lp:sudoku-app
bzr branch lp:stock-ticker-mobile-app

Code contributions are submitted as merge requests.

Bzr

Bzr is the version control system used. You can see a reference chart here as well as a quick start guide if you need help on the basic commands you'll need to contribute a testcase.

Writing an autopilot testcase

Get the current testcases

Make a new folder for you to develop under.

mkdir ~/autopilottests

For this example, let's utilize ~/autopilottests.

Open up a terminal and change directory into the folder.

cd ~/autopilottests

Now, branch the current testcases for the ubuntu core app you wish to develop testcases for. See the branch listings above.

bzr branch <BRANCH>

This will grab a copy of the current testcases used by the project. We can then use bzr to add a new testcase we'll write below and then finally commit the testcase back to the project.

Choose an application

Choose a feature you wish to write the testcase for. An excellent place to begin is by looking at the needed tests. Each application has a listing of needed testcases. Follow the link that corresponds to the application you've chosen below to choose a feature.

Tests needed for all the ubuntu core apps

By application:

ubuntu-calendar-app
ubuntu-clock-app
ubuntu-weather-app
ubuntu-calculator-app
ubuntu-emailclient-app
ubuntu-rssreader-app
ubuntu-filemanager-app
ubuntu-docviewer-app
ubuntu-terminal-app
music-app
dropping-letters
sudoku-app
stock-ticker-mobile-app

Assigning the work to you

Each testcase request is listed as a bug report. Once you've chosen a bug (test) or two to work on, assign yourself to that bug (test) in launchpad. You can use the bug later on as well as part of your merge request.

Familiarize yourself with Autopilot

See the resources on the developer portal.

In addition, you can leverage the existing source code tree you branched for help and inspiration.

Build the application

If the application is a compiled binary, you will need to build it. The core apps projects use cmake and most can be built by running cmake, then make. See the documentation included in the project source or reference developer.ubuntu.com.

http://developer.ubuntu.com/apps/qml/tutorial/building_your_first_qml_app/

If the build fails, check and ensure all the listed dependencies in the debian/control file are installed on your system. If you get stuck, contact the developers or seek others for help in building.

Run the application

Non-compiled applications can be manually executed via qmlscene.

qmlscene path/to/qmlfile.qml

If the application is a compiled binary, it should be run directly.

Write the test

Run the application you've chosen and pick a few of the primary features of the application. Document each feature you've chosen and write down step by step instructions in order to utilize the feature. You should template out your planned feature tests, and document each step as a comment. For example, if I want to ensure the bookmark manager in firefox works properly, I would write tests to test the ability to open, save, delete, edit and order a bookmark. These step by step instructions will be transformed into automated actions to be performed by your code.

Run through the steps you wrote down to ensure they exercise the feature you targeted. As you step through your instructions, record what happens for each step so you can add them to the test case. These will become your assertions in the testcase. For example, If I press Ctrl+o while in gedit, the expected result might be that an open window will appear.

Utilize autopilot's mouse, keyboard, and data manipulation to perform the same steps. Write each feature test as a separate test function. Run your code and ensure all your intended actions are performed.

Add in the assertions your made about the state of the application as the code executes your testcase. Every line doesn't necessarily need an assertion, provided the main actions of the testcase have assertions.

Using Bzr

Now that your changes are complete, it's time to commit them to your local repository and then sync your local repository to launchpad so others can view and utilize your work. If needed, reference the links in the Bzr section at the beginning of this tutorial. Bzr commands require you to run them inside the directory (or a sub-directory) of the repository on your local machine. If needed, cd to the directory of the files you made / changed before running these commands.

First, let's commit to your local repository. Bzr allows you to view the status of a repository using bzr status. This will show you the changes you made to your local repository. For example;

$ bzr status
modified:
  testcases/packages/Mousepad
unknown:
  testcases/packages/nautilus

In this example you can see I've modified the Mousepad testcase, and made a new file for a new testcase for nautilus. The new file shows up as "unknown" to bzr.

If you made a new file, use the bzr add command to add it to the repository. If you only modified an existing file, you can skip this step and jump down to "Commit your work".

Add a file

bzr add /path/to/testcase

This will add the file to the repository so changes to it are tracked. The bzr status should now reflect that you are adding the new file to the repository. For example;

$ bzr add testcases/packages/nautilus 
adding testcases/packages/nautilus
$ bzr status
added:
  testcases/packages/nautilus
modified:
  testcases/packages/Mousepad

Commit your work

bzr commit

This will prompt you to enter a commit message. You must enter a message of some sort in order to commit your changes. The message should be a short description of what you changed. For example, the message for my changes above could say "Added a new test for mousepad to verify keyboard layout. Created a new nautilus testcase". Finish typing the message and exit the editor. For example, if you are using nano, you can save the file and exit by typing ctrl+o, choose a save name and enter; then ctrl+x to exit.

Contribute your new test!

Replace the YOURUSERID field below with your launchpad id. Replace the YOURBRANCHNAME with a unique name for your branch -- it can be anything you wish. Naming the branch after the testcase you created/changed is an excellent choice.

bzr push lp:~YOURUSERID/COREAPPBRANCH/YOURBRANCHNAME

You can confirm your branch is uploaded and view the code on launchpad here.

Now you simply need to submit a merge proposal to ensure your new branch gets reviewed and imported to the main trunk branch.

Contribute your new test!

Linking the branch to a bug

Is this merge to close a specific bug? If so copy the bug number. Click your branch shown on this page and then click "Link a bug report". Paste the bug number, click search, then click OK.

Submitting a merge proposal

Click your branch shown on this page and then select the "Propose for merging" link to start the merge request. Submit a merge request that includes your new testcase to the applicable core apps project. It will be reviewed and merged accordingly. Thanks for contributing!

What next?

Once accepted and merged, your test will be available to run on the jenkins QA instance. This will allow the test to run against the new builds of the package as they become available. It will ensure any new versions of the packages pass the tests and remain regression free.