Your first Autopilot test

This page will walk through writing and contributing an autopilot testcase.

Video Screencast of Tutorial

A video version of this tutorial is coming soon.

Requirements

• Autopilot
  • sudo apt-add-repository ppa:autopilot/1.5

  • sudo apt update && sudo apt install python3-autopilot python3-vis

• Bazaar
  • sudo apt install bzr
• Launchpad/Ubuntu SSO account

  • Sign up if needed

  • Proper setup for code contribution

Things you need to know

Should you get stuck / need help

Don't forget there is a wonderful quality community get help from if you get stuck! Here's a list of resources to help you connect with for help.

* Mailing list:

• ubuntu-quality

* IRC channels:

• #ubuntu-autopilot

• #ubuntu-quality

Use these resources to help you!

Launchpad

Launchpad holds the source code for all autopilot testcases in the autopilot testcase project. Code contributions are submitted as merge requests.

Bzr

Bzr is the version control system used. You can see a reference chart here as well as a quick start guide if you need help on the basic commands you'll need to contribute a testcase.

Writing an autopilot testcase

Get the current testcases

Make a new folder for you to develop under.

mkdir ~/autopilottests

For this example, let's utilize ~/autopilottests.

Open up a terminal and change directory into the folder.

cd ~/autopilottests

Now, branch the current testcases.

bzr branch lp:ubuntu-autopilot-tests

This will grab a copy of the current testcases used by the project. We can then use bzr to add a new testcase we'll write below and then finally commit the testcase back to the project.

Choose an application

Choose an application you wish to write the testcase for. An excellent place to begin is by looking at the needed tests from the list.

Assigning the work to you

Each testcase request is listed as a bug report. Once you've chosen a bug (test) or two to work on, assign yourself to that bug (test) in launchpad. You can use the bug later on as well as part of your merge request.

Familiarize yourself with Autopilot

Reference the official documentation and tutorial.

In addition, you can leverage the existing source code tree you branched for help and inspiration.

Write the test

Run the application you've chosen and pick a few of the primary features of the application. Document each feature you've chosen and write down step by step instructions in order to utilize the feature. You should template out your planned feature tests, and document each step as a comment. For example, if I want to ensure the bookmark manager in firefox works properly, I would write tests to test the ability to open, save, delete, edit and order a bookmark. These step by step instructions will be transformed into automated actions to be performed by your code.

Run through the steps you wrote down to ensure they exercise the feature you targeted. As you step through your instructions, record what happens for each step so you can add them to the test case. These will become your assertions in the testcase. For example, If I press Ctrl+o while in gedit, the expected result might be that an open window will appear.

Utilize autopilot's mouse, keyboard, and data manipulation to perform the same steps. Write each feature test as a separate test function. Run your code and ensure all your intended actions are performed.

Add in the assertions your made about the state of the application as the code executes your testcase. Every line doesn't necessarily need an assertion, provided the main actions of the testcase have assertions.

Using Bzr

Now that your changes are complete, it's time to commit them to your local repository and then sync your local repository to launchpad so others can view and utilize your work. If needed, reference the links in the Bzr section at the beginning of this tutorial. Bzr commands require you to run them inside the directory (or a sub-directory) of the repository on your local machine. If needed, cd to the directory of the files you made / changed before running these commands.

First, let's commit to your local repository. Bzr allows you to view the status of a repository using bzr status. This will show you the changes you made to your local repository. For example;

$ bzr status
modified:
  testcases/packages/Mousepad
unknown:
  testcases/packages/nautilus

In this example you can see I've modified the Mousepad testcase, and made a new file for a new testcase for nautilus. The new file shows up as "unknown" to bzr.

If you made a new file, use the bzr add command to add it to the repository. If you only modified an existing file, you can skip this step and jump down to "Commit your work".

Add a file

bzr add /path/to/testcase

This will add the file to the repository so changes to it are tracked. The bzr status should now reflect that you are adding the new file to the repository. For example;

$ bzr add testcases/packages/nautilus 
adding testcases/packages/nautilus
$ bzr status
added:
  testcases/packages/nautilus
modified:
  testcases/packages/Mousepad

Commit your work

bzr commit

This will prompt you to enter a commit message. You must enter a message of some sort in order to commit your changes. The message should be a short description of what you changed. For example, the message for my changes above could say "Added a new test for mousepad to verify keyboard layout. Created a new nautilus testcase". Finish typing the message and exit the editor. For example, if you are using nano, you can save the file and exit by typing ctrl+o, choose a save name and enter; then ctrl+x to exit.

Contribute your new test!

Replace the YOURUSERID field below with your launchpad id. Replace the YOURBRANCHNAME with a unique name for your branch -- it can be anything you wish. Naming the branch after the testcase you created/changed is an excellent choice.

bzr push lp:~YOURUSERID/ubuntu-autopilot-tests/YOURBRANCHNAME

You can confirm your branch is uploaded and view the code on launchpad here.

Now you simply need to submit a merge proposal to ensure your new branch gets reviewed and imported to the main trunk branch.

Contribute your new test!

Linking the branch to a bug

Is this merge to close a specific bug? If so copy the bug number. Click your branch shown on this page and then click "Link a bug report". Paste the bug number, click search, then click OK.

Submitting a merge proposal

Select the "Propose for merging" link to start the merge request. Submit a merge request that includes your new testcase to the project. It will be reviewed and merged accordingly. Thanks for contributing!

What next?

Once accepted and merged, your test will be available to run on the jenkins QA instance. This will allow the test to run against the new builds of the package as they become available. It will ensure any new versions of the packages pass the tests and remain regression free. You can view the test output here.