Your first Autopilot test
This page will walk through writing and contributing an autopilot testcase.
Video Screencast of Tutorial
A video version of this tutorial is coming soon.
Requirements
- Autopilot
- sudo apt-add-repository ppa:autopilot/1.5
sudo apt update && sudo apt install python3-autopilot python3-vis
- Bazaar
- sudo apt install bzr
- Launchpad/Ubuntu SSO account
Things you need to know
Should you get stuck / need help
Don't forget there is a wonderful quality community get help from if you get stuck! Here's a list of resources to help you connect with for help.
* Mailing list:
* IRC channels:
Use these resources to help you!
Launchpad
Launchpad holds the source code for all autopilot testcases in the autopilot testcase project. Code contributions are submitted as merge requests.
Bzr
Bzr is the version control system used. You can see a reference chart here as well as a quick start guide if you need help on the basic commands you'll need to contribute a testcase.
Writing an autopilot testcase
Get the current testcases
Make a new folder for you to develop under.
mkdir ~/autopilottests
For this example, let's utilize ~/autopilottests.
Open up a terminal and change directory into the folder.
cd ~/autopilottests
Now, branch the current testcases.
bzr branch lp:ubuntu-autopilot-tests
This will grab a copy of the current testcases used by the project. We can then use bzr to add a new testcase we'll write below and then finally commit the testcase back to the project.
Choose an application
Choose an application you wish to write the testcase for. An excellent place to begin is by looking at the needed tests from the list.
Assigning the work to you
Each testcase request is listed as a bug report. Once you've chosen a bug (test) or two to work on, assign yourself to that bug (test) in launchpad. You can use the bug later on as well as part of your merge request.
Familiarize yourself with Autopilot
Reference the official documentation and tutorial.
In addition, you can leverage the existing source code tree you branched for help and inspiration.
Write the test
Run the application you've chosen and pick a few of the primary features of the application. Document each feature you've chosen and write down step by step instructions in order to utilize the feature. You should template out your planned feature tests, and document each step as a comment. For example, if I want to ensure the bookmark manager in firefox works properly, I would write tests to test the ability to open, save, delete, edit and order a bookmark. These step by step instructions will be transformed into automated actions to be performed by your code.
Run through the steps you wrote down to ensure they exercise the feature you targeted. As you step through your instructions, record what happens for each step so you can add them to the test case. These will become your assertions in the testcase. For example, If I press Ctrl+o while in gedit, the expected result might be that an open window will appear.
Utilize autopilot's mouse, keyboard, and data manipulation to perform the same steps. Write each feature test as a separate test function. Run your code and ensure all your intended actions are performed.
Add in the assertions your made about the state of the application as the code executes your testcase. Every line doesn't necessarily need an assertion, provided the main actions of the testcase have assertions.
Using Bzr
Now that your changes are complete, it's time to commit them to your local repository and then sync your local repository to launchpad so others can view and utilize your work. If needed, reference the links in the Bzr section at the beginning of this tutorial. Bzr commands require you to run them inside the directory (or a sub-directory) of the repository on your local machine. If needed, cd to the directory of the files you made / changed before running these commands.
First, let's commit to your local repository. Bzr allows you to view the status of a repository using bzr status. This will show you the changes you made to your local repository. For example;
$ bzr status modified: testcases/packages/Mousepad unknown: testcases/packages/nautilus
In this example you can see I've modified the Mousepad testcase, and made a new file for a new testcase for nautilus. The new file shows up as "unknown" to bzr.
If you made a new file, use the bzr add command to add it to the repository. If you only modified an existing file, you can skip this step and jump down to "Commit your work".
Add a file
bzr add /path/to/testcase
This will add the file to the repository so changes to it are tracked. The bzr status should now reflect that you are adding the new file to the repository. For example;
$ bzr add testcases/packages/nautilus adding testcases/packages/nautilus $ bzr status added: testcases/packages/nautilus modified: testcases/packages/Mousepad
Commit your work
bzr commit
This will prompt you to enter a commit message. You must enter a message of some sort in order to commit your changes. The message should be a short description of what you changed. For example, the message for my changes above could say "Added a new test for mousepad to verify keyboard layout. Created a new nautilus testcase". Finish typing the message and exit the editor. For example, if you are using nano, you can save the file and exit by typing ctrl+o, choose a save name and enter; then ctrl+x to exit.
Contribute your new test!
Replace the YOURUSERID field below with your launchpad id. Replace the YOURBRANCHNAME with a unique name for your branch -- it can be anything you wish. Naming the branch after the testcase you created/changed is an excellent choice.
bzr push lp:~YOURUSERID/ubuntu-autopilot-tests/YOURBRANCHNAME
You can confirm your branch is uploaded and view the code on launchpad here.
Now you simply need to submit a merge proposal to ensure your new branch gets reviewed and imported to the main trunk branch.
Contribute your new test!
Linking the branch to a bug
Is this merge to close a specific bug? If so copy the bug number. Click your branch shown on this page and then click "Link a bug report". Paste the bug number, click search, then click OK.
Submitting a merge proposal
Select the "Propose for merging" link to start the merge request. Submit a merge request that includes your new testcase to the project. It will be reviewed and merged accordingly. Thanks for contributing!
What next?
Once accepted and merged, your test will be available to run on the jenkins QA instance. This will allow the test to run against the new builds of the package as they become available. It will ensure any new versions of the packages pass the tests and remain regression free. You can view the test output here.