Updating Tests in Checkbox

Probably the simplest way to contribute to Ubuntu Friendly is to update a test which is incorrect in some way. There are a number of ways in which a test might be wrong, varying in the amount of technical knowledge and knowledge of Checkbox required to do them. We'll start with the easiest and work our way up.

Easy: Updating a manual test description

Sometimes the test descriptions displayed to the user of Checkbox aren't 100% clear, perhaps they use some ambiguous language or refer to behaviour of the system that has changed since the test was written. For example, the SD card test may refer to the 'Places' menu which is not part of the Unity interface.

To update a test description, you should search for the test job name in the 'jobs' directory of Checkbox (using grep for example). This is stated in the title bar of Checkbox when running it and will be something like 'graphics/resolution'. As a rule of thumb, the part of the name before the forward-slash will match the job file name, so in this case you will find it in 'graphics.txt.in'.

plugin: manual
name: graphics/resolution
 device.category == 'VIDEO'
command: resolution_test
     This test will verify the default display resolution
     1. This display is using the following resolution:
     Is this acceptable for your display?

Each test may have a 'description' field (often prefixed with an underscore to allow the description to be picked up by Ubuntu's translation tools). This is what you should edit to change what is displayed to the tester in Checkbox. Be careful to preserve the PURPOSE, STEPS, INFO and VERIFICATION sections and if you add or remove steps from the STEPS section take care to preserve the numbering of them.

Medium (but possibly easy or very hard): Editing a tests behaviour

Many tests in Checkbox are automated or semi-automated, meaning that a script is used to drive the test but the test still requires some manual intervention (e.g. plugging in hardware or verifying results). In either case, the command run is described in the 'command' field of the job. The command in question is simply executed as it would be in a shell. Some of the programs used in the commands will be scripts specially written for that test and stored in the 'scripts' directory of Checkbox as explained previously.

You may need to end up editing the command run or one of the scripts used by the command, or both. It varies tremendously depending on the problem at hand so all this section can do is explain where things need to be changed.

Hard: Fixing requirements and dependencies

This is not actually that difficult once you understand Checkbox, but it requires some knowledge specific to Checkbox which we'll try to explain here.

<<< Previous: Contributing

UbuntuFriendly/UpdatingTests (last edited 2011-11-24 14:15:42 by brendan-donegan)