Checkbox
Size: 2625
Comment:
|
Size: 3731
Comment:
|
Deletions are marked like this. | Additions are marked like this. |
Line 1: | Line 1: |
## page was renamed from Testing/Automation/LDTP/IntegrationCheckbox ## page was renamed from Testing/Automation/LDTP/IntegrationHwtest |
|
Line 46: | Line 44: |
|| plugin || Plugin name to handle this test || || description || A description on what the test does || |
|| plugin || Plugin name to handle this test. Checkbox provides "manual" and "auto" by default || || description || A description on what the test does. In the case of manual tests, this is the actual question asked to the enduser. In the case of automated tests, this is strictly informational. || |
Line 49: | Line 47: |
==== Optional fields ==== | ==== Optional fields ==== |
Line 54: | Line 52: |
|| command || Command to run for the test. || | || command || Command to run for the test. This is not required for manual tests, but it can be used, anyway. || |
Line 60: | Line 58: |
=== Test Suites === These tests can be grouped by suites. A suite is basically a text file that contains one or more tests. Suites are available under the directory: /usr/share/checkbox/suites All the files under that directory are automatically parsed for tests. Tests suites are parsed by the suites_info plugin (check the Plugin section for more information about plugins) Some of the options that we can see in the configuration file (checkbox.ini) for this plugin are: \[checkbox/plugins/suites_info\] * directories = %(checkbox_directory)s/suites The folder or folders where test suites are stored. * scripts_path = %(checkbox_directory)s/scripts The folder where runnable scripts are stored. ==== Environment variables ==== Apart from the two options explained above, this is the place to define the environment variables that we want for our tests. From the example above this would be: data_path = %(checkbox_directory)s/data |
|
Line 61: | Line 84: |
Introduction
Checkbox (https://launchpad.net/checkbox) is a test runner for Ubuntu. It aims is to provide a common framework to run all types of tests, from hardware tests, to command line tests, unit tests or desktop tests and send their results to Launchpad, automatically.
General Configuration
Checkbox general configuration is stored at /etc/checkbox.d. Each way of calling checkbox (GUI, cli) has its own configuration file.
checkbox.ini
This is the general config file. It will be inherit by all the checkbox flavours.
- [DEFAULT]. Under this section they are covered all general configuration properties.
- version. The version of checkbox to use the configuration with.
- plugins. Folder where checkbox plugins are stored.
- registries. Folder where checkbox registries are stored.
Tests
Test are defined in plain text files, that can be stored anywhere.
They consists in some required fields and some optional fields.
Let's look at one example:
name: audio plugin: manual categories: laptop, desktop requires: alsa.type == 'control' command: audio_playback $data_path/audio_playback.wav description: Testing detected sound card: . $(audio_playback --view) . Did you hear a sound?
Tests fields
Required fields
Field |
Explanation |
name |
Unique name for a test |
plugin |
Plugin name to handle this test. Checkbox provides "manual" and "auto" by default |
description |
A description on what the test does. In the case of manual tests, this is the actual question asked to the enduser. In the case of automated tests, this is strictly informational. |
Optional fields
Field |
Explanation |
architectures |
List of architectures for which this test is relevant: amd64, i386, powerpc and/or sparc |
categories |
List of categories for which this test is relevant: desktop, laptop and/or server |
command |
Command to run for the test. This is not required for manual tests, but it can be used, anyway. |
depends |
List of names of tests on which this test depends. So, if the other test fails, this test will be skipped. |
requires |
Registry expressions which are requirements for this test: 'input.mouse' in info.capabilities |
timeout |
Timeout for running the command. |
optional |
Boolean expression set to True if this test is optional or False if this test is required. |
Test Suites
These tests can be grouped by suites. A suite is basically a text file that contains one or more tests. Suites are available under the directory: /usr/share/checkbox/suites
All the files under that directory are automatically parsed for tests.
Tests suites are parsed by the suites_info plugin (check the Plugin section for more information about plugins)
Some of the options that we can see in the configuration file (checkbox.ini) for this plugin are:
\[checkbox/plugins/suites_info\]
- directories = %(checkbox_directory)s/suites
- The folder or folders where test suites are stored.
- scripts_path = %(checkbox_directory)s/scripts
- The folder where runnable scripts are stored.
Environment variables
Apart from the two options explained above, this is the place to define the environment variables that we want for our tests. From the example above this would be:
data_path = %(checkbox_directory)s/data
Plugins
Registries
Testing/Automation/Checkbox (last edited 2021-10-20 11:49:54 by sylvain-pineau)