AddingTests
Adding a test to run under the CI Infrastructure
Guidelines
Here are some basic guidelines for creating a test that can run under CI:
- Ensure that the test is as self-contained as possible.
- For tests running on phones, for example, it's best if it can run using phablet-test-run without any extra setup provided outside the test itself
- Tests should output results in subunit (preferred) or binary fail/pass
- Ensure that the tests run reliably without random failures as this makes it difficult to determine if failures are real.
- For tests that look at performance, the raw performance data should be part of the results stream, but baselines should be compared to also record a clear pass/fail for each testcase.
Requesting CI to add the test
As soon as you are ready for the CI team to start looking at adding the tests, open a bug at https://bugs.launchpad.net/ubuntu-ci-services-itself and provide the following information:
- Where the tests can be found
- Intended targets the test should run on (arch, vm/hardware, device types, etc)
- Any special hardware required
- Any special setup that has not yet been added to the setup of the test itself
Future Changes
- The CI team is investigating the use of dep8 testing as a broad solution to test setup and execution problem. This includes extensions to cover specific needs such as specifying a machine type.
CI/AddingTests (last edited 2013-11-19 15:22:08 by fginther)