Please check the status of this specification in Launchpad before editing it. If it is Approved, contact the Assignee or another knowledgeable person before making changes.
Launchpad Entry: automatedtesting
Packages affected: None
This should provide an overview of the issue/functionality/change proposed here. Focus here on what will actually be DONE, summarising that so that other people don't have to read the whole spec.
FAUmachine is a virtual machine for i386 and amd64, with a framework which allows elaborate automatic tests (e.g. installer tests) to be conducted. It can be used to conduct many of the test-cases listed on Testing with a minimal administrative overhead.
Having widespread testing ready will reduce the chances of regressions.
Checking for regressions during the development cycle by hand is a burdensome task. Hence performing these tasks automatically seems a plausible goal.
Release manager Andrea wants to know which regressions have been introduced during the development cycle of Ubuntu. Thus, she looks up the web-page containing the results of the most recent automated test run, which shows that 9 of 10 tests have succeeded. She looks at the failed and sees a screenshot of what went wrong.
The test cases listed at Testing need to be analyzed if these can be automated.
- Experiment scripts for the applicable test-cases need to be written in the first place.
- Some framework needs to be written to present the results of different test runs.
- Ideally, these tests are run regularly, picking up the latest alpha/beta/daily images from the development version.
- Over time, some test cases need to be modified (e.g. adapted to UI changes of the installer), but this should be a fairly easy task (merely replacing a screenshot or a string in the test-script).
BoF agenda and discussion
Use this section to take notes during the BoF; if you keep it in the approved spec, use it for summarising what was discussed and note any options that were rejected.