ARMAutomatedTestingFramework

Revision 3 as of 2010-05-03 15:42:33

Clear message

UNDER CONSTRUCTION

Summary

We need to decide on, or create an automated testing framework to improve automated testing in Ubuntu on Arm.

Release Note

No end-user visible change.

Rationale

ARM devices capable of running Ubuntu are available today, but are not yet as pervasive as x86. Therefore, our pool of potential testers is smaller, necessitating a more efficient approach to testing. Automated testing opportunities exist in all of Ubuntu, however it has traditionally been approached without consideration of non-x86 devices.

User stories

Assumptions

Design

You can have subsections that better describe specific parts of the issue.

Standalone Test Client

A standalone client should allow a developer or tester to use a command-line interface to perform basic testing operations such as:

  • Listing available tests
  • Installing tests
  • Executing tests
  • Storing and Listing results
  • Gathering system information and logs
  • Publishing results to a server

This may or may not have a graphical interface, but a text interface is preferable for the client since it could also be used in lightweight environments that lack a GUI.

Test Server

A test server should allow results to be gathered and stored in a fashion that they can be associated with specific builds, hardware, or timestamps. The function of the server is not only for storage of the results, but may also be extended to allow for scheduling test runs on remote clients.

Implementation

This section should describe a plan of action (the "how") to implement the changes discussed. Could include subsections like:

Code Changes

Code changes should include an overview of what needs to change, and in some cases even the specific details.

Migration

Include:

  • data migration, if any
  • redirects from old URLs to new ones, if any
  • how users will be pointed to the new way of doing things, if necessary.

Test/Demo Plan

It's important that we are able to test new features, and demonstrate them to users. Use this section to describe a short plan that anybody can follow that demonstrates the feature is working. This can then be used during testing, and to show off after release. Please add an entry to http://testcases.qa.ubuntu.com/Coverage/NewFeatures for tracking test coverage.

This need not be added or completed until the specification is nearing beta.

Unresolved issues

This should highlight any issues that should be addressed in further specifications, and not problems with the specification itself; since any specification with problems cannot be approved.

BoF agenda and discussion

Use this section to take notes during the BoF; if you keep it in the approved spec, use it for summarising what was discussed and note any options that were rejected.

CategorySpec