Ubuntu Wiki

• Launchpad Entry: qa-hardware-test-result-publication

• Created:

• Contributors:

• Packages affected:

Summary

We use the Canonical certification testing infrastructure to test hardware compatibility leading up a release, on stable release updates and to smoke test the installation images. For the machines that are already certified and available for purchase by the public we should publish the test results in a publicly viewable location.

Rationale

Test reports play an important part in the release process and SRU validation.

Use Cases

• The Ubuntu Release Team depends on testing data to determine release readiness
• The SRU team requires information about regression testing on a wide range of hardware before releasing updates

Design

• Display results for milestone and update testing only, not commercial certification testing.

• Only relevant for hardware hosted in Canonical test labs - compatibility results are excluded
• The pages should be clearly constructed so as to be clear that these are not certification results and are for the development of Ubuntu.

Current milestone report

• Table view sorted by system:

  • System name	Date/Time	Pass	Fail	Skip	Bugs
    CompuBrand X5000	2009-01-19	11	0	2
    CompuBrand X6000	2009-01-19	10 (-1)	1	2 (+1)	#123456

• The system name links to http://webapps.ubuntu.com/certification/hardware/<system-ID>

• The date and time links to a detailed view of all test results on the certification website (restricted access)
• The +/- number shows a change from the previous milestone

Archive view

• Same view as above but showing results from previous milestones through the release cycle

Navigation

The following navigation panel should appear at the bottom of the page:

Implementation

• The certification website exports test data from non-private runs as XML or flat text

• Python code on the QA server parses this data to produce the reports served from http://qa.ubuntu.com/reports/

CategorySpec