CertTestingProposed

Summary

Uploads of certain packages to the -proposed archive should trigger a full test run on all certified hardware. We can currently do this manually but should add more automation.

Release Note

Rationale

When an upload lands in -proposed we want to test it for regressions across all available hardware.

Use Cases

  • A new kernel is uploaded to -proposed which triggers an install + test run on all the hardware in the Canonical test labs.
  • A new xorg is uploaded to -proposed which triggers an install and video+input tests on all laptops (and some desktops?)
  • Steve is testing an SRU for a hardware-specific issue. He searches the hardware certification website and finds a system with the relevant hardware. From the website he can boot and install the system with -proposed enabled. He logs in via KVM and performs his testing manually.

Assumptions

Design

Triggering

By uploads to -proposed

  • An upload to -proposed of one of the following packages flipps a proposed-test-run flag and set a timer:

    • linux, xorg, lbm, udev, hal, ... XXX

    • XXX How do we detect -proposed uploads, parse the mailing-list?

  • packages determine the machine-type and teststo be queued
    • linux, udev, hal: all machines

    • xorg: laptops (and some desktops?) with resolution tests

    • XXX - need complete mapping

  • After a set period (say, 48hrs) the proposed run is initiated

Manually from website

  • XXX Need to decide what pages to trigger runs from and to mock-up a GUI

Publishing results

  • Publish complete test results on the certification website
  • Publish sanetised results in a publicly-viewable location
  • In both cases this could be an XML file with a simple HTML transform

Implementation

UI Changes

A button should be added to the machine page to initiate an install run. Strawman:

Launch install test:
[Milestone drop-down] [x] Enable -proposed

Code Changes

Test/Demo Plan

Test manually on the staging server.

Unresolved issues

BoF agenda and discussion


CategorySpec

QATeam/Specs/CertTestingProposed (last edited 2008-12-10 21:55:15 by cpc1-oxfd8-0-0-cust993)