UDS Intrepid QA Report

Plans for 8.10

Testing

• Upgrade testing - Take over the running of mvo's upgrade scripts. Perform regular upgrade tests leading up to release in VMs with a significant number of packages installed

• Stock Ubuntu KVM images - Provide stock KVM images of different flavours and releases of Ubuntu used for performing SRU, upgrade and install testing (e.g. testing migration and resizing).

• KVM images for upstream software - Provide downloadable KVM images of an upstream versions of key software for testing related to bug forwarding. Initial image based on Debian testing, covering Gnome, OpenOffice and Firefox (replacing Iceweasel).

• Upstream kernel builds - Work with the kernel team to provide PPA builds of upstream kernels for user testing.

• Desktop test automation - Implement and deploy a basic set of desktop tests to be run regularly in the data centre on the development archive

• Mobile test automation - Write test scripts for basic functionality of the UME desktop to be run in VMs and on mobile devices

• ISO testing automation - Use pre-seeding to to test desktop images as well as server images (currently performed by the server team)

• Improved test case management Improve the structure and level of detail in the test cases in the wiki. Fork the current cases into an Intrepid and Hardy LTS set.

Triage

• Kernel bug migration - merge and close bugs from 2.6.x packages with 'linux'.

• Confirmed -> Triaged bug migration - Move a significant proportion of bugs from Confirmed to Triaged.

• Bug upstreaming documentation - Improve the documentation explaining how to file bugs upstream for individual upstream projects.

• Bug escalation procedures - Provide mechanism for trusted community members to escalate serious bugs

Tracking & Monitoring

• Bug metrics - Continue to improve the set of available bug metrics, focusing increasingly on information that can be used to guide development as well as triage

• Package-specific status pages - Implement weather-report style status pages for key packages located at to status.qa.ubuntu.com

• SRU bug tracking - Improve tracking of the SRU bug lists with a set of script-generated web pages that extracts status, test case text and tag information from SRU process bugs. Provide views of main and universe as well as current and archive views

• Release bug tracking - Clarification and documentation of bug milestones and nominations

Specs

QA Intrepid specs are listed ["QATeam/Specs"].

Sessions

Kernel Bug Migration

See [:QATeam/KernelBugMigration].

Bug response times

Spec: foo

Agenda

• How can we measure it?
• How can we improve it?
• Display on per-package status page?
• Larger picture: global bug volume

Notes

• How can we measure it?
  • some date transitions are recorded in Launcpad
    • available on a per task basis
      • date-left-new - particularly useful
        • this is the first time the bug's state changed
      • date-confirmed
      • date-triaged
      • date-incomplete
      • date-triaged - particularly useful
      • date-fixcommited
      • date-fix-released - particularly useful
      • date-closed
      • date-inprogress
      • date-assigned
      • date-created
        • the date-created for a bug watch would show when the upstream task was created
        • the activity log, bug-watch-updater first comment, shows when the watch was created
    • available for the bug in general
      • date reported
      • date updated
    • missing?
      • date importance set - in activity log but potentially hard to parse
    • look at the average or median response times over a 1,3,6 month period
    • a new tool in the bughelper suite could be used to show time in state statistic infromation
• How can we improve it?
  • making information publicly available
• Display on per-package status page
  • also combine some packages for example linux-source, linux-restricted-modules, linux-ubuntu-modules, linux-backports-modules
• Display on a per repository basis

• Display on a per team basis using something like https://bugs.launchpad.net/~ubuntu-printing/+packagebugs

  • these are packages that a team is subscribed to

Bug escalation procedures

Spec: foo

Agenda

• Case 1: World breaking regressions in stable or late devel version
  • Using the critical importance with an escalation bot
• Case 2: Important bugs that need be made more prominent before release
  • Formal and informal escalation paths within the QA team

Notes

• Notes

Upstream Bug Workflow

Agenda

Notes

Extending the test tracker

Agenda

• SRU Fix validation facility
• More granular test case recording

Notes

Extending qa.ubuntu.com

Agenda

• Landing page
• Per-package weather reports

Notes