BugHandling

Revision 1 as of 2009-11-27 12:21:49

Clear message

Review of Karmic

Developer approach

In Karmic we changed the approach for developers to concentrate more on fixing bugs than on triaging new ones.

This was overall a success: The entire Ubuntu team in total (including community) increased bug fixing by 20% (see bugs fixed in jaunty and karmic. Canonical desktop team members got an increase as well.

So in total we had a good improvement, and given that Jaunty was a QA cycle and in Karmic we did a lot of upstream-ish work and did not really "mean" to fix a lot of bugs, the outcome was very satisfying.

Testing/Reporting

Many bugs only get reported after release (e. g. hardware/driver specific problems). This is the usual chicken-egg problem of too few testers before release.

Bryce reports that X driver testing is hard: most developers use Intel hardware because it is better supported. Then non-developers finally test NVIDIA upon release.

In Karmic we disabled the +filebug Launchpad page for non-ubuntu-bugcontrol-members, which led to richer bug reports, but not necessarily better descriptions nor fewer bug reports.

Things that take time for Desktop developers:

  • Duplicates that cannot be automatically marked duplicate, because they have wildly different titles. These indicate the reporters are not finding the existing reports.
  • Bug reports from prolific posters of low quality bugs
  • Dead bugs that have been fixed but haven't been closed (clutters up bug list).
  • Bug storms from automatic duplication (i. e. everyone getting noticed everytime there is a duplicate then people doing a reply-all to try and unsubscribe/complain).

Bug report quality

There is a large difference between true defect reports and support requests which are often "something happened, how do I actually write a useful defect report?". Most incoming reports are unfortunately in the latter class.

It is currently hard to separate high-quality from low-quality bug reports, as well as seeing bugs which affect a lot of people.

Expectations from different groups

Value of bug reports:

  • Users: support tool
  • Developers: improve Ubuntu, pool of potential issues to work on
  • Marketing opportunity to help sell Ubuntu
    • phone-a-friend to have your bug reproduced/confirmed or getting help for (connecting to local community)
    • look at these other bugs about this package to try and confirm them to help out
  • Expectations from both sides are not encoded anywhere

Goals for improvements

  1. Solve scalability issue
  2. Reduce incoming bugs
  3. Higher quality bug reports

Changes

Workflow

  • Disable menu item for reporting a bug in the stable release; only leave it for the development release
  • Automatically time out incomplete bugs without response for Ubuntu which do not have an assignee (needs to be enabled in Launchpad). This would work better with a new "Expired" state (which can't be set from the web UI), but "Invalid" will do for now.
  • After a new Ubuntu release, set new bugs to incomplete and ask whether it's still an issue in the newer version; this goes hand in hand with automatic time out. This should be done on a per-package basis with a script invoked by the package maintainer.

Apport

  • Auto-subscribe QA team by the apport retracer once it gets the tenth duplicate, for creating bug patterns.
  • Change apport symptoms and appropriate package hooks to produce default bug titles, so that Launchpad's "proposed duplicates" becomes useful for bugs.
  • Marc Tardiff to provide list of types of problems that we should make symptoms for.

Launchpad

  • Change Launchpad bug interface to setting expectations and next steps (after a user has submitted a bug) (deryck)
  • Work with Launchpad engineers to improve/fine tune parameters for "bug heat".

Community

  • Dynamic is against the core team keeping up with incoming bugs, so stop trying.
  • Generalize hug day concept and involve the community more: "15 minutes a day ... please triage these three bugs"
  • Adopt a package for triaging, give triagers a sense of ownership
  • The qa team discussed this topic on the review of the bugsquad mentoring program.