Ubuntu bug workflow

Goals

Clearly the fundamental goal of a Quality Assurance effort is to raise the overall quality of the distribution. There are at least two aspects to quality: eliminating serious failures (hereunder security flaws and data loss) to the extent possible and providing general tuning and polish. In terms of the bugs we work with this translates to the following goals:

• Bring important bugs to the surface more reliably

• Churn through the open bugs more efficiently

Bug stages

In Launchpad we use three classifications to place a bug within the workflow: status, importance and milestone. A bug should also be linked to a package and will often be assigned to someone. Tags and team assignments can also be used to organise the bugs.

The aim of bug triage is to move a bug report from its initial New state through to one where the developer can deal with it efficiently and to weed out the bugs that are unlikely to be useful. It is often natural for a developer to perform the last few steps of the triage process which may involve obtaining some specific information from the reporter. Even so triage is invaluable in finding duplicates and sorting the critical and high priority bugs from the trivial ones.

In order to be considered fully triaged, a bug should be linked to a package, have an importance, a state (other than New or Incomplete) and an assignee. The bug should also have an accurate and descriptive title / summary that will allow other reporters to identify it as being the same as their bug. However, these do not always have to be applied in the same order but each should occur as soon as possible.

Personal workflow vs. global bug flow

The bug workflow cannot be seen independently from the workflow of the individual triager or developer. Each person has a set of skills and working style. One can debate what is the optimal personal workflow but there is no one right answer for everyone.

Ubuntu bugflow

Gathering information

Anyone with a Launchpad account can make comments on bugs and set the state. Volunteer triagers such as the bugsquad can ask for further information using the debugging guidelines in the wiki and set the status to Incomplete when appropriate. A triager can also try to reproduce the bug, provide detailed steps to reproduce the bug, provide a workaround for the bug, clarify the bug title, add descriptive tags to the bug, set the right package and search for upstream bugs or file a new bug upstream.

Rejecting

If a bug has no realistic chance of being fixed in Ubuntu the right action is often to reject it.

• TBD: When, why, how
• TBD: invalid vs. won't fix

Setting importance

Because there are only a limited number of people (devs and ubuntu-bugcontrol) who can set a bug's importance this is a useful indicator for whether a bug has been looked at by an experienced person. The importance setting of a bug is used by different groups (developers, teams, triagers) to manage the process of bug fixing and triaging and as such can often be changed several times during the life cycle of a bug.

It has been common practice to not set the importance of a bug until the required data has been collected. This may be a mistake because it does nothing to separate the critical bugs from the trivial ones, causing important problems to get lost. I propose the follow guideline:

• ubuntu-bugcontrol and developers should set the importance as early as possible, ideally when the bug is first looked at.

Getting it exactly right is not crucial as someone else who knows the package well will look at it later, especially if the bug has a high enough importance. The key thing is to separate bugs into Wishlist, Low and Important bugs, where Important bugs are Medium, High or Critical. The Low and Wishlist bugs are usually easy to spot and it is important to separate these out from the few important ones.

The worst thing you can do is leave the bug at Undecided priority because that adds no new knowledge to the bug and it will remain hidden in the vast sea of such bugs until someone else looks at it.

Assigning bugs

The ability to assign bugs was originally used by developers to assign bugs for themselves to work on. This creates a to-do list for that developer or team. With a larger community of triagers, assigning is now also being used to when a community member has taken responsibility for triaging the bug so that there will be more useful information available when the developer looks at it. The problem with this is that important bugs can stay in a triaging limbo for a long time and may not get the attention of a developer before it's too late. So, I propose the follow guideline:

• A bug should be assigned to the person or team who will work on fixing it.

In a large community people will be naturally reluctant to assign work to other people (or they should be ...), especially when those people are volunteering their time. As a rule teams or individuals should assign bugs to themselves after confirming that they are filed against the right package and are important enough to be worked on for the current release cycle.

Assigning to a team

Triagers or developers should assign well triaged and well defined (or incomplete but clearly important) bugs to a development team such as kernel, mozilla, desktop or xorg. The teams should in turn have a (weekly?) review process where they evaluate new bugs and distribute them within the team.

Assigning to a developer

You should generally not assign a bug directly to an individual developer unless you have explicitly been asked to do so. Most developers use the assignment list as a personal to do list and need to be able to control that themselves. Bugs are generally pulled into this assignment list by the developers themselves or by their manager or team lead.

Bug statuses in Launchpad

Launchpad's Bug Tracker offers you seven main bug statuses two bug statuses only available to bug contacts - Triaged and Won't Fix. The following over view is based on the BugStatuses page in the Launchpad help wiki, but adds more detail relevant for Ubuntu workflow.

Available to everyone

• New: the bug is newly reported.

• Incomplete: the bug report is incomplete and needs more information before it can be triaged.

• Invalid: the report describes the software's normal behaviour, or is unsuitable for any other reason.

• Confirmed: a member of the community believes that this report describes a genuine bug in enough detail that a developer could start work on a fix.

• In Progress: a developer has taken responsibility to fix the bug and has begun work.

• Fix Committed: a developer has committed his/her fix to the project's codebase.

• Fix Released: a new version of the software, featuring the bug fix, has been released.

Only available to the Bug Contact

• Triaged: a member of the Bug Contact team considers that the bug report contains all information a developer needs to start work on a fix.

• Won't Fix: this is acknowledged as a genuine bug but the project has no plans to fix it.

Providing feedback

In a large project like Ubuntu where many different people with different levels of experience and areas of expertise work do bug triage it is vital to establish routines for disseminating information about individual bugs and the triaging process. A mix of bug tracker features and guidelines should be used to this end.

• Confirmed -> Triaged: any community member can set a bug to the Confirmed state while only those with recognised experience can use Triaged (ubuntu-bugcontrol and ubuntu-dev). In every other sense the two states are the same. This can be used to encourage a review process where members of ubuntu-bugcontrol can move bugs from Confirmed to Triaged and give feedback to the original triager.

• Email: Sometimes it useful to give feedback to a bug triager regarding how a bug was handled, but it might be best not to add bug mail noise.

Comments

• BrianMurray:

• Are you proposing that teams be assigned bugs that are Incomplete? That is what I gather from the assignment portion. If that is the case, to whom should bugs be assigned that have no package and are Incomplete?