Triage

Revision 94 as of 2009-06-23 22:10:38

Clear message

In a hospital, triage happens the instant a patient arrives through the emergency room doors. His vital signs are checked, his status assessed, and he gets sorted in amongst all the other patients waiting for treatment.

Bug triage is a lot like that as we assess bugs to determine whether or not they have enough to be worked on and assign a priority to them as soon as possible. Without any risk of death.

Ubuntu receives an incredibly large number of bug reports every day through our bug tracking system. Each one of these needs to be read, assessed, and sorted so it can be fixed. This is where we could use your assistance with HelpingWithBugs. For a visual representation of the bug triage process, see these nice Flow Charts.

Untriaged bugs

Untriaged bugs can be found via this link. Once a bug's status is no longer New it will no longer show up in this search.

You can use the Advanced search in the BugTrackingSystem to find New bugs for a specific package. Go to the source package's bugs advanced search page and look for:

  • Status: New

  • Importance: Undecided

  • Assigned to: Nobody

New bugs

Every bug report is a conversation with the reporter. The first contact any reporter usually has with the Ubuntu community is through a bug triager, who tries to put together a complete bug report. It's very important that we give a good impression, so please be polite and try to use your best English.

In order to help triage bugs, you have to be able to find them. Fortunately, this is easy. You can find out about new reports in one of three ways:

The first method is far more preferable, as you won't get a mailbox stuffed full of mail each time you read it. I tried the third option for a while and couldn't keep up at all.

If you enter #ubuntu-bugs-announce, you'll soon see the following types of messages fly by:

New bug: #1 in Ubuntu "Microsoft has a majority market share" [Undecided,New] http://launchpad.net/bugs/1

To start, just pick one of the recent ones and open the link in your favourite browser. Try to pick bugs which affect software or parts of the system that you are familiar with yourself, as this will help you decide how complete the report is, and make it easier for you to reach the grail of reproducing the bug. If no one else has commented yet, then this bug could be yours!

information_little.png This page is part of the Bug Squad’s KnowledgeBase - pages with information about how to triage bugs.

Duplicates

Many of the reports filed about Ubuntu are actually duplicates of bugs already reported. This can happen after a high profile bug has been introduced into Ubuntu, causing a lot of users to report it. Other times, reporters don't know how to check if the same bug has already been filed, or it is hard for them to determine if their bug is the same as another. Finding these duplicate bugs and aggregating information in one bug report is a very valuable contribution.

Bugs are duplicates when they have the same root cause. Determining this is a skill that you'll pick up as you become more familiar with a particular package or subsystem. Bugs are not necessarily duplicates if they have the same effects. For example, many different bugs can cause X not to start. Determining which bug a particular report refers to is part of triaging. If in doubt, ask for a second opinion. It is probably also sensible to ask the reporter to take look at the possible duplicate and to help with the decision. Reporters are normally interested in helping with their own bug reports!

When you first look at a new bug, try to find an existing bug in the system that describes the new one. Here's how:

Warning /!\ Kernel Team policy requires that only a kernel maintainer can set a kernel bug as a duplicate of another bug. See the kernel policy for details.

  • Click the "List open bugs" link at the bottom of the bug page or

  • Click the "Bugs" tab at the top of the page
    • Both ways will produce a list of bugs about the same package
  • Look for bugs with similar descriptions or related titles.
  • If they describe the same root cause, decide which report should be the primary one. This should be the one that's the most understandable and contains the most information, not necessarily the oldest (lowest numbered) bug.
  • For the other report, add a comment like this standard reply

    Include: Nothing found for "^== A duplicate ==$"!

    information_little.png This page is part of the Bug Squad’s KnowledgeBase - pages with information about how to triage bugs.

    These standard responses along with other amazing scripts are also available as a Firefox extension in a PPA at:

    https://launchpad.net/~gm-dev-launchpad/+archive/ppa

    When you update one of these responses, you should also update the response used by the Firefox extension. These responses can be found in the bazaar branch lp:ubuntu-bugcontrol-tools in the file gm-xml-files/bugsquad-replies.xml. To check out a branch, use

    bzr branch lp:ubuntu-bugcontrol-tools

    Only members of bug-control can commit to this branch. But you can submit a merge proposal, Brian Murray will review/merge it (see this email message), and they can be used when replying to a bug report if merged.

    Not described well

    Thank you for taking the time to report this bug and helping to make Ubuntu better. Unfortunately, we cannot work on this bug because your description didn't include enough information. You may find it helpful to read "How to report bugs effectively" http://www.chiark.greenend.org.uk/~sgtatham/bugs.html. We'd be grateful if you would then provide a more complete description of the problem.

    We have instructions on debugging some types of problems at http://wiki.ubuntu.com/DebuggingProcedures.

    At a minimum, we need:

    1. The specific steps or actions you took that caused you to encounter the problem.
    2. The behavior you expected.
    3. The behavior you actually encountered (in as much detail as possible).

    Please also ensure that you include the release and flavour of Ubuntu that you are using.

    Thank you!

    Missing Steps to Recreate Bug

    Thank you for taking the time to report this bug and helping to make Ubuntu better. Please answer these questions:
    * Is this reproducible?
    * If so, what specific steps should we take to recreate this bug?

    This will help us to find and resolve the problem.

    Missing Apport Information

    Thank you for taking the time to report this bug and helping to make Ubuntu better. Please execute the following command only once, as it will automatically gather debugging information, in a terminal:
    apport-collect BUGNUMBER

    When reporting bugs in the future please use apport by using 'ubuntu-bug' and the name of the package affected. You can learn more about this functionality at https://wiki.ubuntu.com/ReportingBugs.

    Package or domain specific questions

    The following packages or type of bugs require the specific debugging information:

  • Then click the "Mark as Duplicate" link at the top left of the bug report page, and enter the number of the primary bug.

The default searches in Launchpad and used above will only look at Open bugs. It may also be worthwhile to go through the list of Invalid and Won't Fix bugs which you can look for by using the Advanced search. There is also a standardized bug tag for ones likely to have lots of duplicates - metabug.

When marking a bug report as a duplicate of another (master) bug report, please also check whether the master bug report is marked as a private. If so, the master bug report might not be visible to the current bug reporter. When the parent bug is indeed marked as private please check why it is so. If it's only private because apport makes all bugs private by default, but the coredump has been removed and none of the apport attachments contain anything private, it may be made public. If it does contain confidential information, the bug should remain as private and it is better to search for another bug which could be safely marked as the master bug. For any guidance regarding the private status of master bug and marking another bug a duplicate of it, please ask in the IRC Channel of the Bug Squad (#ubuntu-bugs).


CategoryBugSquad

Apport crash reports

A considerable number of bugs concern program crashes which are reported semiautomatically with Apport and get pre-processed automatically by some bots in the Canonical data center. These bots try to generate a fully symbolic stack trace and check for duplicates.

In Feisty and early Gutsy, those bugs were public, so that everyone could see them. This created a privacy problem, though, since core dumps and stack traces could contain potentially sensitive information. Also, crash reports generate a lot of bug email noise. With the automatic duplicate checking, a fair amount of the reported bugs are completely irrelevant for triagers.

In current Gutsy, these problems have been mitigated: bugs are filed with the "private" flag enabled, i. e. only the reporter and subscribers can see it. The reprocessing bots will subscribe the ubuntu-bugcontrol team, but without sending bug email to the team members.

Thus crash bugs differ from other bugs in two important aspects: they need to be checked for sensitive data, and there will not be any initial bug mail for them until they become public. Triagers should check the following things:

  • If the crash still has a CoreDump.gz attachment, then it was not possible to automatically get a fully symbolic stack trace and check for duplicates. In this case, the bug will be tagged with apport-failed-retrace. If the stack trace looks good enough, the CoreDump.gz attachment should be removed with the (edit) link in the attachment box. If the retrace failed completely, just leave the bug alone.

  • If there is no Stacktrace.txt (retraced) attachment, then the most probable reason is that the CoreDump.gz attachment is broken. Please check with Martin Pitt (pitti in IRC, martin.pitt@ubuntu.com) about the reason, he can look into log files.

  • Check if the Stacktrace.txt (retraced) attachment has anything that looks like sensitive data passed as function arguments. Examples are passwords, things that look like bank account numbers, CSS keys, etc. If you don't find anything, you may mark the bug as public ("Visibility/security" in the top left pink box). This is not required, though, it is fine to keep the bug private throughout its lifetime.

Except for those privacy issues, crash reports should be handled like normal bugs in terms of duplicate searching/marking, upstream forwarding, etc.

information_little.png This page is part of the Bug Squad’s KnowledgeBase - pages with information about how to triage bugs.

Improving a bug report

Part of the bug triaging process involves ensuring that the bug is valid, well described, and contains adequate information for a developer to start working on it.

To be considered complete, a bug report should normally contain:

  • The version of Ubuntu that the reporter is running
  • The version of the package the reporter is using
  • The actions taken to produce the problem
  • Whether or not it is possible for the reporter to reproduce the bug
  • The expected result of these actions
  • The actual result of these actions

Not every bug reported contains all this information though. So as triagers we should ask for all of the above if they are missing. Sometimes a particular piece of information may be clear from the rest of the report, or clearly not needed for a particular report. A good test is to see if you can reproduce the bug yourself on the basis of the available information. If in doubt, it may be better to discuss with the reporter before marking the bug as incomplete.

Additionally, certain classes of bugs and specific packages require more detailed information like configuration and log files. The DebuggingProcedures page contains a list of links to detailed information to gather.

Since most reports probably won't be complete, you'll have to start a conversation with the bug reporter. Ask the reporter for more information by doing the following:

  • Click on the bug task name, which is usually the package name, in the yellow horizontal line.
  • Change the "Status" field to "Incomplete".
  • Ask for the additional information required in the "Comment on this change" field.
  • Click the "E-mail me about changes to this bug report" check box, so that you'll be subscribed to the bug
  • Click "Save changes".

As a subscriber to the bug, you will be e-mailed when the reporter responds.

Even if the bug report is complete it could probably still use some improvement.

  • Is the bug's summary descriptive of the bug? This will help people find bugs easier.
    • consider "no r5xx support in radeon driver (X1300, X1400, X1600, X1800, X1900, X1950)" vs

    • "update-manager" or "Screen Saver Issues" or "Buffer I/O Error"

Incomplete bug expiration

Ubuntu doesn't have the resources to address every bug, so we want to focus our resources on the bugs we really need to fix. Having tens of thousands of bug reports in the system which we can't and won't get to is not constructive, and it makes the bug fixing we CAN do less efficient. We focus our effort on bugs introduced in the Ubuntu packaging process, or specific to the Ubuntu versions of packages, and bugs which create very significant issues for many users. We also focus attention on bugs which are well-defined and complete, because those are the ones developers have the best chance of addressing in a reasonable time.

Also, many bugs are fixed from release to release because they are addressed in new upstream releases which get packaged for the new Ubuntu release. It can be difficult to get the attention of the bug reporter if they have upgraded to a new version. However, it is important to try and establish if an incomplete bug has been addressed in a new version.

Therefore, bugs which are marked Incomplete will eventually be "expired", and drop off the primary reporting lists for developer attention.

If a bug is not fully defined, please set it to Incomplete and ask if it is still valid. If a new release of Ubuntu has been made, it is also reasonable to ask if the bug still manifests in the new release and set the bug to Incomplete. Unless the reporter resets the bug status the bug report will be set to expire.

Confirming

When you have a complete report, and there is enough information to debug the program, you can confirm the report. How do you know there is enough information? Here are some example criteria, there is no order to these criteria and you can do them in the order of your preference, ANY of which is sufficient:

  • Is there a patch that claims to fix the bug?
  • Are there sufficient log files and crash dumps, as outlined in DebuggingProcedures?

  • Can you reproduce the bug yourself?
  • Does another distribution have a complete and confirmed bug?
  • Does the upstream author have a complete and confirmed bug?

If ANY of these conditions are satisfied, you can Confirm the report. To do this:

  • Change the "Status" field to "Confirmed".
  • Assign the appropriate "Importance" value, according to Bugs/Importance.

  • Change the "Assigned to" field to "Nobody".
  • Click "Save changes".

Do not be worried if a bug you have confirmed, which cannot be sent upstream stays "confirmed" for a very long time. Ubuntu has many, many bugs and the devs will look at confirmed bugs first.

Status and Importance

If in doubt, the maintainer (or the one working on the bug) should change the Status and Importance. It usually reflects the status of this work or reflects how the bug fits into her/his working time.

Please check Bugs/Importance and see Bugs/Status.

Forwarding upstream

Upstream Bug Resources

  • The Upstream Report - Bug linkage percentages for the top 50 projects in Ubuntu, sorted by open bugs. Please note that this report is still in Beta. Instructions for using the report.

  • Bugs that need forwarding to the Upstream bugtrackers.

  • Unlinked upstream bugs - Sometimes people add a link to an upstream bug tracker in the comments of a bug but don't bother creating an upstream task. This is a list of possible targets that can be linked. (Lots of low hanging fruit in this list, but also a bunch of dupes and ones inappropriate to be linked.)

  • How to set a bug watch

  • Why Upstream? - Thoughts from freedesktop.org on why sending things upstream is a good idea.

  • There are also some cases where you don't need to file a bug upstream, for those cases please have a look at When not to Forward a bug upstream

Upstream bug trackers

Instructions for how to file bugs in specific upstream bug trackers. (Feel free to add more!)

Marking a Bug as Requiring Forwarding

If you find a bug that needs forwarding, it is best to forward it immediately. However, if you are running short on time, you can still mark the bug as requiring to be forwarded. To do this:

  • Open an upstream task (you click on "Also affects project"), but do not assign a bug watch (after the page with "Record as affecting another project" on the page with "Confirm project", you click on the "I just want to register that it is upstream right now; I don't have any way to link it."). Mind the wording (there is a bug filled against LP bug #353097)

  • This will mark the bug as "needs forwarding"
  • You can search for those bugs in the "Advanced Search" by selecting the "Show only bugs that need to be forwarded to an upstream bugtracker" option.

Invalidating

Sometimes, you will have to invalidate a bug report. This may be because the problem is not reproducible, the program was designed to behave a certain way.

There will be a few bug reporters who never get back to you and there is not enough information for the bug to be worked on. You will want to mark those bugs also as Invalid.

The best thing to do here is to politely decline the report while thanking the user for submitting it. There are some useful Bugs/Responses that you can use in these cases.

There is no need to reject bugs that are already marked as a duplicate of another bug. Doing so creates bug mail noise as every subscriber of the duplicate bug and the master bug will receive e-mail that the status of the bug changed from something to Invalid.

Include: Nothing found for "^== Incomplete bugs without a response from submitter =="!

information_little.png This page is part of the Bug Squad’s KnowledgeBase - pages with information about how to triage bugs.

These standard responses along with other amazing scripts are also available as a Firefox extension in a PPA at:

https://launchpad.net/~gm-dev-launchpad/+archive/ppa

When you update one of these responses, you should also update the response used by the Firefox extension. These responses can be found in the bazaar branch lp:ubuntu-bugcontrol-tools in the file gm-xml-files/bugsquad-replies.xml. To check out a branch, use

bzr branch lp:ubuntu-bugcontrol-tools

Only members of bug-control can commit to this branch. But you can submit a merge proposal, Brian Murray will review/merge it (see this email message), and they can be used when replying to a bug report if merged.

Not described well

Thank you for taking the time to report this bug and helping to make Ubuntu better. Unfortunately, we cannot work on this bug because your description didn't include enough information. You may find it helpful to read "How to report bugs effectively" http://www.chiark.greenend.org.uk/~sgtatham/bugs.html. We'd be grateful if you would then provide a more complete description of the problem.

We have instructions on debugging some types of problems at http://wiki.ubuntu.com/DebuggingProcedures.

At a minimum, we need:

1. The specific steps or actions you took that caused you to encounter the problem.
2. The behavior you expected.
3. The behavior you actually encountered (in as much detail as possible).

Please also ensure that you include the release and flavour of Ubuntu that you are using.

Thank you!

Missing Steps to Recreate Bug

Thank you for taking the time to report this bug and helping to make Ubuntu better. Please answer these questions:
* Is this reproducible?
* If so, what specific steps should we take to recreate this bug?

This will help us to find and resolve the problem.

Missing Apport Information

Thank you for taking the time to report this bug and helping to make Ubuntu better. Please execute the following command only once, as it will automatically gather debugging information, in a terminal:
apport-collect BUGNUMBER

When reporting bugs in the future please use apport by using 'ubuntu-bug' and the name of the package affected. You can learn more about this functionality at https://wiki.ubuntu.com/ReportingBugs.

Package or domain specific questions

The following packages or type of bugs require the specific debugging information:

Feature Requests

If the report is actually a feature, the Importance of the bug should be set to 'Wishlist' by someone in the 'Bug Control' group. Paste the bug # in #ubuntu-bugs and say that you think the bug should be set to 'wishlist'. Someone will notice and set it for you although not necessarily immediately.

In the meantime, you can leave this standard comment:

Thank you for taking the time to make Ubuntu better. Since what you submitted is a Feature Request to improve Ubuntu, you are invited to post your idea in Ubuntu Brainstorm at [WWW] https://brainstorm.ubuntu.com/ where it can be discussed, voted by the community and reviewed by developers. Thanks for taking the time to share your opinion!

Regressions

It is especially important that we catch bug reports concerning regressions and bring them to the attention of the developers. This includes regressions in current stable releases, in updates, in the proposed queue or potential regressions in the next release. These should be labeled with the following tags respectively: regression-release, regression-update, regression-proposed and regression-potential.

Once tagged, the bugs will appear on the regression tracking page and will from there be escalated to the development teams. See QATeam/RegressionTracking for more information.

Special types of bugs

Not all bugs are the same. Most bugs are code or packaging defects, but some groups also use bugs for other things. Careful attention must be paid to these bugs so as not to disrupt team processes. These classes of bugs have special rules and meanings for different statuses. They may even have different meanings depending on where Ubuntu is in its release cycle. Unless a triager is familiar with the specific process in question, adjustments to these bugs are likely to be problematic.

Developer Process Bugs

You should generally not change bugs of this type unless working with a developer/packager.

For packages in Universe/Multiverse, please see Sponsors Queue page for more details.

Needs Packaging Bugs

You may find a bug with a subject like: [needs-packaging] <package>. These bugs are used to track software requested for inclusion in Ubuntu. For these bugs, please:

  • check to see if it is already in Ubuntu (see http://packages.ubuntu.com, or run rmadison <package>): if it is already in the archives, then mark it as Invalid, and add a comment as to why;

  • if not in Ubuntu, then check Debian (see http://packages.debian.org, or run rmadison -u debian <package>). If it is in Debian, mark it as Invalid, and add a comment stating:

Packages for this software appear to exist in Debian already. Ubuntu has semi-automatic tools to sync new packages from Debian so it will most likely appear in the next Ubuntu release.
  • if not in Debian or Ubuntu, then set the status to Confirmed, the importance to Wishlist (if possible), and add the tag needs-packaging if absent.

  • additionally one could check the Work-Needing and Prospective Packages page at Debian. In the event you find a bug for the same software requested in the Debian bug tracker please add an upstream bug watch for it.

  • see the Example Request for desired information.

Warning /!\ Note: not all package names are intuitive! Also use package descriptions to help guide you.

  • If you are unsure as to whether the bug that you are looking at fits in one of these categories, feel free to ask in #ubuntu-bugs or #ubuntu-motu on Freenode.

Security Bugs

The SecurityTeam uses a similar but slightly different process for Triaging bugs. If you think this may be a security bug or are triaging a bug flagged as security, please be sure to read SecurityTeam/BugTriage before proceeding.

Looking through your bugs

Perhaps you want to look at a list of bugs you are working on, to see if anyone has replied.

A good way to get a list of these bugs is to look at your related bugs. This can be found in your "Bugs" page at Launchpad:

You can see you subscribed bugs at https://bugs.launchpad.net/people/+me/+subscribedbugs, or the bugs you've commented on at https://bugs.launchpad.net/people/+me/+commentedbugs. Links to both are available at your main "Bugs" page.

You should go through your bugs fairly often, maybe once a week, to make sure you haven't missed any new comments or changes. This includes bugs you have reported upstream. If there is an important fact from an upstream bug you should add it to the ubuntu bug report.


Go back to HelpingWithBugs.

CategoryBugSquad