UsefulBugMetrics
Summary
Identify additional bug statistics that will be useful to the Ubuntu team during the cycle to drive quality improvements and assist during release periods.
Rationale
There are fair number of existing bug statistics however, some of these could be expanded and some new ones should be created to take advantage of new items being tracked in Launchpad.
Currently, information is being gathered on the quantities of bugs with a specific status or importance. It would be useful to expand this out to include status and importance. Additionaly, the Incomplete should be more granular by breaking those numbers down into Incomplete w/ Response and Incomplete w/o Response. Another useful quantity are bugs reported via apport - those tagged as 'apport-bug' and 'apport-crash'.
The bug tracking system of Launchpad has been expanded to record some additional date information regarding bug tasks. The information currently being recored includes the following:
- date-created (the date the bug task was created)
- date-left-new (the date the bug task's status was first changed)
- date-incomplete (the date the bug task's status was last set to Incomplete)
- date-confirmed (the date the bug task's status was last set to Confirmed)
- date-triaged (the date the bug task's status was last set to Triaged)
- date-inprogress (the date the bug task's status was last set to In Progress)
- date-fixcommitted (the date the bug task's status was last set to Fix Committed)
- date-fixreleased (the date the bug task's status was last set to Fix Released)
- date-closed (the date the bug task's status was set to a end status)
- date-assigned (the last date the bug was assigned to someone)
Currently, this information is only available via the +text interface for a bug report. Additionally, this information is not particularly useful by itself. Rather, these dates becoming more useful when considering the difference between any two dates and over a package as a whole. Subsequently, a new tool should be developed to find the differences and aggregate the information for packages or teams.
Use Cases
- The Mozilla team wants to analyse their bug workflow and throughput from New to Fixed to identify the bottlenecks where they should focus their efforts. The useful bug metrics displayed on a Mozilla summary page lets them target their effort and monitor their progress.
- Pedro is planning a bug day to focus on the bugs that have been left in the New and Incomplete states for the longest times. The new bughelper tools let's him produce the appropriate wiki pages.
Assumptions
Design
Currently the bughelper suite of applications complements the bug tracking system by providing statistics regarding quantities of bug reports and specific lists of bug reports. The bugnumbers tool has the ability to determine the quantity of bug reports matching specific criteria. However, bugnumbers has an overwhelming number of options and is primarly used to create lists of bug numbers matching specific criteria. Subsequently, the statistics options should become a separate tool and be expanded to include new options. The new bug lifecycle analysis tool should also be included in the bughelper package.
Implementation
- A new lifecycle tool will be added to the bughelper suite which will also take over some of bugnumber's current fetures (leaving a simpler bugnumbers)
- A range of new throughput statistics will be produced with the bug-lifecycle tool for various slices of time (a given week, month, from the start of the project). Examples:
Time in New: Sum(date_left_new - date_created) / Count(bugs_in_slice)
Time to triage: Sum(date_triaged - date_created) / Count(bugs_in_slice)
Time to assigned: Sum(date_assigned - date_created) / Count(bugs_in_slice)
Time to fixed: Sum(date_closed - date_created) / Count(bugs_in_slice)
Triaged to fixed: Sum(date_closed - date_triaged) / Count(bugs_in_slice)
need more examples
Key statistics will be displayed for select packages (or rather package groups) See: [:QATeam/Specs/PackageStatus:PackageStatusPages].
Code Changes
The statistics features of bugnumbers will become a separate tool from bugnumbers. This new tool's functionality will be expanded to return quantities of bugs in a state and importance. Quantities of Incomplete bug reports will also be broken down into Incomplete with and without a response.
The bughelper package relies on python-launchpad-bugs for the parsing of bug pages and accessing attributes of a bug report. Therefore python-launchpad-bugs will need to be extended to make these new bug task dates accessible.
After these new dates are accessible it will be possible to create a new application in the bughelper package to calculate the time for a change to occur for a bug task. This tool should accept any list of bugs, whether it be a package's bugs, a team's bugs, a project's bugs or a whole distribution's bugs, and calculate the time to any change.
After this functionality has been added the most useful statistics should be made publicly available for packages and teams with a substantial quantity of bug reports.