Ubuntu Wiki

• Launchpad Entry: kernel-karmic-hwdb-workshop

Summary

The goal of this spec is to produce a set of scripts which will allow the kernel team to mine data from the HWDB.

Release Note

A series of HWDB scripts have been uploaded to lp:~adeuring/+junk/hwdb-scripts. These scripts provide the ability to extract hardware information from a subset of the Ubuntu user base who have chosen to upload their hardware profiles. This data can help determine statistics such as the number of users of a specific device and bugs reported against that device. Those metrics can help prioritize work based on the number of affected users with that hardware.

Rationale

Leading up to the Jaunty UDS the Ubuntu kernel team had been holding discussions with the Launchpad team to investigate building a hardware database that would allow end users to submit their hardware profiles to Launchpad. In theory, those profiles could then be linked to bug reports or have their data mined in general to determine how widespread a hardware specific issue may affect the overall Ubuntu user base. Having an API/scripts to extract this data would help the Ubuntu kernel team prioritize work to be done and resources to be allocated. The API/scripts could also help with the overall kernel bug triaging process. They could be used to quickly determine which bugs relate to a specific piece of hardware or driver. This could then facilitate advanced warnings to users for known issues, calls for testing, or grouping issues to a master bug.

An initial set of API's/scripts were written during the Jaunty development cycle but there remains areas of improvement.

User stories

• We ran into a real world use case with bug 359392 for Jaunty. It would have been helpful if we could have queried all hwdb profiles to discover who had an i965 graphics controller [8086:2a02] and how widespread this issue would have been for the Ubuntu user base.
• Another real world use case was bug 263555 - "2.6.27 e1000e driver places Intel ICH8 and ICH9 gigE chipsets at risk". We could have queried the hwdb for affected users and sent them an advanced warning email notifying them of the issue.

• The kernel team would like to query all bugs tagged suspend/resume and return a list of bugs which use the rtl8187 driver. See the ubuntu kernel-team ml thread "Commonly found hardware in suspend/resume bugs.

Design

All scripts will be written in python and use launchpadlib. In particular, refer to the hwdb api

Implementation

Scripts

Some scripts have already been written and are available at http://bazaar.launchpad.net/~adeuring/+junk/hwdb-scripts/files

• bug-related-hardware.py

  • Walk through all hardware reports submitted by users being affected by a bug and count how often a device appears in these reports. If a device is mentioned more than once in a report, it is counted only once. You must specify a bug number(s) or tag.
  • Examples:

  • python bug-related-hardware.py 359392

  • python bug-related-hardware.py -t suspend

• subscribed-bugs-of-device-owners.py

  • Search for users owning the given device or having a device controlled by the given driver, and return Ubuntu bugs these people are subscribed to. You must specify a driver name, or bus, vendor ID and product ID.
  • Examples:

  • python subscribed-bugs-of-device-owners.py -d iwl3945

• device-statistics.py

  • Queries the entire hwdb and returns total number of devices owners, submitters, submissions etc.
  • Examples:

    $./device-statistics.py -nor -b PCI -v 0x8086 -p 0x2a02
    
    device owners: 45301 all submitters: 877776
    submissions with device: 55983 all submissions: 1179316
    total number of devices:  56537

    • Query only HWDB reports for Intrepid:

      • ./device-statistics.py  -b PCI -v 0x8086 -p 0x2a02 -s intrepid

    • Only intrepid/amd64:

      • ./device-statistics.py  -b PCI -v 0x8086 -p 0x2a02 -s intrepid -a amd64

    • You can also limit the counts to a driver and/or a (kernel) packagename, though this returns 0 for this device. Seems that most HWDB reports do not relate any driver with it. You'll get a bit more interesting driver/package-related numbers for the PCI device [0x10de:0x0429] (an NVidia graphics card), for example.

• device-related-bugs.py

  • Provides ability to search for bugs related to a device or a device driver
  • Examples:
    • Search for bugs tagged "suspend" or "resume" and have a hw profile which references the rtl8187 driver

      ./device-related-bugs.py -D rtl8187 -t "suspend resume" -r -T Any

    • Search for bugs tagged "suspend" and "resume" against the 8.10 distroseries and use the rtl8187 driver

      ./device-related-bugs.py -D rtl8187 -t "suspend resume" -r -T All -S 8.10

    • Same as above but restrict against the linux source package

      ./device-related-bugs.py -D rtl8187 -t "suspend resume" -r -T All -p linux

    • Can also search against a specific device

      ./device-related-bugs.py -d PCI:0x8086:0x2a02 -p linux -r

Current Unresolved issues

1. Doesn't work on Jaunty.

   • Bug: https://bugs.launchpad.net/ubuntu/+source/linux/+bug/358004

   • Possibly fixed upstream: https://bugs.launchpad.net/ubuntu/+source/linux/+bug/336067/comments/3

   • I've now upgraded my last system to Jaunty so I can not do launchpad script development until this is fixed.

   • This is a non-issue for Karmic

2. Performance. It doesn't matter how good the documentation or APIs are if it takes too long to get the information out. Queries should be taking seconds, not hours. For Example: Running `subscribed-bugs-of-device-owners.py -b PCI -v 0x8086 -p 0x2a02` to see how many people may be affected by the i965 freeze issues (bug 359392) took 10+ hours to run! I actually don't know the exact total run time because I walked away after 10hrs and came back in the morning to see if it had finished.

   • We should indeed provide a number of API methods. Queries "canned on SQL level" should be quite fast compared with the existing scripts which need thousands of API calls to achieve a result.

   • Abel has a branch which should improve the performance of the scripts he's currently written. ETA for branch being pushed is Fri 12, June 2009

   • Branch has been pushed

3. The main issue is the lack of linkage between bugs and hw-info. From what Abel has explained, the hardware information is submitted via checkbox while the bug reports are generated by apport (or filed through launchpad directly). The two utilities don't seem to know about each other. The linkage between hw profiles and bug reports needs to be bi-directional. Also, the way that existing scripts try to match up hw profiles to a bug doesn't take into account that a single user could have made multiple hw-info submissions related to different systems (desktop vs. laptop for example).

   • Right, the best option would be to automatically create a HW profile when apport reports a bug and to link the profile and the bug. That's something we should discuss during UDS, I think. This next-best option would be to allow bug reporters to link a bug with a hardware profile via a web UI. The problem here is that ca. 1/3 of all people who provided at least one HW profile actually submitted two or more profiles: It can be quite difficult to figure out which HW report belong to the machine affected by a bug. Martin had some interesting ideas how we can present the HW profiles on a web page, but I am not sure if/when we will have enough "developer capacity" to implement this. The third option is to provide a webservice API method which creates these links (would be also needed for option 1), and to write a script which creates these links. I think we should discuss options for better ways to link bug reports and hardware profiles at UDS in Barcelona. There are so many different people involved -- the QA team, checkbox developers, apport developers, LP developers -- that a meeting during UDS it is probably the best way to get something started.

   • Issue being tracked in Bug 379393 (Fix Released) - Currently launchpadlib has the ability to link hw profiles to bug reports. schwuk to update checkbox to store submission id (target karmic-alpha3 for completion of this). pitti to then update apport to use the submission id to link the profile and bug report.

   • Also see bug Bug 424382 (Fix Released in apport, Fix Released in Launchpad) - "FileBugViewBase should detect the header HWDB-Submission in data sent by Apport"

4. The api documentation is pretty poor. For example, you have to just "know" that most if not all properties of "team" are also available for "person". The documentation doesn't indicate that in any way. Bug 363440 was filed a bug about this.

   • HWDB API documented at https://edge.launchpad.net/+apidoc/#hwdb

5. Bug 338772 - Data from lshw should be submitted to Launchpad

   • 20090713 - Abel to start a discussion with Marc and Dave about this. Below are notes from Abel:

     1. I am not very familiar with the details how HAL will be replaced, but as I understand it, the successor is DeviceKit, which will provide similar data. So we could switch to DeviceKit. This would not solve the problem that server installations don't have HAL installed, or anything else DBus-related,

     2. use lshw. The problem here is that lshw does not list for example any USB device. This might not be a big issue from a kernel developer's viewpoint, but getting statistics about printer or scanner usage would be fine. And things like bug linking can also be useful for application developers. (And USB-related kernel drivers can also have bugs

     3. checkbox could traverse sysfs and use its own code to find the relevant data.

6. Bug 399319 - Remove the HAL dependency from Launchpad HWDB and checkbox

   • 20090714 - Abel, "I think we should not use lshw as a replacement for HAL data, but the HWDB client can of course include lshw output and Launchpad can store and present it."

7. 20090713 - Push scripts to a central hwdb project repository

   • Abel starting the process to get permission to start the project

Script Requests

BoF agenda and discussion

Translated into table above.

CategorySpec