UbuntuHardwareDatabase

Launchpad Entry: ubuntuhardwaredatabase
Created: komputes
Contributors: amorphous
Packages affected: checkbox, gnome-device-manager
Sites affected: hwdb.ubuntu.com , wiki.ubuntu.com/HardwareSupport

Summary

Ubuntu Hardware Database. Find Hardware. Find Answers.

Find Hardware - Look up hardware you already own. Make a pre-purchase decision on hardware thats doesn't advertise it's Linux compatibility status. Devs can be matched with hardware testers.
Find Answers - Hardware doesn't work - Find a fix/solution/bug. Upload your hardware info, rate your hardware. Share or find a solution to your hardware issue.

Release Note

As this spec deals with an ancillary project, it will not affect the average Ubuntu desktop. The use of a client, such as System Testing (checkbox), simplifies the process by submitting hardware information to Launchpad (which it already does - over 1 mission submissions already). The user would then import a submissions and add its components to the hardware database. Integration into a client such as device manager or jockey would allow the user to see the status of detected hardware. We can basically notify a user a user that hardware requires a certain action to be usable.

After developing this site, two software clients can be used in conjunction with the hardware compatibility. These should be the next steps to be taken.

checkbox submissions integration A global overview of the hardware database for the device in question is needed. Visually representing the database (making it useful) is a challenge that must be tackled. The current hardware database entries go to launchpad and sit there. From the hardware database, you would be able to import the submission from Launchpad, creating a hardware profile for a machine on the hardware database. This profile would be usable (readable, linkable, sortable, search-able) form of the data currently submitted to Launchpad. Once the submission has been ingested, the user would be presented with their hardware as well as the users results of each test, community results for each test, and editable fields allowing them to change an answer, or add more comments.

gnome-device-manager integration If future releases, it has been requested that hardware support and testing be integrated into a device manager which detects hardware, hardware status and notifies the user to open a package manager and get a package from the Ubuntu repositories, or when this is not available to visit the device page for a solution. This device manager would allow people to report individual hardware components to the database without having to submit a report for an entire computer.

Rationale

The Ubuntu community has been in need of a properly designed central web-based hardware database for many years now (started discussion in 2005). Users either need to find hardware or find answers. This site would allow ubuntu users to browse and extend hardware information contained within. This will allow more users to adopt Ubuntu with full hardware/driver support.

Here are the current issues we currently have which can be corrected with this project:

No central reference point for device compatibility information (hw problems, particular setup, purchase, recommendations).
Not all hardware available works out of the box with Ubuntu.
There is no "official" status of which hardware work and which does not.
Hardware bugs are many and disorganized.
We are not collecting and using useful information in the best manner.
We are not properly tracking hardware regressions in between releases.

User stories

By visiting the hardware database, $USER thinking of migrating $X computers to Ubuntu can check if owned hardware will work.
By visiting the hardware database, $USER thinking of buying $MODEL for use with Ubuntu. $USER can check if hardware will work before making the purchase.
By visiting the hardware database, $USER can have a look at the components of MB/PC $MODEL and any issues that may exist with this hardware.
By visiting the hardware database, $USER can check if hardware works with their particular release of Ubuntu.
Bug report about hardware can now be tracked to specific users who are able to test fixes.
Reporters of issues who are able to preform test can be directly contacted if they chose to enable MatchMaker. Reporter can link to (one of) their hardware submissions when communicating.

The benefits of creating such a resource:

Allowing users to know if a particular device will work before purchasing it.
Allowing users with a malfunctioning device, understand what needs to be done to make it work in Ubuntu.
Community participation/contributions resulting in increased quality of the operating system as it relates the hardware and the Linux kernel.
Centralized knowledge of what hardware requires attention. Keeping count of unique visits per page will allow us to detect issues.
More efficient bug reports resulting in faster fixes
Larger user-base and better first time end-user retention.
Properly tracking hardware regressions in between releases.

Assumptions

Every hardware device implementation has it's own page/entry, (Device + Subsystem, Architecture,Audio Card + Codec, Revision Number, BIOS Revision) have its sub-page attached to the parent device.
A subsystem (PC/MB Model) links to multiple component device pages/entries
Users can look up devices by make/model number, PCI ID and USB ID
Users participate by creating/commenting/rating hardware
Each page has the following features: picture(s), specs, ratings, comments, popularity, subscribe, matchmaker, profiles, status (partner, works, fix avail, unsupported)

Design

Here are second draft mock-ups:

Device Page

Import Hardware information

Implementation

One page per hardware device implementation. Each page contains: Search, Info, Release Filter, Rating, Device Picure,Related Bugs,Subscribers, Comment, Matchmaker. Integration into Launchpad is important. Whether the site interacts with Launchpad through an API (SSO, Bugs, HWDB submissions), becomes part of Launchpad, or is based on Launchpad code is yet to be decided. The user could submit information manually on the site, or pick a existing submission from the Launchpad hardware database to ingest. Once the information has been parsed, the user is presented with an option to rate their hardware and leave a comment for it. Each user in Launchpad should be able to create hardware profiles for each computer they use with Ubuntu.

Search: Quick search for PCI_ID USB_ID or device name.
Info Box: Field editable by hwdb admin only, holding an "official" status of hardware. Any approved solutions, links to device tech specs, alternate names / aliases, would also be found here.
Ratings: Does this device work "Out Of The Box". Similar to "This affects me" in Launchpad, this field would allow one vote per user per hardware implementation (and allow changes).
Picture: A picture of the device for quick recognition.
Related Bugs: Link to bug and specify release/arch affected.
Subscribers: Subscribe to a device and get notified whenever someone posts a comment. Non-working hardware? Be notified the minute a "fix" comment has been added.
Comment: Lists comments from users, denoting release/arch/IsAFix on each comment. Each comment is rated +1/-1 to have the most useful comment get on top and possibly be implemented into the InfoBox.
Matchmaker/Profiles: See who has this hardware. When submitting info, you have a choice to join the matchmaker program, announcing that you will help developers test this hardware.
Info Box/Specs
- Any technical specifications that can be imported from other sites or submitted by the users in this wiki field. These includes date hardware was first seen (based on pci/usb .ids $version), manual import or links to official manufacturer or 3rd party site specifications (wikipedia or other). The official status should only be changeable by members of the Canonical Group in Launchpad. The choices are partner, plug&play, fix/workaround available, unsupported. For hardware that has been certified through our Partner Hardware Program, there is a "seal" picture to be used on that page that clearly states "certified: works with ubuntu"
Ratings
- A star or number based rating system based on current owners of the hardware. Once the hardware info has been ingested, the user can now start rating the detected hardware. They mark their release and architecture and rate each device as "Works out of the box", "Works with fix", "Doesn't work" + "Comment/IsAFix". Contact me for MatchMaker.
Picture
- Allowing users to quickly identify their hardware is important in having a large range of users on this site.
Comments
- Comments will allow users to post workarounds and solutions pertaining to setting up the hardware device in Ubuntu. These comments will allow users to quickly resolve their hardware issue. To filter these comments we do what the following pages do: Digg (collapse bad threads and thumbs up good ones), eBay (user rating system), Yahoo Answers (raise best answer to top) and and even the Ubuntu Brainstorm (+ and - votes) and the Launchpad Answer Tracker (highlights best answer) have proved themselves to be effective. Comments have headers which contain information about the poster and his/her hardware (distro/release/comment_is_fix/comment_is_problem_report).
Popularity
- This will be an indication of what hardware Ubuntu users are using the most/least.
Subscribe
- This will simply subscribe you to the device comments without any input to the page. This is perfect for users with hardware that does not yet have a solution. They will be notified of "any comment" or "any solution/fix" on the page.
MatchMaker
- Owners of the hardware and developers can be matched up for troubleshooting. Users or developers who own the hardware can be seen as Protégées, and users or developers who are able to assist others in hardware support are seen as Mentors (device specific role). Mentors may ask Protégées to preform test in attempts to resolve a hardware issue. Integration with Launchpad will allow us to give karma to Mentors who have their solution voted as the best solution. Each member who has a device can sign up as a mentor if they are interested in helping. Owners of the hardware will be asked the following questions:
What Releases/Architectures/Kernel Versions did you test this hardware on?
How satisfied were you with the quality and functionality of the device [5 star Rating for each release]
How easy was it to make this work in Ubuntu - Works, Fix Avail (Comment/IsAFix), Unsupported
Would you like to join matchmaker (Mentor/Protégé)
Would you like to reference any documentation links? (specifications, help documents, bugs)
Profiles
- Who has this hardware? In what computer/configuration? Users can chose to make their profiles publicly available so that others can see who has the hardware. Users who have a matchmaker "Protege" status are asking to be contacted to troubleshoot or try solutions.

Code Changes

Looking to use many aspects of Launchpad instead of re-inventing the wheel.
- PPA has a "per-release" filter
- Need to link hardware device to bug and be notified of status
- Subscribers
- Affects me too (+1/-1)
- Flame algorithm concept

Migration

Data migration from multiple sources is to be done. Scripts are to import hardware status from different sources (API or Scrape). Here are a few sources that we already have eyes on:

Basic information on USB and PCI devices has been imported from pci.ids and usb.ids files
Importing checkbox data submitted to launchpad
HardwareSupport on wiki.ubuntu.com (users will be redirected to the new site)
UVC Webcams
OpenPrinting (Linux Foundation - Network Print/Scan/Fax is especially important)
SANE
ALSA

Test/Demo Plan

Stage one of planning:

Create the database tables per type PCI, USB, SUBSYSTEMS
Import data from pci.ids (includes devices and subsystems) and usb.ids
Manually migrate or find a way to scrape/automate data import from the Ubuntu Wiki
Import results from checkbox

Unresolved issues

Scheduled import of pci.ids, usb.ids and keeping track of name changes, versions and revisions, noting when a product was first seen, importing differentials.
Do we ignore anonymous submissions to hwdb.ubuntu.com or use all submissions in a "flame-like algorithm" where anonymous submission hold less weight?
Certain hardware may look the same but work differently based on device revision, BIOS revision, or audio codecs. How to display this clearly to users?
BIOS/ACPI - Solutions for this need to be per PC/MB model
Which computer system has this hardware (subsystem to device relation). Relation between device and subsystem (comments from device on subsystem are posted to device page but not vice-versa)
Who has this hardware to use Ubuntu SSO / Launchpad ID
Page content filter: by release, see all, or see only solution.
How to identify/classify Firewire/SCSI devices?
How to avoid messiness in the comment area over time: Managed Threads (vBulletin, StackExchange, LP Answers), Purging/Hiding/Filtering older comments.

BoF agenda and discussion

Other recommendations (future considerations):
- Less use of unofficial pages such as ubuntuforums.org and ubuntuHCL.org for Hardware compatibility information
- Integration into hardware resellers who sell Ubuntu-compatible devices and components (average cost, best price)
- Advertise machines that are certified and where to buy them
- Users can set bounties (using flattr or similar) for non-supported hardware so that if a driver is written, the developer is compensated.
- Translations
Related links (research of community needs/requirements used to build this spec):

CategorySpec