RestrictedManagerRewrite

Differences between revisions 5 and 6
Revision 5 as of 2007-11-01 17:56:03
Size: 4742
Editor: 12
Comment: add design
Revision 6 as of 2007-11-15 22:59:02
Size: 10486
Editor: p54B3A19E
Comment: turn this into a proper spec
Deletions are marked like this. Additions are marked like this.
Line 10: Line 10:
The original code architecture of restricted-manager was very narrow, focusing on non-free kernel drivers in a Gnome application. A lot of new features have been bolted on this original structure, such as the KDE frontend, grouping of handlers, support for firmware installation, or non-free support packages of drivers which are free by themselves. This made the code very interdependent and hard to maintain/bugfix/understand.

We review the list of existing and proposed features and create a new code architecture which will be rewritten from scratch (with reusing most of the existing code modules, of course).
We propose a new code and driver database design for
restricted-manager which will make it maintainable again and also be
suitable as a proper upstream project, so that other distributions can
use it as well.

We review the list of existing and proposed features and create a new
code architecture which will be rewritten from scratch (with reusing
most of the existing code modules, of course).
Line 16: Line 21:
This section should include a paragraph describing the end-user impact of this change. It is meant to be included in the release notes of the first release in which it is implemented. (Not all of these will actually be included in the release notes, at the release manager's discretion; but writing them is a useful exercise.)

It is mandatory.
The behaviour of restricted-manager should not change in a visible
way, so this should not be mentioned in the release notes.
Line 22: Line 26:
This should cover the _why_: why is this change being proposed, what justifies it, where we see this justified. The original code architecture of restricted-manager was very narrow,
focusing on non-free kernel drivers in a Gnome application. A lot of
new features have been bolted on this original structure, such as the
KDE frontend, grouping of handlers, support for firmware installation,
or non-free support packages of drivers which are free by themselves.
This made the code very interdependent and hard to
maintain/bugfix/understand.

The current version also has a lot of Ubuntu specifics hardcoded,
which renders it hard to use for other distributions without
significant changes.
Line 26: Line 40:
== Assumptions ==  * Jon is a RedHat developer. He decides that the next Fedora release
 should get an easy to use frontend for installing restricted
 drivers. He creates a branch of restricted-manager, implements the
 small distro specific "OSLib" class, and has it running. Since this
 was so easy, he has some time to fix some bugs, which he
 contributes back to upstream.

 * Richard does not want to provide an interface for restricted
 drivers in his distribution, since they are evil and from the dark
 side of the force. However, he needs a GUI for providing free
 third-party drivers which are hosted outside of his distro's
 repository and which are maintained separately. He calls
 restricted-manager in "free third-party drivers" mode. For this
 reason upstream renamed the project to something less focused on
 restricted drivers.

 * Martin writes a handler for installing the Broadcom firmware on
 demand. Three months after releasing Hardy the download URL
 changes. He just sets a new URL in the central database and users
 can continue to get the firmware with a snap of a finger without
 the need to provide an SRU.
Line 30: Line 64:
You can have subsections that better describe specific parts of the issue.  * Move away from being Ubuntu specific to a proper upstream project.

 * Handlers are DE agnostic, but they sometimes need to ask questions
 → need to call abstract UI message boxes etc.

 * Application logic and UI workflow is encapsulated in an abstract UI
 class. This gets subclassed to implement KDE/GNOME/CLI/a
 noninteractive test suite implementation.

 * Basic handler must not make assumptions about kernel modules, etc.,
 since there are many device drivers which are not tied to a kernel
 module (e. g. sl-modem-daemon, printer drivers, file system
 drivers, etc.).

 * As a consequence of the previous point, each driver needs to be
 able to decide on its own whether it is applicable on the current
 system. The application framework should provide common
 functionality like modalias reading/overriding, lsmod/lspci evaluation,
 etc. (i. e. provide a list of available and enabled hardware),
 kernel module blacklisting, etc.

 * Supports different handler types:
  * kernel module (vmware/OSS)
  * kernel module + additional magic (nvidia/ati, sl-modem-daemon)
  * kernel module + shipped firmware (ipw3945)
  * kernel module + non-shipped firmware (bcm43xx)
  * grouping (all three vmware modules, or all two dozen OSS drivers)
  * printer driver
  * Fuse implementation to support a new file system type.

 * A handler knows whether it has a free or restricted driver.

 * The application can be started in any of three modes:
  * Show non-free drivers only (as in Ubuntu, since we do not want to
  promote providing free drivers without getting them into the main
  kernel)
  * Show free drivers only (Richard's use case from above, there is an
  actual real-life request for this feature)
  * All available drivers (for vendors that want it).

 The user interface and strings change accordingly.

 * Encapsulate all distro specific code in an abstract `OSLib` class
 and provide distro specific implementations in concrete subclasses.
 This should be the only class downstreams have to override and
 implement (aside from providing concrete custom handlers). As an
 example, the Ubuntu implementation will be aware of handling
 `/etc/default/linux-restricted-modules-common`.

 * Drop the `runme_as_root()` madness, since it is inappropriate for other distros, makes assumptions about the authentication backend, and is only used for a corner case (enabling nvidia driver in compiz control panel, but this could run r-m through gksu itself if it wants to).

 * The handler → driver mapping should not be hardcoded any more. The
 mapping is maintained on a central location (initially on
 http://www.drivertool.org) where all interested distros can
 maintain their own mapping to a distro package, URL, etc., and where
 distros can collaborate to maintain this database.

 * For this phase, distros will ship all the available handlers by
 default, so that they can detect availability. In the future, it is
 desirable to move the handlers themselves to a central place, so that
 third-party vendors can provide their own. We currently do not
 provide a solution how to get them from that central place to the
 distribution. However, the new program design should not assume any
 tight coupling between the core code and handlers. (Note: this is
 already true for the current design, any package can ship a handler,
 at which point it becomes available in r-m.)
Line 34: Line 133:
This section should describe a plan of action (the "how") to implement the changes discussed. Could include subsections like:

=== UI Changes ===

Should cover changes required to the UI, or specific UI that is required to implement this

=== Code Changes ===

Code changes should include an overview of what needs to change, and in some cases even the specific details.

=== Migration ===

Include:
 * data migration, if any
 * redirects from old URLs to new ones, if any
 * how users will be pointed to the new way of doing things, if necessary.
=== Handler-Driver mapping ===

http://www.drivertool.org is a distribution neutral place which can be
used for hosting the central mapping. This site is currently hosted
privately by Jon Masters, but if traffic becomes an issue, it would
need some contributions by Canonical, RedHat, and other Linux vendors
which participate. However, the amount of data that gets exchanged is
very small.

Any computer can do a query on this database. As an input, it provides
a type/value pair, where type is 'modalias', 'PCI vendor/product ID',
'Printer manufacturer and model', etc., and the value is the
respective value for that type. This ensures that

 * the system is extensible to accomodate any kind of hardware
 identification
 * the amount of data sent by the client is minimal (in particular, it
 does *not* send the OS vendor, version, any serial numbers, etc. for
 privacy reasons).

The response is a list of available entries, where each entry has the
following fields:

 * OS Vendor
 * OS version
 * set of type/value pairs

The type/value pairs can be interpreted freely for each OS vendor
(distribution), but there should be some standard types like 'url',
'packagename', 'md5sum', 'sha1sum', etc.. For Ubuntu, we additionally
define the data type 'repository', which would be an URL to an apt
repository.

With this design, each client always gets the full list of entries for
all OS vendors. This compromise is done to avoid exposing the OS
vendor and version, and other information which could be used for
tracking, collecting data for security attacks (like providing an IP
list of known-vulnerable OS versions); the lists are not expected to
become very big, too (in the order of 50 entries, which should amount
to some 5 KB).

==== Example 1: bcm43xx firmware ====

Client → DB:
{{{pci
14E4/4325}}
DB → Client:
{{{
Ubuntu/all
    url http://ubuntufriends.org/firmware/bcm43xx.o
    sha1sum DEADBEEF42
SuSE/10.3
    package bcm43xx_firmware
    component non-free
}}}

=== Example 2: nVidia driver ====

Client → DB:
{{{modalias
pci:v000010DEd00000098sv*sd*bc03sc*i*}}
DB → Client:
{{{
Ubuntu/7.10
    package nvidia-glx-new
    package linux-restricted-modules-2.6.22
Ubuntu/8.04
    package nvidia-glx
    package linux-restricted-modules-2.6.24
Fedora/8
    repository http://www.fedora-nonfree.org/fedora8/
    package nvidia-glx
}}}

=== Class layout ===
attachment:restricted-manager.png

(Dia source: attachment:restricted-manager.dia)

=== Collaboration with other distributions ===

 * Contact Debian, Novell, Mandriva for joining the project, contact with
 RedHat is already established with Jon Masters.

 * Create upstream product on Launchpad for trunk, invite distros to
 host their branches there, too (this is not a requirement, though, LP
 just provides a great hosting platform for bzr). (OK with Jon Masters
 from RedHat).

 * Create two mailing lists on a vendor neutral platform (LP does not
 have MLs yet), for now this will happen on
 http://www.linux-foundation.org/en/Driver_Backport:
  * ML for design and code coordination between vendors. (Posting
  limited to members, moderated subscription, should have open
  archives)
  * ML as an official contact point for interested third parties which
  want to provide their drivers in the central DB (Free posting for
  all, no public archive).

 * As one of the first bikeshedding discussion adventure, the ML will
 discuss a proper name for the project. "restricted-manager" does not
 suit the "free third-party drivers" use case, and is not a proper
 noun either. The name should not actually appear prominently in the
 UI, it's just the project name as appearing in Launchpad, the source
 code, etc. (Current candidate: "3PO" or "Threepio", suggestions
 welcome).
Line 53: Line 242:
(TODO)
Line 58: Line 249:

This should highlight any issues that should be addressed in further specifications, and not problems with the specification itself; since any specification with problems cannot be approved.

== BoF agenda and discussion ==

Use this section to take notes during the BoF; if you keep it in the approved spec, use it for summarising what was discussed and note any options that were rejected.

Design needs to accommodate these features:

 * move away from being Ubuntu specific to a proper upstream project

 * DE agnostic handlers, but they sometimes need to ask questions → need to call abstract UI message boxes etc.

 * HW detection logic and workflow with abstract UI → KDE/Gnome/CLI/noninteractive test suite implementations

 * basic handler must not make assumptions about kernel modules, etc.
 
 * handler types:
  * restricted kernel module (vmware/OSS)
  * restricted kernel module + additional magic (nvidia/ati)
  * free kernel module + additional magic (sl-modem-daemon)
  * free kernel module + firmware in l-r-m (ipw3945)
  * free kernel module + non-shipped firmware (bcm43xx)
  * grouping (all three vmware modules, or all two dozen OSS drivers)
  * support other use cases, too
  * discuss: more cases?
 
 * handlers must be aware of
 `/etc/default/linux-restricted-modules-common` without hardcoding this too deeply, since it is highly Ubuntu specific

 * framework provides modalias reading/overriding and other common
 hardware detection functions, but individual handlers need to
 extend/override those (some handlers do not have a kernel module
 attached)

 * Needs a sensible CLI for integration into desktop effects
 configuration, etc.

 * Drop the `runme_as_root()` madness, since it is inappropriate for other distros, makes assumptions about the authentication backend, and is only used for a corner case (enabling nvidia driver in compiz control panel, but this
could run r-m through gksu itself if it wants to).

== Proposed new design ==

attachment:restricted-manager.png

(Dia source: attachment:restricted-manager.dia)

Please check the status of this specification in Launchpad before editing it. If it is Approved, contact the Assignee or another knowledgeable person before making changes.

Summary

We propose a new code and driver database design for restricted-manager which will make it maintainable again and also be suitable as a proper upstream project, so that other distributions can use it as well.

We review the list of existing and proposed features and create a new code architecture which will be rewritten from scratch (with reusing most of the existing code modules, of course).

Release Note

The behaviour of restricted-manager should not change in a visible way, so this should not be mentioned in the release notes.

Rationale

The original code architecture of restricted-manager was very narrow, focusing on non-free kernel drivers in a Gnome application. A lot of new features have been bolted on this original structure, such as the KDE frontend, grouping of handlers, support for firmware installation, or non-free support packages of drivers which are free by themselves. This made the code very interdependent and hard to maintain/bugfix/understand.

The current version also has a lot of Ubuntu specifics hardcoded, which renders it hard to use for other distributions without significant changes.

Use Cases

  • Jon is a RedHat developer. He decides that the next Fedora release should get an easy to use frontend for installing restricted drivers. He creates a branch of restricted-manager, implements the small distro specific "OSLib" class, and has it running. Since this was so easy, he has some time to fix some bugs, which he contributes back to upstream.

  • Richard does not want to provide an interface for restricted drivers in his distribution, since they are evil and from the dark side of the force. However, he needs a GUI for providing free third-party drivers which are hosted outside of his distro's repository and which are maintained separately. He calls restricted-manager in "free third-party drivers" mode. For this reason upstream renamed the project to something less focused on restricted drivers.
  • Martin writes a handler for installing the Broadcom firmware on demand. Three months after releasing Hardy the download URL changes. He just sets a new URL in the central database and users can continue to get the firmware with a snap of a finger without the need to provide an SRU.

Design

  • Move away from being Ubuntu specific to a proper upstream project.
  • Handlers are DE agnostic, but they sometimes need to ask questions → need to call abstract UI message boxes etc.
  • Application logic and UI workflow is encapsulated in an abstract UI class. This gets subclassed to implement KDE/GNOME/CLI/a noninteractive test suite implementation.
  • Basic handler must not make assumptions about kernel modules, etc., since there are many device drivers which are not tied to a kernel module (e. g. sl-modem-daemon, printer drivers, file system drivers, etc.).
  • As a consequence of the previous point, each driver needs to be able to decide on its own whether it is applicable on the current system. The application framework should provide common functionality like modalias reading/overriding, lsmod/lspci evaluation, etc. (i. e. provide a list of available and enabled hardware), kernel module blacklisting, etc.
  • Supports different handler types:
    • kernel module (vmware/OSS)
    • kernel module + additional magic (nvidia/ati, sl-modem-daemon)
    • kernel module + shipped firmware (ipw3945)
    • kernel module + non-shipped firmware (bcm43xx)
    • grouping (all three vmware modules, or all two dozen OSS drivers)
    • printer driver
    • Fuse implementation to support a new file system type.
  • A handler knows whether it has a free or restricted driver.
  • The application can be started in any of three modes:
    • Show non-free drivers only (as in Ubuntu, since we do not want to promote providing free drivers without getting them into the main kernel)
    • Show free drivers only (Richard's use case from above, there is an actual real-life request for this feature)
    • All available drivers (for vendors that want it).
    The user interface and strings change accordingly.
  • Encapsulate all distro specific code in an abstract OSLib class and provide distro specific implementations in concrete subclasses. This should be the only class downstreams have to override and implement (aside from providing concrete custom handlers). As an example, the Ubuntu implementation will be aware of handling /etc/default/linux-restricted-modules-common.

  • Drop the runme_as_root() madness, since it is inappropriate for other distros, makes assumptions about the authentication backend, and is only used for a corner case (enabling nvidia driver in compiz control panel, but this could run r-m through gksu itself if it wants to).

  • The handler → driver mapping should not be hardcoded any more. The mapping is maintained on a central location (initially on

    http://www.drivertool.org) where all interested distros can maintain their own mapping to a distro package, URL, etc., and where distros can collaborate to maintain this database.

  • For this phase, distros will ship all the available handlers by default, so that they can detect availability. In the future, it is desirable to move the handlers themselves to a central place, so that third-party vendors can provide their own. We currently do not provide a solution how to get them from that central place to the distribution. However, the new program design should not assume any tight coupling between the core code and handlers. (Note: this is already true for the current design, any package can ship a handler, at which point it becomes available in r-m.)

Implementation

Handler-Driver mapping

http://www.drivertool.org is a distribution neutral place which can be used for hosting the central mapping. This site is currently hosted privately by Jon Masters, but if traffic becomes an issue, it would need some contributions by Canonical, RedHat, and other Linux vendors which participate. However, the amount of data that gets exchanged is very small.

Any computer can do a query on this database. As an input, it provides a type/value pair, where type is 'modalias', 'PCI vendor/product ID', 'Printer manufacturer and model', etc., and the value is the respective value for that type. This ensures that

  • the system is extensible to accomodate any kind of hardware identification
  • the amount of data sent by the client is minimal (in particular, it does *not* send the OS vendor, version, any serial numbers, etc. for privacy reasons).

The response is a list of available entries, where each entry has the following fields:

  • OS Vendor
  • OS version
  • set of type/value pairs

The type/value pairs can be interpreted freely for each OS vendor (distribution), but there should be some standard types like 'url', 'packagename', 'md5sum', 'sha1sum', etc.. For Ubuntu, we additionally define the data type 'repository', which would be an URL to an apt repository.

With this design, each client always gets the full list of entries for all OS vendors. This compromise is done to avoid exposing the OS vendor and version, and other information which could be used for tracking, collecting data for security attacks (like providing an IP list of known-vulnerable OS versions); the lists are not expected to become very big, too (in the order of 50 entries, which should amount to some 5 KB).

Example 1: bcm43xx firmware

Client → DB:

DesktopTeam/Specs/RestrictedManagerRewrite (last edited 2008-08-06 17:01:12 by localhost)