CheckboxConvertRegistryIntoDBusBackend

Differences between revisions 1 and 5 (spanning 4 versions)
Revision 1 as of 2009-05-17 04:28:51
Size: 2625
Editor: 208-151-246-43
Comment:
Revision 5 as of 2009-06-01 22:13:07
Size: 7277
Editor: modemcable178
Comment: Removed notes taken during the session copied while drafting this spec.
Deletions are marked like this. Additions are marked like this.
Line 10: Line 10:
Technical details of extracting the registry into it's own DBus backend, along with identify other projects who may be interested in using this. The registry in Checkbox should be expressed as a DBus backend.
Line 14: Line 14:
This section should include a paragraph describing the end-user impact of this change. It is meant to be included in the release notes of the first release in which it is implemented. (Not all of these will actually be included in the release notes, at the release manager's discretion; but writing them is a useful exercise.)

It is mandatory.
Applications needing consistent access to system information should use a
central location. Ideally, this location should be exposed in a language
agnostic way so that any application can access the same information.
Line 20: Line 20:
This should cover the _why_: why is this change being proposed, what justifies it, where we see this justified. During the Jaunty cycle, Checkbox was extended so that parts of the
registry needing root privileges to gather information could be run by
the system DBus. This experience has shown that it would be worthwhile
to expose the whole registry through DBus so that it could be loaded
independently of the application.

Furthermore, the registry was originally designed with the singleton
pattern in mind. This was successfully implemented at the application
layer but, with the system DBus, it would be possible to implement this
pattern at the system layer which would be even more appropriate.
Line 24: Line 33:
 * Test scripts could use the same registry as Checkbox instead of having
   to re-parse the output of various commands such as lshal, lshw, etc.
 * Test scripts could make use of a command line tool in order to query
   the registry instead of having to access the DBus backend directly.
 * Checkbox could run more quickly by deferring processing of system
   information to the DBus backend asynchronously.
 * Users could query the registry themselves, either through a DBus browser
   or the command line to gather information about their system consistenly.
 * Contributers could write better test definitions by running queries
   against the registry from the command line for the requires field.
 * Apport would like to make use of the information gathered in a consistent
   way by the registry, such as the system id for example.
Line 25: Line 47:

As of writing this specification, the ubuntu-server team doesn't seem to
have any compelling reason to run DBus by default. Therefore, if we would
like Checkbox installed by default at least during the development cycle,
it would be preferable not to enforce DBus as a dependency. Instead,
it should be possible to implement the backend as a library which could
be imported directly when the application is run as root.

Also, the information gathered by the DBus backend should be sensitive
to the user accessing it. For example, it should only expose gconf
information for the user rather than for all users.
Line 28: Line 61:
You can have subsections that better describe specific parts of the issue. The design of moving the registry from the core to a DBus backend will
affect both the core and the plugins.

As far as the core is concerned, the concept of the registry should be
exposed as a singleton accessed by a global handler. This handler will
in turn abstract whether the registry is being accessed as a library
when run as root or through DBus when run as a normal user.

As for the plugins, they currently access the registry as an attribute
of the plugin manager. Instead, this attribute should be pulled out
and the registry should then be accessed using the handler described
above.

Since the registry would run under the system DBus, it would need to
expose a query interface which does not consist of eval'ing a Python
expression. Instead, the registry should be stored in an SQL table
and expose an SQL query interface.
Line 32: Line 81:
This section should describe a plan of action (the "how") to implement the changes discussed. Could include subsections like: Part of the implementation has already been completed during the Jaunty
cycle by exposing a coarse grained interface to some registries through
the DBus backend. This interface exposed a very simple interface returning
the string representation of registries which deferred parsing to the
client application. Instead, the interface should be refined so that the
backend could become responsible for the parsing. The implication is that
the interface would then have to become much finer grained and define
types for all the information exposed.

Furthermore, the current implementation already supports the concept
of accessing the registry directly by the application when run as
root. So, this abstraction might actually be reusable or might need some
refinement. This would require further exploration or might be revealed
during implementation.

As for the query interface, it would be preferable to use a lightweight
SQL database such as SQLite. In order to support recursive searching,
the database should be populated as if it were recursive. For example,
the registry key:

 * foo.bar.baz.blah

Should be populated as:

 * foo.bar.baz.blah
 * bar.baz.blah
 * baz.blah

This would make it possible to search for:

 * baz.blah = 'meh'
Line 36: Line 115:
Should cover changes required to the UI, or specific UI that is required to implement this As far as the current command line and graphical interfaces are concerned,
there should be no noticeable changes from moving the registry behind the
DBus backend.

As far as the API is concerned, the plugins should become slightly simpler
to implement. The reason is mostly that the contributers will need to know
less about private attributes inherited from the Plugin class.

Last, it would be convenient to provide another command line application
in order to query the registry. Instead of solely relying on the test
definitions to query the registry, it would be quite useful to be able
to perform queries like this:

 * checkbox_registry 'package.name = "apache2"'

This should return all the information known by the registry about the
package named "apache2" assuming it is installed.
Line 40: Line 135:
Code changes should include an overview of what needs to change, and in some cases even the specific details. Interestingly, the existing regitries themselves should not need any changes
at all. Instead, the code changes should strictly consist of wrapping the
registries behind an abstraction layer which could either be called directly
by the application or through the DBus backend. So, most of the implementation
will consist of extending the current coarse grained interface to a much finer
grained interface.

As far as the plugins are concerned, it would actually be possible to make the
changes backward compatible with the current implementation. However, it would
be preferable to change the plugins in order to access the registry singleton
instead of using private attributes from the plugin manager.
Line 44: Line 149:
Include:
 * data migration, if any
 * redirects from old URLs to new ones, if any
 * how users will be pointed to the new way of doing things, if necessary.
Optionally, it would be desirable for the plugins to be migrated in
order to use the registry singleton. As far as the current registries
are concerned, as mentionned above, they should not need any changes.
Line 51: Line 155:
It's important that we are able to test new features, and demonstrate them to users. Use this section to describe a short plan that anybody can follow that demonstrates the feature is working. This can then be used during testing, and to show off after release. Please add an entry to http://testcases.qa.ubuntu.com/Coverage/NewFeatures for tracking test coverage.

This need not be added or completed until the specification is nearing beta.
 1. Run the command: checkbox_registry 'package.name = "apache2"'
 2. This should return a package entry from the registry
Line 57: Line 160:
This should highlight any issues that should be addressed in further specifications, and not problems with the specification itself; since any specification with problems cannot be approved. It might be worthwhile to explore existing DBus browsers which could
expose the information provided by the registry in a convenient way.
Line 59: Line 163:
== BoF agenda and discussion ==

Use this section to take notes during the BoF; if you keep it in the approved spec, use it for summarising what was discussed and note any options that were rejected.
Depending on how long the backend is expected to keep running, it might
become necessary to start listening for events such as new packages
installed, new hardware plugged, etc.

Summary

The registry in Checkbox should be expressed as a DBus backend.

Release Note

Applications needing consistent access to system information should use a central location. Ideally, this location should be exposed in a language agnostic way so that any application can access the same information.

Rationale

During the Jaunty cycle, Checkbox was extended so that parts of the registry needing root privileges to gather information could be run by the system DBus. This experience has shown that it would be worthwhile to expose the whole registry through DBus so that it could be loaded independently of the application.

Furthermore, the registry was originally designed with the singleton pattern in mind. This was successfully implemented at the application layer but, with the system DBus, it would be possible to implement this pattern at the system layer which would be even more appropriate.

User stories

  • Test scripts could use the same registry as Checkbox instead of having
    • to re-parse the output of various commands such as lshal, lshw, etc.
  • Test scripts could make use of a command line tool in order to query
    • the registry instead of having to access the DBus backend directly.
  • Checkbox could run more quickly by deferring processing of system
    • information to the DBus backend asynchronously.
  • Users could query the registry themselves, either through a DBus browser
    • or the command line to gather information about their system consistenly.
  • Contributers could write better test definitions by running queries
    • against the registry from the command line for the requires field.
  • Apport would like to make use of the information gathered in a consistent
    • way by the registry, such as the system id for example.

Assumptions

As of writing this specification, the ubuntu-server team doesn't seem to have any compelling reason to run DBus by default. Therefore, if we would like Checkbox installed by default at least during the development cycle, it would be preferable not to enforce DBus as a dependency. Instead, it should be possible to implement the backend as a library which could be imported directly when the application is run as root.

Also, the information gathered by the DBus backend should be sensitive to the user accessing it. For example, it should only expose gconf information for the user rather than for all users.

Design

The design of moving the registry from the core to a DBus backend will affect both the core and the plugins.

As far as the core is concerned, the concept of the registry should be exposed as a singleton accessed by a global handler. This handler will in turn abstract whether the registry is being accessed as a library when run as root or through DBus when run as a normal user.

As for the plugins, they currently access the registry as an attribute of the plugin manager. Instead, this attribute should be pulled out and the registry should then be accessed using the handler described above.

Since the registry would run under the system DBus, it would need to expose a query interface which does not consist of eval'ing a Python expression. Instead, the registry should be stored in an SQL table and expose an SQL query interface.

Implementation

Part of the implementation has already been completed during the Jaunty cycle by exposing a coarse grained interface to some registries through the DBus backend. This interface exposed a very simple interface returning the string representation of registries which deferred parsing to the client application. Instead, the interface should be refined so that the backend could become responsible for the parsing. The implication is that the interface would then have to become much finer grained and define types for all the information exposed.

Furthermore, the current implementation already supports the concept of accessing the registry directly by the application when run as root. So, this abstraction might actually be reusable or might need some refinement. This would require further exploration or might be revealed during implementation.

As for the query interface, it would be preferable to use a lightweight SQL database such as SQLite. In order to support recursive searching, the database should be populated as if it were recursive. For example, the registry key:

  • foo.bar.baz.blah

Should be populated as:

  • foo.bar.baz.blah
  • bar.baz.blah
  • baz.blah

This would make it possible to search for:

  • baz.blah = 'meh'

UI Changes

As far as the current command line and graphical interfaces are concerned, there should be no noticeable changes from moving the registry behind the DBus backend.

As far as the API is concerned, the plugins should become slightly simpler to implement. The reason is mostly that the contributers will need to know less about private attributes inherited from the Plugin class.

Last, it would be convenient to provide another command line application in order to query the registry. Instead of solely relying on the test definitions to query the registry, it would be quite useful to be able to perform queries like this:

  • checkbox_registry 'package.name = "apache2"'

This should return all the information known by the registry about the package named "apache2" assuming it is installed.

Code Changes

Interestingly, the existing regitries themselves should not need any changes at all. Instead, the code changes should strictly consist of wrapping the registries behind an abstraction layer which could either be called directly by the application or through the DBus backend. So, most of the implementation will consist of extending the current coarse grained interface to a much finer grained interface.

As far as the plugins are concerned, it would actually be possible to make the changes backward compatible with the current implementation. However, it would be preferable to change the plugins in order to access the registry singleton instead of using private attributes from the plugin manager.

Migration

Optionally, it would be desirable for the plugins to be migrated in order to use the registry singleton. As far as the current registries are concerned, as mentionned above, they should not need any changes.

Test/Demo Plan

  1. Run the command: checkbox_registry 'package.name = "apache2"'
  2. This should return a package entry from the registry

Unresolved issues

It might be worthwhile to explore existing DBus browsers which could expose the information provided by the registry in a convenient way.

Depending on how long the backend is expected to keep running, it might become necessary to start listening for events such as new packages installed, new hardware plugged, etc.


CategorySpec

QATeam/Specs/CheckboxConvertRegistryIntoDBusBackend (last edited 2009-06-01 22:15:35 by modemcable178)