ArchiveReorganisation

Revision 42 as of 2009-03-09 13:10:59

Clear message

THIS IS AN UNAPPROVED DRAFT. EXPECT SUBSTANTIAL CHANGES.

Problem statement

Current archive layout

Since its inception, the Ubuntu archive has been split into four components, organised as follows:

Component

Licensing

Supportedness

Uploader team

Permitted dependencies

main

Free software

Supported

ubuntu-core-dev

main

restricted

Non-free software (drivers)

Supported

ubuntu-core-dev

main, restricted

universe

Free software

Unsupported

ubuntu-dev

main, universe

multiverse

Non-free software

Unsupported

ubuntu-dev

main, restricted, universe, multiverse

This scheme was created before Ubuntu was announced publicly, and before the MOTU team was established to take care of the universe and multiverse components. It also predates the creation of Ubuntu derivatives hosted in the Ubuntu archive, such as Xubuntu and Ubuntu Studio. Of late it has been creaking at the seams. Problems have included:

  • It has been unclear whether packages required for CD images of Ubuntu derivatives should go in main. Our documentation is at best unclear on this.

  • The MOTU team is responsible for an extremely wide range of tasks, and is fragmented by needing to cover Ubuntu derivatives as well as general packages untended in other ways.
  • Developers are under pressure to join ubuntu-core-dev in order to upload to relatively small subsets of the archive. For example, developers interested in the use of Ubuntu in education need to join ubuntu-core-dev in order to upload packages on the Ubuntu Education Edition CD images.

  • It is often not clear exactly what Canonical supports. Originally this was simply "everything in main", but a distinction is now required between "maintained for security updates" and "commercially supported by Canonical".

  • Users have very little guidance on what "supported" means, and the available user interfaces only indicate whether a package is maintained for security updates, not whether it is commercially supported (by Canonical or otherwise) or whether it is well-tested with the flavour of Ubuntu they are using (for example, unless they know otherwise, Kubuntu users will likely want to default to using KDE applications rather than GNOME applications).
  • When a package is moved from universe to main, many of the people who previously tended to it lose the ability to upload it.

Much of the metadata needed to solve these problems already exists, but is not exposed to users in any useful form; instead, everything of concern to "supported" derivatives is conflated into "main", losing the ability to express very much more than a binary state.

Tracking of packages

The contents of Ubuntu installation images, together with the basic data used to divide the archive into supported and unsupported components (henceforth simply main and universe for brevity; please read restricted and multiverse as well where appropriate), are tracked using seeds. These are formed as lists of package names that are "interesting" in particular categories; for example, an Ubuntu desktop needs to have the gnome-panel package installed. Dependencies and build-dependencies need not be explicitly listed in seeds, because germinate will expand them as necessary.

Conceptually, therefore, the contents of main are defined by the union of the output of germinate for all supported flavours (i.e. Ubuntu, Kubuntu, etc.). However, in practice it is desirable to have an additional review step before allowing packages into main, as described in the MainInclusionProcess: briefly, the submitter writes a main inclusion report (MIR), which is checked by a small team of reviewers; if passed, archive administrators move the package to main. Since this must, by policy, be followed for all new source packages entering main (with occasional exceptions for "trivial" changes, such as source packages being renamed for new versions of a library), it is thus not generally possible for an uploader to introduce unsuitable dependencies into main either unintentionally or maliciously. This system may briefly be characterised as human moderation of access control changes.

Problems with access control

The ubuntu-core-dev and motu/ubuntu-dev teams were originally intended as rungs on a ladder. Contributors would start out by submitting patches until they were deemed capable of uploading a limited set of packages without prior review; once they had accumulated enough experience there, they would proceed to ubuntu-core-dev where they would be able to upload to the entire archive.

Over time, this has proven to be inadequate in various specific ways, though basically sound in general. For example, in order to be able to upload the kernel, kernel developers must spend time developing extensive packaging experience in order to gain ubuntu-core-dev (and possibly MOTU beforehand), despite the fact that their interests or even jobs are not particularly aligned with this. The Technical Board has on occasion worked around this by ad-hoc exceptions on the condition that people ask for review before uploading packages outside their areas of interest, but this is rather opaque and requires a certain degree of persuasion in each individual case. Such restrictions cannot be enforced. They are only effective based on the restraint of the developer. Similarly, Kubuntu developers must generally demonstrate a general ability to understand the Ubuntu system beyond their own flavour in order to gain ubuntu-core-dev, even though without that they could easily be trusted to do sensible things with packages only on the Kubuntu CD images.

Nothing in the following proposal is intended to diminish the rĂ´le of ubuntu-core-dev. That team should still consist of highly experienced and competent developers with a wide-ranging understanding of the Ubuntu system. They will often be called upon to arbitrate among conflicting requirements from different Ubuntu flavours, or to make changes with an understanding of their effect on a variety of different subsystems. However, this is a team where quality rather than quantity is important, and it makes sense both to reduce the pressure on people to join before they are ready in order to get their work done, and to reduce the demands on ubuntu-core-dev members for sponsorship.

Proposal

This proposal is divided into three parts. The first (and simplest) deals with exceptions useful in specific cases; the second and third deal with wider issues. The three parts complement each other, and are not intended to be contradictory.

Additional uploaders

As of Launchpad 1.2.5 (though there is no UI yet), it is possible to define additional uploaders for an individual package. For small numbers of packages, this process may well be suitable. For example, it is likely to make sense for experienced kernel team members to be able to upload the kernel regardless of whether they have substantial enough packaging experience to gain ubuntu-core-dev privileges. Similarly, when a single package that has been well-cared-for in universe moves to main (or has its uploader team changed, as in the second part of this proposal) due to the good support it has been receiving from its de facto maintainer, it is usually desirable for that maintainer to be able to continue their work. Additionally, if a single package in universe is primarily cared for by a developer not yet in ubuntu-dev, that individual may be granted direct upload access.

Prospective uploaders would apply to the Technical Board for permission, with such permission generally being granted based on experience with the package in question more than on experience with the rest of the Ubuntu system. Naturally, some level of familiarity with and respect for Ubuntu processes (such as freeze procedures) should be required. As usual, the Technical Board may delegate this as they think appropriate.

As of late 2008, this process is implemented. The Technical Board uses edit_acl.py to manipulate lists of additional uploaders, and anyone with access to the Launchpad API can use this to query additional uploaders.

Upload permission changes

This part of the proposal has been split out to /Permissions.

Component reorganisation

This part of the proposal has been split out to /Components.