UbuntuDevelopment

Revision 122 as of 2007-10-18 08:08:32

Clear message

Anchor(Overview)

Overview of Development

Ubuntu is developed by a team of UbuntuDevelopers. There are two types of UbuntuDevelopers: [http://launchpad.net/~ubuntu-core-dev core developers] and [http://launchpad.net/~motu MOTU]. This process is transparent to the public, and open to any contributor who demonstrates the necessary skills and commitment to the project.

Ubuntu is based on [http://www.debian.org/ Debian], sharing many of its packages, tools and techniques with that project. Differences between Ubuntu and Debian are described in UbuntuForDebianDevelopers.

Ubuntu is [#ReleaseProcess periodically released] according to a set schedule.

Like most operating systems, Ubuntu is complex, and it can help to get a broad overview of its architecture first. For that, see UbuntuArchitecture.

If you have been directed to this page for advice on contributing to Ubuntu as a developer, you may also be interested in ContributeToUbuntu.

Anchor(OtherDevelopers)

Working with Other Developers

You are not alone! Ubuntu is the work of many developers, and we devote some effort to enabling efficient collaboration with tools, infrastructure, government and a cooperative spirit.

Anchor(StartingPoints)

Starting points

UbuntuDevelopers explains the role of developers in the Ubuntu project and how to join the team.

The [:MOTU] team, in addition to their development activities, provide information and guidance for new and prospective Ubuntu developers. If you're newly interested in Ubuntu development and looking for answers, introduce yourself and listen in!

If you're looking for tasks which need doing, many of those are tracked in the bug tracking system. The BugSquad maintains several lists of them at ["Bugs/Tags"].

If you already have experience working with Debian packages, most of your knowledge applies equally well to Ubuntu packaging. If you are a Debian developer, UbuntuForDebianDevelopers summarizes some of the differences between the projects, and later sections in this document provide details of our infrastructure.

To submit patches for review or to help reviewing patches, please refer to [:/CodeReviews:the Code Review process].

Anchor(Communication)

Communication

Email discussion among Ubuntu developers takes place on the [http://lists.ubuntu.com/mailman/listinfo/ubuntu-devel ubuntu-devel mailing list], which is moderated (excepting registered Ubuntu developers). The [http://lists.ubuntu.com/mailman/listinfo/ubuntu-devel-discuss ubuntu-devel-discuss mailing list] is available for open discussion about Ubuntu development. All UbuntuDevelopers should subscribe to the [http://lists.ubuntu.com/mailman/listinfo/ubuntu-devel-announce ubuntu-devel-announce mailing list], where important development events are announced. [https://lists.ubuntu.com/mailman/listinfo/ Various other mailing lists] are available, some of which focus on specific areas of development.

The #ubuntu-devel channel on the FreeNode IRC network is home to many Ubuntu developers for real-time communication.

[#Notification Automated notifications of development activity] are also useful for keeping up with what other developers are working on.

A comprehensive matrix of communication channels can be found on DeveloperCommunication.

Anchor(Bugs)

Bugs and the BugSquad

[http://bugs.launchpad.net/ubuntu Ubuntu bug reports] are tracked in Launchpad. HelpingWithBugs contains information about how they are handled. The BugSquad documentation describes how to cooperate with other developers and volunteers working on bug triage; it is required reading for new developers, as developers will typically need to spend a significant amount of time working with the bug tracking system.

The BugSquad (and the ["UbuntuQA"] team, which is comprised of more experienced triagers and is responsible for prioritising bugs) is here to help you as a developer. If you are responsible for a non-trivial number of bugs, it is a good idea to spend some time on helping them help you. A useful starting point is to add specific information about your packages to DebuggingProcedures: this may include both special tricks for debugging them effectively and any particular policy you have on how you want your bugs to be handled (e.g. assignment, tags, etc.). When adding significant chunks of new information to DebuggingProcedures, please send a note to ubuntu-bugsquad@lists.ubuntu.com about it.

Members of the bugsquad can be found on the #ubuntu-bugs channel on the FreeNode IRC network. There is also the [https://lists.ubuntu.com/mailman/listinfo/ubuntu-bugsquad ubuntu-bugsquad mailing list] for general discussion regarding bugs and bug triaging.

Anchor(RCBugs)

Release Critical bugs

Anchor(ReleaseProcess)

The Release Process

Each release cycle follows the same general pattern, with the following major phases. UbuntuDevelopers are expected to track this process closely, and ensure that their work is aligned with that of others. TimeBasedReleases mean that we need to be well coordinated in our efforts to produce an on-time release.

Beginning a new Release

At the beginning of each cycle, the Ubuntu infrastructure is prepared for a new branch of development. The package build system is set up, the toolchain is organized, seeds are branched, and many other pieces are made ready before development can properly begin. Once these preparations are made, the new branch is officially announced and opened for package uploads.

Anchor(Planning)

Planning

The features to be targeted for each release cycle are organised using [:SpecSpec:specifications]. Some of these come from strategic priorities for the distribution as a whole, some are proposed by individual developers, and some are drawn from the IdeaPool or the [https://blueprints.launchpad.net/ubuntu list of existing specifications (long)]. The proposed features are discussed at the DeveloperSummit at the beginning of each release cycle, and soon after, the TechnicalBoard and its advisors review proposed features and select a set of development projects which fit together well into the overall plan for the release.

The planned feature list for the current release cycle (Ubuntu 7.10, Gutsy) can be found at http://blueprints.launchpad.net/ubuntu/gutsy/+specs, and the schedule at GutsyReleaseSchedule.

Anchor(MergeProcess)

Merging with Upstream

The first phase of the release cycle is characterized by bringing new releases of upstream components into Ubuntu, either directly or via Debian. We merge from Debian because it's the single most effective way to keep up to date with upstream code (Debian maintainers package new upstream releases on a frequent basis, often faster than we are able to do so), and because Debian and Ubuntu are similar in many ways so their bug-fixes are often bug-fixes for us too. We do it every release cycle rather than taking the occasional cycle off because if we didn't it would be significantly harder ever to come back into sync.

Anything that we don't modify - strictly, anything whose version number does not contain the substring "ubuntu", and which isn't in the [http://people.ubuntu.com/~ubuntu-archive/sync-blacklist.txt manually-maintained blacklist] - is "synced" from Debian semi-automatically: the source package is copied verbatim and rebuilt on our buildds. Anything that we've modified either needs to be merged by a developer, or needs a developer to request that the package be synced overriding the Ubuntu changes in the event that Debian took all our changes or they no longer apply for some other reason. Managing to get a package back into sync is usually a good thing, as it saves us from having to put maintenance effort into that package.

For more information on the practicalities of this process, see the [#SyncingAndMerging Syncing and Merging] section below.

This phase can be said to end when all packages have been brought up to date at least once, and the result has been sufficiently stabilized to produce the first milestone, with installable CD images. This must be no later than the DebianImportFreeze for the release cycle.

Feature Development

During this phase, the focus is on development projects which have been planned for the release. These projects often begin while merging is still underway, and accelerate the pace of their development once the package archive is reasonably consistent and usable.

Anchor(Freezes)

Stabilization (Freeze)

During the course of development, we gradually exercise greater caution in making changes to Ubuntu, in order to ensure that we reach a stable state in time for the final release date. The typical order and length of the various freezes can be seen on the current ReleaseSchedule and the ReleaseScheduleTemplate. During freezes, uploads are sometimes held in a queue for manual approval. A [http://people.ubuntu.com/~ubuntu-archive/queue/ mirror] of this queue is available to ease coordination during these periods.

Exceptions to the current freeze criteria can be requested using the FreezeExceptionProcess.

  • Open Development - unrestricted general development activity, new packages and versions automatically imported from Debian
  • DebianImportFreeze - unrestricted general development activity, new packages/versions from Debian only on request, all packages merged with Debian

  • FeatureFreeze - no new features, packages, or APIs/ABIs, only bug fixes

  • UserInterfaceFreeze - no user interface changes (stabilize for documentation and screenshots)

  • BetaFreeze - uploads by release manager approval only (stabilize for beta release validation)

  • DocumentationStringFreeze - no string changes in master documentation any more (stabilize for doc translations)

  • KernelFreeze - no new kernels (stabilize for final hardware compatibility checks, deadline for kernel regression fixes)

  • ReleaseCandidate - uploads by release manager approval only (stabilize for final release validation)

Anchor(Milestones)

Milestones

During the development phase of a release we regularly create and test CD images, which mark milestones in development and are used for more widespread testing of the current development release status:

  • The first Alpha milestone after about two months, when the initial flood of new upstream versions, merges with Debian, and first set of new features landed.
  • Three Alpha milestones in intervals of three weeks until feature freeze and upstream version freeze, to get early testing of new features.
  • Two more Alpha milestones in intervals of two weeks for stabilizing the new features and fixing the worst bugs.
  • The Beta milestone three to four weeks before the final release.

The Alphas are verified to check that the installation succeeds and at least give a working system. They often require some tweaks and workarounds afterwards to get it actually usable (like some applications not starting by default, or getting a lot of crash reports). Thus these are aimed at developers and interested technically savvy users from the community. This is particularly important to exercise the installer and getting early feedback about new features.

The Beta release is feature complete and free of major bugs (like the installer failing, installed applications do not work at all, or the desktop crashing consistently). They get extensive testing on a lot of hardware. It is recommended for a larger (but still technically inclined) audience. From then on, upgrades to the final release are supported and guaranteed to succeed.

All Alphas announced on the [https://lists.ubuntu.com/mailman/listinfo/ubuntu-devel-announce ubuntu-devel-announce mailing list]. Due to the more widespread focus of the Beta, its announcement goes to [https://lists.ubuntu.com/mailman/listinfo/ubuntu-announce ubuntu-announce]. The announcements contain general information about the focus and download addresses.

For each Alpha and Beta there is also a more in-depth article on the [http://www.ubuntu.com/testing/ Testing page] of the [http://www.ubuntu.com Ubuntu website], which describes new features, caveats, and other important information.

The process of creating a milestone and the communications involved from the Release Manager's point of view are explained in detail in MilestoneProcess. Developers need to be aware that the archive gets frozen a few days before a Milestone is scheduled, in order to stabilize the archive, avoid jeopardizing installability, and giving control to the release manager what goes into the Alpha milestone CDs.

Finalization

As the final release approaches, our focus narrows to fixing "showstopper" bugs and performing very thorough validation of the installation images. Every image is tested to ensure that the installation methods work as advertised. Low-impact bugs and other issues are deprioritized in order to focus developers on this effort.

This phase is important, as severe bugs which affect the experience of booting or installing the CD must be fixed prior to the final release. Ordinary bugs which affect the installed system, in contrast, can be fixed with StableReleaseUpdates.

Anchor(StableReleases)

Stable Releases

Released versions of Ubuntu are intended to be stable. This means that users should be able to rely on their behaviour remaining the same, and therefore, updates are only released under special circumstances. These criteria, and the procedure for preparing such an update, are described in StableReleaseUpdates and SecurityUpdateProcedures.

Anchor(Packaging)

Packaging

If you're interested in packaging work, but don't have much experience yet, you should get in touch with [:MOTU:the MOTU team], who provide mentoring for new developers. [:MOTU/Mentoring:Mentoring] is available too, via dedicated mentors and/or a [http://lists.ubuntu.com/mailman/listinfo/ubuntu-motu-mentors mailing list].

Anchor(DebianPackages)

Working with Debian-format Packages

Ubuntu uses the Debian packaging format. The following resources explain how to create and modify Debian-format packages.

Anchor(UbuntuPackages)

Working with Ubuntu Packages

  • Set the target suite in debian/changelog to be the code name of the current development branch, e.g. "dch -D gutsy"

  • When working with a package which originated in Debian, use a version number derived from the Debian version number with ubuntu<revision> appended. e.g. Debian 1.0-2 becomes 1.0-2ubuntu1, followed by 1.0-2ubuntu2, etc.

  • When creating a new package which may later be added to Debian, use a revision of the form -0ubuntu1
  • Remember to include the orig.tar.gz if this is a new upstream version of a non-native package but you have already patched it before upload. A missing original tarball may cause the upload to be rejected or silently dropped. Use dpkg-buildpackage -S -sa to generate such an upload. If the orig.tar.gz is already in the distribution then you don't need to upload it again.

  • Always be aware of the release schedule and any applicable [#Freezes freezes]. The cooperation of all developers is needed in order to ensure a successful release!
  • If your changes may affect the work of other developers, it is a good idea to discuss them on a mailing list first

Anchor(Bazaar)

Revision control (Bazaar)

Bazaar, an open source revision control system and Canonical sponsored project, is the preferred revision control system in Ubuntu. Many Ubuntu packages are [:BzrMaintainerHowto:maintained in Bazaar], which makes it easy for other developers to [:BzrContributorHowto:contribute changes to them], which can be easily merged by the maintainer.

Note that, as a practical matter, many packages are not yet maintained in Bazaar, but in other revision control systems or none on a case-by-case basis. Where no revision control system is used, the history of uploads recorded in [https://launchpad.net/ubuntu/+search Launchpad] may be useful.

Anchor(Derivatives)

Derivative Distributions

Several derivatives of Ubuntu are available (also sometimes called "flavours"). A number of people in the Ubuntu community work on one or more of these derivative versions. Kubuntu, Edubuntu, and Xubuntu are all maintained directly in the Ubuntu archive.

Anchor(Building)

Building

You should always build and test packages locally before submitting them to Ubuntu. Failure to do so will waste the time of other members of the community, so please be considerate.

Anchor(Tools)

Tools

You will find some tools explained in the [:MOTU/Recipes:MOTU Recipes], also from gutsy on, you will find ubuntu-dev-tools in the archive, which contains [:UbuntuDevTools:tools for developing Ubuntu].

Anchor(NewPackages)

New Packages

Criteria

In order for a piece of software to be included in Ubuntu, it must meet the [http://www.ubuntu.com/community/ubuntustory/licensing Ubuntu License Policy].

Requesting a new package for Ubuntu

To get a package into Ubuntu, please [https://launchpad.net/ubuntu/+filebug?field.tags=needs-packaging file a bug in Launchpad] and make sure it has the tag [https://lists.ubuntu.com/archives/ubuntu-motu/2007-March/001471.html needs-packaging]. Please mention where to get the source for it and which license it is under. Make sure you check which [https://launchpad.net/ubuntu/+bugs?field.tag=needs-packaging packages have already been requested]. For packages in Debian, but not in ubuntu [https://launchpad.net/ubuntu/+filebug file a bug] with the summary field "please sync package <packagename> from debian <distro>" where packagename is the package you would like to see.

Packaging it yourself

Packages which are not in Ubuntu yet, require extra scrutiny and go through a special review process, before they get uploaded and get a final review by the [http://launchpad.net/~ubuntu-archive archive admins].

The ["MOTU"] team uses the following workflow:

  • New MOTU contributors (who are not member of the motu team yet), need to get their packages reviewed and signed off by at least two MOTUs or maintainers to get them uploaded to Ubuntu.

  • MOTUs are encouraged to get their package reviewed, but need no ACK. (cf. ["MOTU/Council/Meetings/2007-02-23"])

A package NEW to Ubuntu should be actively maintained upstream and receive security- and bugfixes regulary. If this can't be fulfilled, the package maintainer is 100% responsible for the above.

Maintainers are encouraged to check if work is already being done by Debian maintainers at (http://www.debian.org/devel/wnpp/requested and http://www.debian.org/devel/wnpp/being_packaged). Following up on existing bug reports with a link to your source package or an informal announce of the package to utnubu-discuss@lists.alioth.debian.org is appreciated.

Also of interest:

  • [http://mentors.debian.net/ mentors.debian.net], a website where people interested in getting their packages inside Debian can upload their packages. You need to [http://mentors.debian.net/debian/pool/ browse the directories] to find packages.

  • [http://svn.debian.org/wsvn Debian's WebSVN] It's possible that a package has been worked on for Debian but has a status of UNRELEASED. Check the appropriate directories that begin with "pkg" that your package may fall under. For example, game packages would be under "pkg-games".

  • Getting it reviewed: ["REVU"]

Anchor(Archive)

The Package Archive

All current official Ubuntu packages are stored in the master archive, which is widely [:Mirrors:mirrored]. A search interface is available at [http://packages.ubuntu.com]. Old versions can be retrieved from [http://launchpad.net/ubuntu Launchpad].

It is administered by the [http://launchpad.net/~ubuntu-archive archive administration team].

Anchor(Uploading)

Uploading

If you are not yet an official Ubuntu developer, you can arrange for your package to be uploaded via the SponsorshipProcess.

Packages are uploaded via FTP to ftp://upload.ubuntu.com/ using dput or dupload.

Notes for preparing your upload:

  • Make source-only uploads, i.e. use "dpkg-buildpackage -S"

  • When uploading to [:REVU], please include the orig tarball as well (use parameters -S -sa)

When your upload is processed (typically within a matter of minutes), you will receive an email with the result of your upload, whether it succeeds or fails, unless you use an unregistered email address. The system will only send mail to an address which belongs to a launchpad account which is a member of the relevant team for uploading. E.g. [http://launchpad.net/people/ubuntu-dev ubuntu-dev] for universe and [http://launchpad.net/people/ubuntu-core-dev ubuntu-core-dev] for main.

Your upload must be signed by GPG key registered in launchpad. If the signature cannot be traced to a member of the appropriate team, then the upload will be silently rejected.

To add a new package to Ubuntu, simply upload it as usual. Any new packages uploaded are put in a [https://launchpad.net/ubuntu/gutsy/+queue queue] to be checked by the administrators before being included.

Anchor(Publishing)

Autobuilding and Publishing

Once an upload has been accepted, it takes some time to be [#Autobuilders built] and published in the archive. For simple packages, this is usually on the order of an hour, but varies depending on release activity (uploads may be temporarily suspended), the time needed to build the package (including other packages in the build queue), and other factors.

Anchor(Notification)

Notification of changes

Notifications of uploads are sent to a mailing list. A different list is used for each Ubuntu release:

RSS Feeds of these messages are available at [http://media.ubuntu-nl.org/rss/].

Changelogs for all packages are available at http://changelogs.ubuntu.com/changelogs/ (this is the source used by update-manager and Synaptic).

Anchor(SyncingAndMerging)

Syncing and Merging

(See the [#MergeProcess rationale] above.)

Most packages in Ubuntu originate elsewhere, including Debian and related package repositories.

A sync copies a source package verbatim from an external repository into Ubuntu, overwriting any package of the same name. This is used when a newer version of it is available, and should be included in Ubuntu, and happens automatically during some phases of the release cycle. To request a sync, follow the SyncRequestProcess.

A merge is a three-way merge of a package which originated in an external repository. This is used when there is a newer version available from the external repository, but the package has also been modified (branched) in Ubuntu. [http://merges.ubuntu.com/ Merge-o-Matic] assists with this work, and ["MOTU/Merging"] explains how and when to merge. Packages which are [#Bazaar maintained in Bazaar] can and should be merged using Bazaar itself.

The last changelog entry of a merged package should contain the list of Ubuntu changes remaining in the package.

xxxxx (vvv-1ubuntu1) edgy; urgency=low

  * Merged with Debian. Changes are:
    + Modified build-dep on libfoo to build on edgy.
    + Added .desktop file.

The "Last Uploader" column in the Merge-o-Matic output is the default assignee for the merge, following the touched-it-last maintenance principle. However, you can and often should grab other people's merges if they don't have time or you feel you can do a better job. It's polite and often a good idea (though not mandatory) to contact the other person first to make sure you aren't duplicating work.

Backports work similarly to syncs, but have somewhat different requirements. To request a backport, follow the BackportRequestProcess.

Anchor(Consistency)

Consistency

The archive is periodically checked for various inconsistencies, such as incorrect dependency relationships between packages.

A special case is the installer:

Anchor(Components)

Managing Components

Ubuntu packages are classified into components according to [http://www.ubuntu.com/ubuntu/components maintenance and licensing criteria], a process which is described in SeedManagement.

Packages sometimes move from one component to another, according to policy or licensing changes, as managed by the archive administrators. Special consideration is necessary when packages move into main or restricted, as this implies a commitment of ongoing maintenance. Such changes must follow the MainInclusionProcess.

Anchor(Autobuilders)

Autobuilders

Ubuntu source packages are automatically built for a variety of platforms by Launchpad, which provides [https://launchpad.net/ubuntu/+builds build status information]. Build log files are available from Launchpad as well, by [https://launchpad.net/ubuntu/ searching for the package] and selecting a version.

Some supplementary information about the build infrastructure is available on BuildDaemons.

Anchor(Removals)

Removing Packages

Packages which are removed from Debian are semi-automatically removed from Ubuntu universe on a regular basis by the administrators. However, packages are not removed from Ubuntu main without explicit request, nor are packages which originated elsewhere. To request removal of such a package, file a bug against the package and subscribe the ubuntu-archive team.

The bug must have the following elements:

  • which release to remove it from (ie, gutsy)

  • mention to remove both source and all binaries
  • a rationale for why to remove them
  • make sure that it has no rdepends

If you are not an [:UbuntuDevelopers:Ubuntu developer] use the following [:SponsorshipProcess:process]. Also, if you need help deciding whether a package ought to be removed, please discuss on the ubuntu-devel mailing list rather than asking the archive administrators.

http://people.ubuntu.com/~ubuntu-archive/removals.txt has information on which packages were removed and why.

Anchor(Architectures)

Architectures

Packages are typically built for each of several architectures. For example, the hello package is built on i386, amd64, sparc, etc. These are divided into two categories according to their level of official support by the project.

Official Architectures

These are officially supported and maintained by the Ubuntu project. Canonical Ltd. provides server resources to build, store and distribute packages and installation media for them, and the core development team is responsible for their upkeep. Build failures on these architectures are considered serious bugs. Each official Ubuntu release and update includes appropriate support for these architectures. There may or may not be a team which is specifically responsible for architecture-specific issues. The kernel team builds and tests the Ubuntu kernel on these architectures.

  • i386

  • amd64

  • sparc

Ports

These are maintained on a best-effort basis by interested volunteers in the Ubuntu community. Each architecture has a corresponding community team formed of the developers who support it. Canonical Ltd. provides server resources to build, store and distribute packages and installation media for ports, however, the porting teams are responsible for their operation and maintenance, including the kernel, toolchain and build infrastructure. Build failures are not considered a serious issue by the core team. Ports may issue new releases or updates out of sync with official Ubuntu releases.

The ports system was announced here: https://lists.ubuntu.com/archives/ubuntu-announce/2005-October/000040.html

Anchor(InstallationMedia)

Installation media

Anchor(CDImages)

CD/DVD Images (ISOs)

An automated system builds CD and DVD images based on the packages in the archive. Its log files are available from [http://people.ubuntu.com/~ubuntu-archive/cd-build-logs/], and the set of packages included on them is driven by the [:SeedManagement:seeds].

New builds are usually produced on a daily basis and published at [http://cdimage.ubuntu.com/ cdimage.ubuntu.com]. Officially released images are published at [http://releases.ubuntu.com/ releases.ubuntu.com]. Both are available via HTTP, [:RsyncCdImage:rsync] and BitTorrent.

If you test a CD image, be sure to [:Testing/ReportingResults:report your results] to the development team.

There are two main varieties of images. One type primarily contains packages, and uses the [#DebianInstaller "alternate" installer] (e.g., the server ISO and the alternate ISO). The other contains a casper filesystem and uses the [#Ubiquity Ubiquity installer] (e.g., the desktop ISO)

Anchor(Installers)

Installers

Further details regarding installer development, including source code access, can be found on the InstallerDevelopment page.

Anchor(Ubiquity)

Ubiquity

Ubiquity is a graphical installation program which, simply speaking, copies a pre-installed ("live") Ubuntu system and reconfigures it for the system's hardware. It is designed to handle common installation scenarios quickly and easily. Details can be found in the [http://codebrowse.launchpad.net/~ubuntu-core-dev/ubiquity/trunk/annotate/?file_id=README-20051205083553-550dab3cb68ad622 Ubiquity README].

Anchor(DebianInstaller)

Alternate (Debian) Installer

The Debian installer builds up the installed system from scratch using .deb packages. It is designed for flexibility, and supports more complex installation scenarios. A good overview can be found in the [http://d-i.alioth.debian.org/doc/talks/debconf6/paper/ debian-installer paper from DebConf 6].

Anchor(Resources)

Other Resources

These resources should be incorporated into new or existing sections elsewhere in this document, but are temporarily recorded here so that we remember to come back to them later:


CategoryProcess