UbuntuDevelopment
ContentsBRTableOfContents |
Overview of Development
Ubuntu is developed by a [http://launchpad.net/~ubuntu-dev team] of UbuntuDevelopers, including both [http://launchpad.net/~ubuntu-core-dev core developers] and [http://launchpad.net/~motu MOTU]. It is based on [http://www.debian.org/ Debian], and [#ReleaseProcess periodically released] according to a schedule.
This process is transparent to the public, and open to contributions from the community.
If you have been directed to this page for advice on contributing to Ubuntu as a developer, you may also be interested in ContributeToUbuntu.
Working with Other Developers
You are not alone! Ubuntu is the work of many developers, and we devote some effort to enabling efficient collaboration with tools, infrastructure, government and a cooperative spirit.
Starting points
UbuntuDevelopers explains the role of developers in the Ubuntu project. NewDeveloperProcess explains how to join the team.
The [:MOTU] team, in addition to their development activities, provide information and guidance for new and prospective Ubuntu developers. If you're newly interested in Ubuntu development and looking for answers, introduce yourself and listen in!
If you're looking for tasks which need doing, many of those are tracked in the bug tracking system. The BugSquad maintains several lists of them at ["BugSquad/Tags"].
If you already have experience working with Debian packages, most of your knowledge applies equally well to Ubuntu packaging. If you are a Debian developer, UbuntuForDebianDevelopers summarizes some of the differences between the projects, and later sections in this document provide details of our infrastructure.
Communication
Email discussion among Ubuntu developers takes place on the [http://lists.ubuntu.com/mailman/listinfo/ubuntu-devel ubuntu-devel mailing list], which is moderated (excepting registered Ubuntu developers). The [http://lists.ubuntu.com/mailman/listinfo/ubuntu-devel-discuss ubuntu-devel-discuss mailing list] is available for open discussion about Ubuntu development. All UbuntuDevelopers should subscribe to the [http://lists.ubuntu.com/mailman/listinfo/ubuntu-devel-announce ubuntu-devel-announce mailing list], where important development events are announced. [https://lists.ubuntu.com/mailman/listinfo/ Various other mailing lists] are available, some of which focus on specific areas of development.
The #ubuntu-devel channel on the FreeNode IRC network is home to many Ubuntu developers for real-time communication.
[#Notifications Automated notifications of development activity] are also useful for keeping up with what other developers are working on.
A comprehensive matrix of communication channels can be found on DeveloperCommunication.
Bugs and the BugSquad
[http://bugs.launchpad.net/ubuntu Ubuntu bug reports] are tracked in Launchpad. HelpingWithBugs contains information about how they are handled. The BugSquad documentation describes how to cooperate with other developers and volunteers working on bug triage; it is required reading for new developers, as developers will typically need to spend a significant amount of time working with the bug tracking system.
The BugSquad (and the ["UbuntuQA"] team, which is comprised of more experienced triagers and is responsible for prioritising bugs) is here to help you as a developer. If you are responsible for a non-trivial number of bugs, it is a good idea to spend some time on helping them help you. A useful starting point is to add specific information about your packages to DebuggingProcedures: this may include both special tricks for debugging them effectively and any particular policy you have on how you want your bugs to be handled (e.g. assignment, tags, etc.). When adding significant chunks of new information to DebuggingProcedures, please send a note to ubuntu-bugsquad@lists.ubuntu.com about it.
Members of the bugsquad can be found on the #ubuntu-bugs channel on the FreeNode IRC network. There is also the [https://lists.ubuntu.com/mailman/listinfo/ubuntu-bugsquad ubuntu-bugsquad mailing list] for general discussion regarding bugs and bug triaging.
Release Critical bugs
[https://bugs.launchpad.net/ubuntu/+bugs?field.searchtext=&orderby=-importance&search=Search&field.status%3Alist=Unconfirmed&field.status%3Alist=Needs+Info&field.status%3Alist=Confirmed&field.status%3Alist=In+Progress&field.status%3Alist=Fix+Committed&field.importance%3Alist=Critical&assignee_option=any&field.assignee=&field.bug_reporter=&field.bug_contact=&field.milestone%3Alist=200&field.component=1&field.component=2&field.component-empty-marker=1&field.status_upstream=&field.status_upstream-empty-marker=1&field.omit_dupes.used=&field.omit_dupes=on&field.has_patch.used=&field.tag=&field.has_cve.used=&field.has_no_package.used= Showstopper bugs] Critical and milestoned to the relevant release. Those bugs will hold up the release if not fixed.
[https://bugs.launchpad.net/ubuntu/+bugs?field.searchtext=&orderby=-importance&search=Search&field.status%3Alist=Unconfirmed&field.status%3Alist=Needs+Info&field.status%3Alist=Confirmed&field.status%3Alist=In+Progress&field.status%3Alist=Fix+Committed&field.importance%3Alist=High&field.importance%3Alist=Critical&assignee_option=any&field.assignee=&field.bug_reporter=&field.bug_contact=&field.milestone%3Alist=200&field.component=1&field.component=2&field.component-empty-marker=1&field.status_upstream=&field.status_upstream-empty-marker=1&field.omit_dupes.used=&field.omit_dupes=on&field.has_patch.used=&field.tag=&field.has_cve.used=&field.has_no_package.used= Release-critical bugs] Critical and high importance bugs milestoned to the relevant release. Those bugs need to be fixed or worked around/documented before the release.
The Release Process
Each release cycle follows the same general pattern, with the following major phases. UbuntuDevelopers are expected to track this process closely, and ensure that their work is aligned with that of others. TimeBasedReleases mean that we need to be well coordinated in our efforts to produce an on-time release.
Beginning a new Release
At the beginning of each cycle, the Ubuntu infrastructure is prepared for a new branch of development. The package build system is set up, the toolchain is organized, seeds are branched, and many other pieces are made ready before development can properly begin. Once these preparations are made, the new branch is officially announced and opened for package uploads.
Planning
The features to be targeted for each release cycle are organised using [:SpecSpec:specifications]. Some of these come from strategic priorities for the distribution as a whole, some are proposed by individual developers, and some are drawn from the IdeaPool or the [https://blueprints.launchpad.net/ubuntu list of existing specifications (long)]. The proposed features are discussed at the DeveloperSummit at the beginning of each release cycle, and soon after, the TechnicalBoard and its advisors review proposed features and select a set of development projects which fit together well into the overall plan for the release.
The planned feature list for the current release cycle (Ubuntu 7.10, Gutsy) can be found at http://blueprints.launchpad.net/ubuntu/gutsy/+specs
Merging with Upstream
The first phase of the release cycle is characterized by bringing new releases of upstream components into Ubuntu, either directly or via Debian. We merge from Debian because it's the single most effective way to keep up to date with upstream code (Debian maintainers package new upstream releases on a frequent basis, often faster than we are able to do so), and because Debian and Ubuntu are similar in many ways so their bug-fixes are often bug-fixes for us too. We do it every release cycle rather than taking the occasional cycle off because if we didn't it would be significantly harder ever to come back into sync.
Anything that we don't modify - strictly, anything whose version number does not contain the substring "ubuntu", and which isn't in the [http://people.ubuntu.com/~ubuntu-archive/sync-blacklist.txt manually-maintained blacklist] - is "synced" from Debian semi-automatically: the source package is copied verbatim and rebuilt on our buildds. Anything that we've modified either needs to be merged by a developer, or needs a developer to request that the package be synced overriding the Ubuntu changes in the event that Debian took all our changes or they no longer apply for some other reason. Managing to get a package back into sync is usually a good thing, as it saves us from having to put maintenance effort into that package.
For more information on the practicalities of this process, see the [#SyncingAndMerging Syncing and Merging] section below.
This phase can be said to end when all packages have been brought up to date at least once, and the result has been sufficiently stabilized to produce the first milestone, with installable CD images. This must be no later than the DebianImportFreeze for the release cycle.
Feature Development
During this phase, the focus is on development projects which have been planned for the release. These projects often begin while merging is still underway, and accelerate the pace of their development once the package archive is reasonably consistent and usable.
Stabilization (Freeze)
Freeze states are described on the release schedule for the current release ([:GutsyReleaseSchedule:gutsy]), and the ReleaseScheduleTemplate
To request an exception, follow FreezeExceptionProcess
During freezes, uploads are sometimes held in a queue for manual approval. A [http://people.ubuntu.com/~ubuntu-archive/queue/ mirror] of this queue is available to ease coordination during these periods.
Milestones
Finalization
Stable Releases
Released versions of Ubuntu are intended to be stable. This means that users should be able to rely on their behaviour remaining the same, and therefore, updates are only released under special circumstances. These criteria, and the procedure for preparing such an update, are described in StableReleaseUpdates and SecurityUpdateProcedures.
Packaging
If you're interested in packaging work, but don't have much experience yet, you should get in touch with [:MOTU:the MOTU team], who provide mentoring for new developers. [:MOTU/Mentoring:Mentoring] is available too, via dedicated mentors and/or a [http://lists.ubuntu.com/mailman/listinfo/ubuntu-motu-mentors mailing list].
Working with Debian-format Packages
Ubuntu uses the Debian packaging format. The following resources explain how to create and modify Debian-format packages.
If you are already familiar with Debian development, UbuntuForDebianDevelopers explains some of the differences between the projects.
All Ubuntu developers should be familiar with the [http://www.debian.org/doc/maint-guide/ Debian New Maintainer Guide] and the [http://www.debian.org/doc/debian-policy/ Debian Policy Manual]; though be aware that there are many differences (technical, social and procedural) between Ubuntu and Debian of which they must also be aware.
[http://women.debian.org/wiki/English/PackagingTutorial A packaging tutorial] is available from the Debian Women project, as is [http://women.debian.org/wiki/English/MaintainerScripts an explanation of maintainer scripts] and [http://women.debian.org/wiki/English/AdvancedBuildingTips further help with building packages].
Many packages use tools to help manage multiple patches. [https://wiki.ubuntu.com/MOTU/School/PatchingSources Patching Ubuntu packages] (from the MOTU school) explains how to work with them.
The [https://perso.duckcorp.org/duck/cdbs-doc/cdbs-doc.xhtml CDBS Manual] explains how to work with packages using the CDBS packaging scripts, one example of a patch system (and more)
Packaging shared libraries is a delicate task, and getting it wrong can cause upgrade headaches for users. The [http://www.netfort.gr.jp/~dancer/column/libpkg-guide/libpkg-guide.html Debian Library Packaging Guide] can be useful in avoiding some of the common traps.
For a deeper understanding of the packaging process, you might want to have a look at this one: http://women.debian.org/wiki/English/BuildingWithoutHelper
http://people.debian.org/~calvin/unofficial/ contains information on maintaining your own archive of unofficial packages.
Tamir wrote an article on [http://ubuntuforums.org/showthread.php?t=51003&page=1&pp=10 How to make Debian-standard debs from scratch] on the Ubuntu Forums
[https://lists.ubuntu.com/archives/ubuntu-motu/2006-February/000443.html how to write watch files] for sourceforge.net hosted projects
Working with Ubuntu Packages
Set the target suite in debian/changelog to be the code name of the current development branch, e.g. "dch -D gutsy"
When working with a package which originated in Debian, use a version number derived from the Debian version number with ubuntu<revision> appended. e.g. Debian 1.0-2 becomes 1.0-2ubuntu1, followed by 1.0-2ubuntu2, etc.
- When creating a new package which may later be added to Debian, use a revision of the form -0ubuntu1
Remember to include the orig.tar.gz if this is a new upstream version of a non-native package but you have already patched it before upload. A missing original tarball may cause the upload to be rejected or silently dropped. Use dpkg-buildpackage -S -sa to generate such an upload. If the orig.tar.gz is already in the distribution then you don't need to upload it again.
- Always be aware of the release schedule and any applicable [#Freezes freezes]. The cooperation of all developers is needed in order to ensure a successful release!
- If your changes may affect the work of other developers, it is a good idea to discuss them on a mailing list first
Revision control (Bazaar)
Bazaar, an open source revision control system and Canonical sponsored project, is the preferred revision control system in Ubuntu. Many Ubuntu packages are [:BzrMaintainerHowto:maintained in Bazaar], which makes it easy for other developers to [:BzrContributorHowto:contribute changes to them], which can be easily merged by the maintainer.
Note that, as a practical matter, many packages are not yet maintained in Bazaar, but in other revision control systems or none on a case-by-case basis. Where no revision control system is used, the history of uploads recorded in [https://launchpad.net/ubuntu/+search Launchpad] may be useful.
Derivative Distributions
Several derivatives of Ubuntu are available (also sometimes called "flavours"). A number of people in the Ubuntu community work on one or more of these derivative versions. Kubuntu, Edubuntu, and Xubuntu are all maintained directly in the Ubuntu archive.
Building
You should always build and test packages locally before submitting them to Ubuntu. Failure to do so will waste the time of other members of the community, so please be considerate.
You may want to build them in a [:DebootstrapChroot] or in [:PbuilderHowto:pbuilder]
Backports are explained at [:BackportsHowto]
The Package Archive
All current official Ubuntu packages are stored in the master archive, which is widely [:Mirrors:mirrored]. A search interface is available at [http://packages.ubuntu.com]. Old versions can be retrieved from [http://launchpad.net/ubuntu Launchpad].
It is administered by the [http://launchpad.net/~ubuntu-archive archive administration team].
Uploading
If you are not yet an official Ubuntu developer, you can arrange for your package to be uploaded via the SponsorshipProcess.
Packages are uploaded via FTP to ftp://upload.ubuntu.com/ using dput or dupload.
Notes for preparing your upload:
Make source-only uploads, i.e. use "dpkg-buildpackage -S"
When uploading to [:REVU], please include the orig tarball as well (use parameters -S -sa)
When your upload is processed (typically within a matter of minutes), you will receive an email with the result of your upload, whether it succeeds or fails, unless you use an unregistered email address. The system will only send mail to an address which belongs to a launchpad account which is a member of the relevant team for uploading. E.g. [http://launchpad.net/people/ubuntu-dev ubuntu-dev] for universe and [http://launchpad.net/people/ubuntu-core-dev ubuntu-core-dev] for main.
Your upload must be signed by GPG key registered in launchpad. If the signature cannot be traced to a member of the appropriate team, then the upload will be silently rejected.
To add a new package to Ubuntu, simply upload it as usual. Any new packages uploaded are put in a [https://launchpad.net/ubuntu/gutsy/+queue queue] to be checked by the administrators before being included.
Autobuilding and Publishing
Once an upload has been accepted, it takes some time to be [#Autobuilders built] and published in the archive. For simple packages, this is usually on the order of an hour, but varies depending on release activity (uploads may be temporarily suspended), the time needed to build the package (including other packages in the build queue), and other factors.
Notification of changes
Notifications of uploads are sent to a mailing list. A different list is used for each Ubuntu release:
[http://lists.ubuntu.com/mailman/listinfo/dapper-changes/ dapper]
[http://lists.ubuntu.com/mailman/listinfo/edgy-changes/ edgy]
[http://lists.ubuntu.com/mailman/listinfo/feisty-changes/ feisty]
[http://lists.ubuntu.com/mailman/listinfo/gutsy-changes/ gutsy]
RSS Feeds of these messages are available at [http://media.ubuntu-nl.org/rss/].
Changelogs for all packages are available at http://changelogs.ubuntu.com/changelogs/ (this is the source used by update-manager and Synaptic).
Syncing and Merging
(See the [#MergeProcess rationale] above.)
Most packages in Ubuntu originate elsewhere, including Debian and related package repositories.
A sync copies a source package verbatim from an external repository into Ubuntu, overwriting any package of the same name. This is used when a newer version of it is available, and should be included in Ubuntu, and happens automatically during some phases of the release cycle. To request a sync, follow the SyncRequestProcess.
A merge is a three-way merge of a package which originated in an external repository. This is used when there is a newer version available from the external repository, but the package has also been modified (branched) in Ubuntu. [http://merges.ubuntu.com/ Merge-o-Matic] assists with this work, and ["MOTU/Merging"] explains how and when to merge. Packages which are [#Bazaar maintained in Bazaar] can and should be merged using Bazaar itself.
The "Last Uploader" column in the Merge-o-Matic output is the default assignee for the merge, following the touched-it-last maintenance principle. However, you can and often should grab other people's merges if they don't have time or you feel you can do a better job. It's polite and often a good idea (though not mandatory) to contact the other person first to make sure you aren't duplicating work.
Backports work similarly to syncs, but have somewhat different requirements. To request a backport, follow the BackportRequestProcess.
Consistency
The archive is periodically checked for various inconsistencies, such as incorrect dependency relationships between packages.
http://people.ubuntu.com/~ubuntu-archive/testing/ - packages which are uninstallable due to unsatisfiable dependencies
http://people.ubuntu.com/~robertc/possible-conflicts/ - paths included in multiple packages without declaring Replaces or Conflicts or doing a diversion. Contact RobertCollins with suggestions for improvements or questions. (Email or on #ubuntu-devel/#ubuntu-motu).
A special case is the installer:
The Kernel-Version: field in the installer file in all the [https://code.launchpad.net/ubuntu-seeds seeds] needs to be current.
It needs to be built against the current kernel. This can be checked on the [http://archive.ubuntu.com/ubuntu/dists/gutsy/main/installer-i386/current/images/udeb.list gutsy installer's udeb list] (please also verify the other architectures).
Managing Components
Ubuntu packages are classified into components according to [http://www.ubuntu.com/ubuntu/components maintenance and licensing criteria], a process which is described in SeedManagement.
Packages sometimes move from one component to another, according to policy or licensing changes, as managed by the archive administrators. Special consideration is necessary when packages move into main or restricted, as this implies a commitment of ongoing maintenance. Such changes must follow the MainInclusionProcess.
Autobuilders
Ubuntu source packages are automatically built for a variety of platforms by Launchpad, which provides [https://launchpad.net/ubuntu/+builds build status information]. Build log files are available from Launchpad as well, by [https://launchpad.net/ubuntu/ searching for the package] and selecting a version.
Some supplementary information about the build infrastructure is available on BuildDaemons.
Removing Packages
Packages which are removed from Debian are semi-automatically removed from Ubuntu universe on a regular basis by the administrators. However, packages are not removed from Ubuntu main without explicit request, nor are packages which originated elsewhere. To request removal of such a package, file a bug against the package and subscribe the ubuntu-archive team. If you need help deciding whether a package ought to be removed, please discuss on the ubuntu-devel mailing list rather than asking the archive administrators.
http://people.ubuntu.com/~ubuntu-archive/removals.txt has information on which packages were removed and why.
Installation media
CD/DVD Images (ISOs)
An automated system builds CD and DVD images based on the packages in the archive. Its log files are available from [http://people.ubuntu.com/~ubuntu-archive/cd-build-logs/], and the set of packages included on them is driven by the [:SeedManagement:seeds].
New builds are usually produced on a daily basis and published at [http://cdimage.ubuntu.com/ cdimage.ubuntu.com]. Officially released images are published at [http://releases.ubuntu.com/ releases.ubuntu.com]. Both are available via HTTP, [:RsyncCdImage:rsync] and BitTorrent.
If you test a CD image, be sure to [:Testing/ReportingResults:report your results] to the development team.
There are two main varieties of images. One type primarily contains packages, and uses the [#DebianInstaller "alternate" installer] (e.g., the server ISO and the alternate ISO). The other contains a casper filesystem and uses the [#Ubiquity Ubiquity installer] (e.g., the desktop ISO)
Installers
Further details regarding installer development, including source code access, can be found on the InstallerDevelopment page.
Ubiquity
Ubiquity is a graphical installation program which, simply speaking, copies a pre-installed ("live") Ubuntu system and reconfigures it for the system's hardware. It is designed to handle common installation scenarios quickly and easily. Details can be found in the [http://codebrowse.launchpad.net/~ubuntu-core-dev/ubiquity/trunk/annotate/?file_id=README-20051205083553-550dab3cb68ad622 Ubiquity README].
Alternate (Debian) Installer
The Debian installer builds up the installed system from scratch using .deb packages. It is designed for flexibility, and supports more complex installation scenarios. A good overview can be found in the [http://d-i.alioth.debian.org/doc/talks/debconf6/paper/ debian-installer paper from DebConf 6].
Other Resources
These resources should be incorporated into new or existing sections elsewhere in this document, but are temporarily recorded here so that we remember to come back to them later:
Ubuntu Packaging Guide: https://help.ubuntu.com/6.10/ubuntu/packagingguide/C/index.html http://doc.ubuntu.com/ubuntu/packagingguide/C/index.html
Debian Python Policy: http://www.debian.org/doc/packaging-manuals/python-policy/