Ubuntu Wiki

Include(KernelTeamHeader)

This document is geared towards gutsy and later releases. For feisty, please see KernelMaintenanceFeisty.

This is meant as a reference for the kernel team.

Getting the source

Ubuntu kernel source is available via git. More information can be obtained from KernelGitGuide.

Build system breakdown

This requires some knowledge of the Debian build system (including the debian/rules file, etc). The gutsy build system was rewritten from the ground up, to mainly exclude kernel-package as a built tool, since it was getting too heavy and restrictive for our needs. We still use kernel-wedge for udeb creation.

Flavours

Flavours are how we define different types of kernels on the same architecture. Each flavour can be targeted to a specific CPU, or provide some functionality for a specific purpose.

For example, the amd64 (x86_64) architecture has these flavours:

• generic - The default install kernel.
• server - Optimized for server installations.

When building on the amd64 architecture, these are the kernels that are built. In reality, they only differ in slight variations of kernel config options.

Configs

Each flavour is built using a pre-defined configuration file (used by the kernel build system). Each architecture contains it's config files in debian/config/<arch>/. The files are split for each flavour in debian/config/<arch>/config.<flavour>, and common options are shared in debian/config/<arch>/config.

When the kernels are built, the common and flavour specific config files are concatenated together in the build directory.

debian/control

The debian/control file is generated from debian/control.stub by the kernel-wedge command.

The debian/control.stub file is generated from debian/control.stub.in by a debian/rules target. This is populated by files defining the flavours in debian/control.d/vars.*.

The vars files define package descriptions for each flavour.

debian/changelog

The changelog file is where we track all changes made to the repository. This is a standard debian/changelog. However, changes are not added to it manually. Instead, it is done automatically at release time using git-log. See below for process.

ABI

The ABI is probably one of the most important features of the build system. In the kernel package naming, the ABI is the numerical component immediately following the kernel version. E.g.

2.6.22-4-generic

The ABI in the above example is 4. The ABI is represented in the debian/changelog as:

linux-source-2.6.20 (2.6.22-4.10) feisty; urgency=low

The minor version after the ABI is an ever increasing revision. It basically denotes how many uploads have been made for this kernel version.

The kernel ABI is derived from the Modules.symver file produced during the kernel build. Each exported function is given a hash defining the function's arguments and return value, making it easy to detect changes which can break modules built against the kernel (such as our own linux-restricted-modules package).

The ABI files for the previous build are located in debian/abi/<prev-rev>/. The layout is similar to the debian/config/ directory in that each architecture is a subdirectory, and each flavour is a file with the ABI for that flavour in it.

When the kernel build process finishes, it checks each flavour against the prev ABI. If the previous and current ABI's are the same version (e.g. 6), and the ABI has changed, the build will fail. This signifies that an ABI bump is needed (see below).

As of now, a single ABI change in any flavour results in an ABI bump for the entire build. There is no per-flavour or per-architecture ABI number. This is for sanity reasons.

Module lists

Module lists are tracked similarly to the ABI, and are kept in debian/abi/<prev-ver>/<arch>/<flavour>.modules. The file is a basename sorted list, sans .ko extension, of all the modules in a particular flavour. This is checked regardless of ABI, since it's meant to track release-to-release to avoid modules going missing by mistake.

The ABI check will honor debian/abi/<prev-ver>/<arch>/ignore to ignore an entire architecture, or debian/abi/<pre-ver>/<arch>/<flavour>.ignore for a specific kernel flavour. This is useful for architectures that failed to build the last ABI.

The module check will honor a list of modules in debian/abi/<prev-ver>/modules.ignore that lists modules known to have gone missing since the last upload.

Development cycle

debian/scripts/misc/getabis 2.6.22 4.10

In the above example, 2.6.22 is the kernel version, and 4.10 is the last uploaded revision.

This will download all the previous deb's, and extract the ABI files, and module lists. You may find it easier to do this from a DC machine where the archive is closer (downloading all the kernel deb's is a bandwidth eater).

Starting a new release

debian/rules startnewrelease

This will generate a new debian/changelog entry for the next revision number, and modify appropriate other files under the debian/ directory.

Add new code

As new drivers are added and merges are done with upstream, it becomes necessary to update our configs in debian/config/. This is easily done for all architectures with one simple command:

debian/rules updateconfigs

This runs silentoldconfig for all flavours, so if any possible config options are unanswered, it will ask you. Alternatively, if you know of a new config option, you can alleviate answering the same question for it on all flavours by appending it to the debian/config/<arch>/config files, and running the above command.

Once the configs are updated, you simply commit the new changes:

git-update-index debian/config/*/*
git-commit -s -F debian/commit-templates/update-configs

cd <kerneldir>/debian/config/<arch>
cp <path>/<whole-config-for-flavour> config.<flavour>
for cfg in config.*; do
    cat config >> $cfg
done
rm config
cd <kerneldir>
debian/rules updateconfigs

If it becomes necessary to increase the ABI, you simply need to edit debian/changelog, increase the ABI in the version, then run this command:

debian/rules debian/control.stub

linux-source-2.6.22 (2.6.22-4.11) gutsy; urgency=low

linux-source-2.6.22 (2.6.22-5.11) gutsy; urgency=low

In debian/commit-templates/ are files used for making git-commit's easier.

You would use these files in this way:

git-commit -s -F debian/commit-templates/<template> -e

Always commit using -s to add your Signed-off-by to the commit log. The -e flag says to edit the log before committing. Some templates do not require editing (e.g. the bumpabi and updateconfigs templates).

The primary one you will use is the patch template. It is commented heavily, so should be self explanatory.

Upstream syncing

• Run git-status to be sure you have no outstanding commits, or extraneous files in your local tree.

• It may also be a good idea to run git-ls-files --others to check for extraneous files.

• Create the changelog entries, with the following commands:

   debian/rules insertchanges
   <open debian/changelog in an editor to verify release and date>
   git-update-index debian/changelog
   git-commit -s -F debian/commit-templates/update-configs

• Tag the release. The -m option is the message for the tag commit, and matches the actual tag name.

   git-tag -s -m Ubuntu-2.6.20-6.11 Ubuntu-2.6.20-6.11

• Push the changes and tag to our repo.
• If the tree has a .orig.tar.gz in the repo, download that to the same directory as the git tree.
• Run dpkg-buildpackage:

   dpkg-buildpackage -S -rfakeroot -I.git -I.gitignore -i'\.git.*'

or to build _with_ source use:

•  dpkg-buildpackage -S -rfakeroot -I.git -I.gitignore -i'\.git.*' -sa

• Upload using whatever tools you wish (dupload for example).

• Submit your patch for upstream inclusion when appropriate, e.g., quirk updates.

It should be noted that for development releases, a new tarball is generated at each upload. However at release time, a .orig.tar.gz is used, which is based on the upstream tarball of the released kernel we are using.

Pushing to our repo

The kernel team should push changes to our repo (see KernelGitGuide for public repo). To make this easy, you can setup a script called push-gutsy with this:

branches="master"
remotes="zinc.ubuntu.com:/srv/kernel.ubuntu.com/git/ubuntu/ubuntu-gutsy.git"

cd /path/to/your/repo

for repo in $remotes; do
        echo "Pushing to: $repo"
        git-send-pack $repo $branches $@
        echo
done

push-gutsy Ubuntu-2.6.22-4.10

On occasion, security patches are applied to stable kernels. For these kernels, you will set the archive in debian/changelog to <rel>-security, where <rel> is some release like edgy.

The security team generally provides patches in the form of CVE's, with GIT SHA URL's to an upstream patch that fixes the problem. We usually use will fetch from the upstream GIT repo, and git-cherry-pick to get the fix into the local tree.

The easiest way to fetch the remote repo's objects is to create a remote file such as .git/remotes/upstream-linux-2.6.20, containing:

URL: git://git.kernel.org/pub/scm/linux/kernel/git/stable/linux-2.6.20.y.git
Pull: master:upstream-linux-2.6.20

git-fetch upstream-linux-2.6.20
git-cherry-pick -e -x aa1812b3e344dbb2c7f30e1b95661cbd36d0cd2b

$ sudo apt-get install linux-kernel-devel devscripts fakeroot libncurses5-dev ccache

Aside from the standard dpkg-buildpackage build method, there are several easier ways to do build for developers. When I actually add new code, I generally do an out-of-tree build for test, like so:

$ cd ubuntu-gutsy
$ mkdir ../build
$ cat debian/config/i386/config{,.generic} > ../build/.config
$ make O=`pwd`/../build oldconfig
$ make O=`pwd`/../build

One other helpful tool for a developer is to install the ccache package. This speeds rebuild by a large factor.

Normal full build

fakeroot debian/rules binary-arch

fakeroot debian/rules binary-generic

So you can change generic with any flavour you want to build. You may also want to do:

fakeroot debian/rules binary-headers

fakeroot debian/rules binary-modules-<flavour>

There are several build options, in the form of environment variables and make variables. Make options are passed on the debian/rules command line.

• Environment

  • CONCURRENCY_LEVEL=X where X is a number to pass using make's -j option.

  • AUTOBUILD - (can be passed to make as well). Tells the build to create a non-official package with generated ABI. Used for daily builds of the kernel.

• Make

  • skipabi=true - Used for test only, causes the build to skip the ABI consistency check.

  • skipmodule=true - Same as above, but for the module check.

  • NOKERNLOG - Used for the insertchanges or printchanges target. Causes git-ubuntu-log to not include upstream kernel changes in changelog output.

  • PRINTSHAS - Include GIT SHAs in printchanges and insertchanges targets.

E.g. When trying to avoid "previous or current ABI file missing" and not wanting to do an ABI-bump:

CONCURRENCY_LEVEL=8 fakeroot debian/rules binary-generic binary-headers skipabi=true skipmodule=true

For an ABI bump, edit debian/rules and find abi_version. Change the value to the new ABI number. Use dpkg-buildpackage as normal after creating a new changelog entry.

• Can someone please put up a bit of info on how to get this package from git (and re-build the package), the way KernelGitGuide does? The same for linux-ubuntu-modules. --BogdanButnaru

  • I'm not certain, but I believe this can't be done with git. instead,

apt-get source linux-restricted-modules-VERSION then look for the abi-version in debian/rules and make sure it corresponds to the abi of your new kernel. then fakeroot dpkg-buildpackage  which should build the package properly.

• Edit: linux-ubuntu-modules are (as far as I can tell) built exactly the same as the kernel, but using   git clone git://kernel.ubuntu.com/ubuntu/ubuntu-gutsy-lum.git ubuntu-gutsy-lum for getting the sources, and fakeroot debian/rules binary-modules-generic for building the package. I'm not sure about linux-restricted-modules; it's not in git, is it? --BogdanButnaru

linux-ubuntu-modules

This package is based on the linux-source build system, so shares the same intricacies. Setting the ABI is the same as in linux-source (via debian/changelog). A quick summary: to checkout, use  git clone git://kernel.ubuntu.com/ubuntu/ubuntu-gardy-lum.git ubuntu-hardy-lum  and to build, use fakeroot debian/rules binary-modules-FLAVOUR Please note that the alsa tree is now in linux-ubuntu-modules, so if you're having sound issues you might want to try building this package in addition to the main kernel package. --MattPrice

linux-meta

For an ABI bump, edit debian/rules and find KERNEL_ABI. Do the same as for linux-restricted-modules.

linux-backports-modules

This package is based heavily on linux-ubuntu-modules.

main, proposed and security, OH MY!

Technically known as <rel>-proposed (e.g. edgy-proposed) is where we can push changes that need testing, and are unsuited for a wide audience of users who expect a rock solid distribution.

Typical changes in this tree are new drivers, and large updates to support new hardware.

The GIT tree is named ubuntu-<rel>-updates.git. It's ABI remains a constant -50, and the changelog for this release never rolls, meaning the it only has one changelog set for every upload. This changelog set is updated every upload to reflect new patches, and to remove ones that are either broken, or have been merged to the stable kernel tree.

See the KernelUpdates page for more information.

security

This is self explanatory, or so it would seem. While the ubuntu-<rel>.git tree is meant primarily for security updates, occasionally stable changes are merged into it from proposed.

CategoryKernel