MainlineBuilds

Differences between revisions 5 and 56 (spanning 51 versions)
Revision 5 as of 2011-10-22 10:00:29
Size: 5847
Editor: apw
Comment:
Revision 56 as of 2018-10-24 21:41:16
Size: 10782
Editor: dsmythies
Comment: try to reflect the current appearance of the build directory, including the linux-modules files
Deletions are marked like this. Additions are marked like this.
Line 5: Line 5:
Generally Ubuntu systems run with the stock Ubuntu kernels. However it is handy to be able to test with unmodified mainline kernels to help locate problems in the Ubuntu kernel patches, or to confirm that upstream has fixed the issue. To this end we now offer mainline kernel builds. These mainline kernels are made from unmodified kernel source but using the Ubuntu kernel configuration files. These are then packaged as Ubuntu .deb files for simple installation. = Introduction =
Line 7: Line 7:
We currently build five sets of mainline kernels. All formal tags from Linus' tree and from the stable trees, plus:
 1. the tip of the master branch from [[http://git.kernel.org/?p=linux/kernel/git/torvalds/linux-2.6.git;a=summary|Linus' tree]] daily,
 1. the tip of the [[http://git.kernel.org/?p=linux/kernel/git/airlied/drm-2.6.git;a=shortlog;h=refs/heads/drm-next|drm-next]] head of Dave Airlie's [[http://git.kernel.org/?p=linux/kernel/git/airlied/drm-2.6.git;a=summary|drm-2.6]] repository daily,
 1. the tip of the [[http://git.kernel.org/?p=linux/kernel/git/ickle/drm-intel.git;a=shortlog;h=refs/heads/drm-intel-next|drm-intel-next]] head of Chris Wilson's [[http://git.kernel.org/?p=linux/kernel/git/ickle/drm-intel.git;a=summary|drm-intel]] repository daily,
 1. the tip of the master branch of the debloat-testing tree daily,
 1. tags from the combined [[http://git.kernel.org/?p=linux/kernel/git/smb/linux-2.6.32.y-drm33.z.git;a=summary|v2.6.32.x.y]] tree (by StefanBader) which is v2.6.32.x with DRM from 2.6.33.y.
By default, Ubuntu systems run with the Ubuntu kernels provided by the Ubuntu repositories. However it is handy to be able to test with unmodified upstream kernels to help locate problems in Ubuntu kernel patches, or to confirm that upstream has fixed a specific issue. To this end we now offer select upstream kernel builds. These kernels are made from unmodified kernel source but using the Ubuntu kernel configuration files. These are then packaged as Ubuntu .deb files for simple installation, saving you the time of compiling kernels, and debugging build issues.
Line 14: Line 9:
This makes these kernels closer to the Lucid kernels which are based on 2.6.32 kernels with DRM backported from the 2.6.33 series. These kernels are not supported and are not appropriate for production use.
Line 16: Line 11:
= How do I install an upstream kernel? =
Line 17: Line 13:
== Mainline Kernels Archive == Following these steps in order will help you successfully test an upstream kernel.
Line 19: Line 15:
The [[http://kernel.ubuntu.com/~kernel-ppa/mainline/?C=M;O=D|mainline kernels archive]] is located at the URL below, there is a directory for each mainline build: == Prepare OS to install an upstream kernel ==
Line 21: Line 17:
  [[http://kernel.ubuntu.com/~kernel-ppa/mainline|http://kernel.ubuntu.com/~kernel-ppa/mainline]] First, if one is using select proprietary or out-of-tree modules (e.g. [[https://launchpad.net/ubuntu/+source/bcmwl|bcmwl]], [[https://launchpad.net/ubuntu/+source/fglrx-installer|fglrx]], NVIDIA proprietary graphics drivers, [[https://launchpad.net/ubuntu/+source/virtualbox|VitualBox]], etc.) unless there is an '''extra''' package available for the version you are testing, you will need to uninstall the module first, in order to test the mainline kernel. If you do not uninstall these modules first, then the upstream kernel may fail to install, or boot.
Line 23: Line 19:
The tagged releases are found under a directory matching their tag name and which kernel configuration they were built with (<tag>-<series>). Daily releases are found in the {{{daily}}} sub-directory named for the date they were made. == Choose the proper upstream kernel files ==
Line 25: Line 21:
Each build directory contains the header and image .deb files for the i386 and amd64 architectures, generic flavour.

== Mainline Kernel Mapping ==

In order to choose which is the closest mainline kernel to any particular Ubuntu release you can use the [[http://kernel.ubuntu.com/~kernel-ppa/info/kernel-version-map.html|Ubuntu to mainline mapping]] table, which contains mappings from Ubuntu releases and pockets to mainline versions.

Most likely that you want to download and test against the most [[http://kernel.ubuntu.com/~kernel-ppa/mainline/daily/current/|current]] version.

== Installing Mainline Kernels ==

To use the mainline kernel as-is you only only need to download and install the *image*.deb package that corresponds to your architecture, however if you need to build any external modules you also need the correct *header*.deb and *source*.deb packages.

To install, download the common headers, architecture specific headers, and the architecture specific image. For example for 2.6.27.15 we have the following files, for i386 you would need those marked with B and C, amd64 take those marked A & C:

    {{{
A linux-headers-2.6.27-02062715-generic_2.6.27-02062715_amd64.deb
B linux-headers-2.6.27-02062715-generic_2.6.27-02062715_i386.deb
C linux-headers-2.6.27-02062715_2.6.27-02062715_all.deb
A linux-image-2.6.27-02062715-generic_2.6.27-02062715_amd64.deb
B linux-image-2.6.27-02062715-generic_2.6.27-02062715_i386.deb
The build directories are nicely organized into per architecture groups. For example, if one is using a 64-bit/amd64 architecture and wants the generic kernel version you would want those files marked A, from the appropriate group. <<BR>> If you want the low latency version, B.
<<BR>>
{{{
  Build for amd64 succeeded (see BUILD.LOG.amd64):
AB linux-headers-4.19.0-041900_4.19.0-041900.201810221809_all.deb
A linux-headers-4.19.0-041900-generic_4.19.0-041900.201810221809_amd64.deb
B linux-headers-4.19.0-041900-lowlatency_4.19.0-041900.201810221809_amd64.deb
A linux-image-unsigned-4.19.0-041900-generic_4.19.0-041900.201810221809_amd64.deb
B linux-image-unsigned-4.19.0-041900-lowlatency_4.19.0-041900.201810221809_amd64.deb
A linux-modules-4.19.0-041900-generic_4.19.0-041900.201810221809_amd64.deb
B linux-modules-4.19.0-041900-lowlatency_4.19.0-041900.201810221809_amd64.deb
Line 47: Line 34:
Once you have those downloaded they will need to be installed using dpkg: == Download upstream kernel files from the Ubuntu archive ==
Line 49: Line 36:
    {{{
sudo dpkg -i *.deb
The upstream kernels archive has a directory for each build. Note, if you are testing for a bug, please do not use the daily folder, but use the latest mainline kernel at the top from:
 [[http://kernel.ubuntu.com/~kernel-ppa/mainline/?C=N;O=D|http://kernel.ubuntu.com/~kernel-ppa/mainline/?C=N;O=D]]

It is best to verify the integrity of downloaded packages as explained below under "Verifying the mainline build binaries".

== Install all upstream kernel files ==

Next, one will execute the following command against each of the downloaded files via a terminal:
{{{
sudo dpkg -i FILENAME.deb
Line 53: Line 48:
When this process completes you should have a new entry on your boot menu representing the mainline kernel. This will appear as an entry like this:

    {{{
Ubuntu Intrepid, kernel 2.6.27-02062715-generic
If no errors show up, reboot while holding Shift then select "Advanced options for Ubuntu", then select and boot into the new entry that looks something like:
{{{
Ubuntu with linux 4.19.0-041900-generic
Line 59: Line 53:
== Uninstalling Mainline Kernels == = Problems installing upstream kernels =
Line 61: Line 55:
The mainline kernels have their own ABI namespace so they install side by side with the stock Ubuntu kernels (each kernel has a separate directory under /lib/modules/VERSION for example). This means that you can keep several mainline and Ubuntu stock kernels installed at the same time and select the one you need from the GRUB boot menu. == Virtualbox installed ==
Line 63: Line 57:
If you would like to uninstall a mainline kernel anyway, first use: Issues can occur installing an upstream kernel due to Virtualbox being installed. For example: {{{
Error! Bad return status for module build on kernel: 3.7.0-030700rc2-generic (x86_64)
Consult /var/lib/dkms/virtualbox/4.1.18/build/make.log for more information.
}}} As per above, you need to either install the '''extra''' package, if available, or uninstall virtualbox.
Line 65: Line 62:
    {{{ == Version of package needed by installer is too old ==

A failure to install can occur due to the OS having packages that are too old for the install to proceed. For example:{{{
depends on libssl1.1 (>= 1.1.0); however: Package libssl1.1 is not installed.
}}} If this is due the version of [[https://launchpad.net/ubuntu/+source/openssl|libssl1.1]] installed is too old then you need to upgrade your OS. However, if libssl1.1 is not installed at all, and the version that comes with your release is new enough, then install libssl1.1.

== Other install errors ==

If for some reason the kernel you attempted to build failed, and it's not due to the above, then continue to test the next most recent kernel version until you can test to the issue.

= Uninstalling upstream kernels =

The upstream kernels have their own ABI namespace, so they install side by side with the stock Ubuntu kernels (each kernel has a separate directory under /lib/modules/VERSION for example). This means that you can keep several mainline and Ubuntu stock kernels installed at the same time and select the one you need from the GRUB boot menu.

If you would like to uninstall an upstream kernel anyway, execute the following to find the exact name of the kernel packages you want to uninstall:
{{{
Line 69: Line 81:
to find the exact name of the kernel packages you want to uninstall, and then do:

    
{{{
and then execute the following to uninstall them:
{{{
Line 75: Line 86:
Remember that several packages belong to one kernel version: common headers, architecture specific headers and the architecture specific image. Remember that several packages can belong to one kernel version: common headers, architecture specific headers and the architecture specific image.
Line 77: Line 88:
<<Include(^Kernel/FAQ/DebuggingMainlineBuilds.*)>> Also, once the mainline packages are removed, one may still see entries for these via the above dpkg command. To purge these entries execute at a terminal: {{{
sudo dpkg --purge ENTRY
}}}
Line 79: Line 92:
== Submitting to kerneloops.org == = Mainline build tool chain =
Line 81: Line 94:
Ubuntu's default infrastructure for handling OOPSes will not work for these builds as they are external to Ubuntu. To contribute this information to kerneloops.org you will need to make a couple of changes. These kernels are built with the tool chain (gcc etc.) from the previous LTS (Ubuntu 8.04/10.04/12.04) depending on version. Therefore, out-of-tree kernel modules built with tools from other versions likely will not work.
Line 83: Line 96:
First, install the kerneloops-applet package which will handle the prompting and submission on any problem. Then edit /etc/kerneloops.conf and delete the ''submit-pipe'' line from the file, so that kerneloops-applet will be used. After a reboot
kerneloops-applet will then prompt you if there are any problems and you can choose to submit the information to kerneloops.org.
= Mainline kernel mapping to Ubuntu kernel =
Line 86: Line 98:
== Mainline Build Tool chain == The Ubuntu kernel is not bit-for-bit the same as the mainline. However, one may find the upstream release that the Ubuntu kernel is based on via the [[http://people.canonical.com/~kernel/info/kernel-version-map.html|Ubuntu to mainline mapping]] table.
Line 88: Line 100:
These kernels are built with the tool chain (gcc etc) from Hardy (Ubuntu 8.04), therefore out-of-tree kernel modules built with tools from other version won't work.
 
= Does the kernel team support the mainline kernel builds? =

The mainline kernel builds are produced for debugging purposes and therefore come with no support. Use them at your own risk.

= Where can I get the source for these builds? =

In each directory there is a COMMIT file which defines the base commit in Linus' master tree from which they were built. The patches in the same directory ????-* are applied on top of this commit to make the build tree. A mirror of Linus' tree is available from {{{git://kernel.ubuntu.com/virgin/linux.git}}}.

First download the COMMIT and patch files ????-* from the mainline build in question to a temporary directory:
{{{
git clone git://kernel.ubuntu.com/virgin/linux.git mainline
cd mainline
git checkout -b `cat ${MAINLINE}/COMMIT`
git am ${MAINLINE}/????-*
}}}

= Verifying the mainline build binaries =

In order to allow verification that the published builds are the builds made by the mainline build system, the individual files are checksummed and the results of that published as CHECKSUMS in the same directory. This file is in turn signed by the mainline builder using the GPG key below which can be obtained from the Ubuntu Keyserver:
{{{
pub 2048R/17C622B0 2008-05-01
      Key fingerprint = 60AA 7B6F 3043 4AE6 8E56 9963 E50C 6A09 17C6 22B0
uid Kernel PPA <kernel-ppa@canonical.com>
}}}

The verification can be done by running the following commands:

  1. Import the above public key to your keyring (if you haven't already done that):
  {{{
$ gpg --keyserver hkps://pgp.mit.edu --recv-key "60AA7B6F30434AE68E569963E50C6A0917C622B0"
}}}

  1. Download the CHECKSUMS and CHECKSUMS.gpg files from the build directory and verify if the CHECKSUMS is signed with the above key:
  {{{
$ gpg --verify CHECKSUMS.gpg CHECKSUMS
gpg: Signature made .... using RSA key ID 17C622B0
gpg: Good signature from "Kernel PPA <kernel-ppa@canonical.com>"
gpg: WARNING: This key is not certified with a trusted signature!
gpg: There is no indication that the signature belongs to the owner.
}}}

  1. Verify the checksums of downloaded deb files:
  {{{
$ shasum -c CHECKSUMS 2>&1 | grep 'OK$'
}}}
  You should get a line ending with "OK" for each of downloaded deb file and each type of checksums that are given in the CHECKSUMS file.

= Upstream kernels in detail =

We currently build five sets of upstream kernels. All formal tags from Linus' tree and from the stable trees, plus:
 1. the daily tip of [[http://git.kernel.org/cgit/linux/kernel/git/torvalds/linux.git|Linus' linux kernel source tree]],
 1. the tip of the [[http://cgit.freedesktop.org/~airlied/linux/log/?h=drm-next|drm-next]] head of Dave Airlie's [[http://cgit.freedesktop.org/~airlied/linux/|linux]] repository daily,
 1. the tip of the [[http://cgit.freedesktop.org/~keithp/linux/log/?h=drm-intel-next|drm-intel-next]] head of Keith Packard's [[http://cgit.freedesktop.org/~keithp/linux/|linux]] repository daily until 2012, after which it has been [[http://blog.ffwll.ch/2012/01/new-drm-intel-next-git-tree.html|taken over]] by Daniel Vetter at [[http://cgit.freedesktop.org/drm-intel/]], and in particular, the [[http://cgit.freedesktop.org/drm-intel/log/?h=drm-intel-next|drm-intel-next]] branch,
 1. the tip of the master branch of the debloat-testing tree daily,
 1. tags from the combined [[http://git.kernel.org/?p=linux/kernel/git/smb/linux-2.6.32.y-drm33.z.git;a=summary|v2.6.32.x.y]] tree (by StefanBader) which is v2.6.32.x with DRM from 2.6.33.y.
This makes these kernels closer to the Lucid kernels which are based on 2.6.32 kernels with DRM backported from the 2.6.33 series.

The tagged releases (as made by Linus and the stable maintainers) are found under a directory matching their tag name and which kernel configuration they were built with (<tag>-<series>).

Daily tip of the tree builds are found in the daily sub-directory named for the date they were made.

Each build directory contains the header and image .deb files for the generic flavour i386 and amd64 architectures, as well lowlatency.

= Can I install and then immediately use a kernel in a live environment? =

No. One has two choices to use a mainline kernel:
 1. Install the mainline kernel in an installed environment, restart, and choose this newly installed kernel.
 1. Build a live environment with the new kernel in it. Given the amount of effort involved in doing this, it is easiest to use an installed OS to test the mainline kernel.

Introduction

By default, Ubuntu systems run with the Ubuntu kernels provided by the Ubuntu repositories. However it is handy to be able to test with unmodified upstream kernels to help locate problems in Ubuntu kernel patches, or to confirm that upstream has fixed a specific issue. To this end we now offer select upstream kernel builds. These kernels are made from unmodified kernel source but using the Ubuntu kernel configuration files. These are then packaged as Ubuntu .deb files for simple installation, saving you the time of compiling kernels, and debugging build issues.

These kernels are not supported and are not appropriate for production use.

How do I install an upstream kernel?

Following these steps in order will help you successfully test an upstream kernel.

Prepare OS to install an upstream kernel

First, if one is using select proprietary or out-of-tree modules (e.g. bcmwl, fglrx, NVIDIA proprietary graphics drivers, VitualBox, etc.) unless there is an extra package available for the version you are testing, you will need to uninstall the module first, in order to test the mainline kernel. If you do not uninstall these modules first, then the upstream kernel may fail to install, or boot.

Choose the proper upstream kernel files

The build directories are nicely organized into per architecture groups. For example, if one is using a 64-bit/amd64 architecture and wants the generic kernel version you would want those files marked A, from the appropriate group.
If you want the low latency version, B.

  Build for amd64 succeeded (see BUILD.LOG.amd64):
AB  linux-headers-4.19.0-041900_4.19.0-041900.201810221809_all.deb
A   linux-headers-4.19.0-041900-generic_4.19.0-041900.201810221809_amd64.deb
B   linux-headers-4.19.0-041900-lowlatency_4.19.0-041900.201810221809_amd64.deb
A   linux-image-unsigned-4.19.0-041900-generic_4.19.0-041900.201810221809_amd64.deb
B   linux-image-unsigned-4.19.0-041900-lowlatency_4.19.0-041900.201810221809_amd64.deb
A   linux-modules-4.19.0-041900-generic_4.19.0-041900.201810221809_amd64.deb
B   linux-modules-4.19.0-041900-lowlatency_4.19.0-041900.201810221809_amd64.deb

Download upstream kernel files from the Ubuntu archive

The upstream kernels archive has a directory for each build. Note, if you are testing for a bug, please do not use the daily folder, but use the latest mainline kernel at the top from:

It is best to verify the integrity of downloaded packages as explained below under "Verifying the mainline build binaries".

Install all upstream kernel files

Next, one will execute the following command against each of the downloaded files via a terminal:

sudo dpkg -i FILENAME.deb

If no errors show up, reboot while holding Shift then select "Advanced options for Ubuntu", then select and boot into the new entry that looks something like:

Ubuntu with linux 4.19.0-041900-generic

Problems installing upstream kernels

Virtualbox installed

Issues can occur installing an upstream kernel due to Virtualbox being installed. For example:

Error! Bad return status for module build on kernel: 3.7.0-030700rc2-generic (x86_64)
Consult /var/lib/dkms/virtualbox/4.1.18/build/make.log for more information.

As per above, you need to either install the extra package, if available, or uninstall virtualbox.

Version of package needed by installer is too old

A failure to install can occur due to the OS having packages that are too old for the install to proceed. For example:

depends on libssl1.1 (>= 1.1.0); however: Package libssl1.1 is not installed.

If this is due the version of libssl1.1 installed is too old then you need to upgrade your OS. However, if libssl1.1 is not installed at all, and the version that comes with your release is new enough, then install libssl1.1.

Other install errors

If for some reason the kernel you attempted to build failed, and it's not due to the above, then continue to test the next most recent kernel version until you can test to the issue.

Uninstalling upstream kernels

The upstream kernels have their own ABI namespace, so they install side by side with the stock Ubuntu kernels (each kernel has a separate directory under /lib/modules/VERSION for example). This means that you can keep several mainline and Ubuntu stock kernels installed at the same time and select the one you need from the GRUB boot menu.

If you would like to uninstall an upstream kernel anyway, execute the following to find the exact name of the kernel packages you want to uninstall:

dpkg -l | grep "linux\-[a-z]*\-"

and then execute the following to uninstall them:

sudo apt-get remove KERNEL_PACKAGES_TO_REMOVE

Remember that several packages can belong to one kernel version: common headers, architecture specific headers and the architecture specific image.

Also, once the mainline packages are removed, one may still see entries for these via the above dpkg command. To purge these entries execute at a terminal:

sudo dpkg --purge ENTRY

Mainline build tool chain

These kernels are built with the tool chain (gcc etc.) from the previous LTS (Ubuntu 8.04/10.04/12.04) depending on version. Therefore, out-of-tree kernel modules built with tools from other versions likely will not work.

Mainline kernel mapping to Ubuntu kernel

The Ubuntu kernel is not bit-for-bit the same as the mainline. However, one may find the upstream release that the Ubuntu kernel is based on via the Ubuntu to mainline mapping table.

Does the kernel team support the mainline kernel builds?

The mainline kernel builds are produced for debugging purposes and therefore come with no support. Use them at your own risk.

Where can I get the source for these builds?

In each directory there is a COMMIT file which defines the base commit in Linus' master tree from which they were built. The patches in the same directory ????-* are applied on top of this commit to make the build tree. A mirror of Linus' tree is available from git://kernel.ubuntu.com/virgin/linux.git.

First download the COMMIT and patch files ????-* from the mainline build in question to a temporary directory:

git clone git://kernel.ubuntu.com/virgin/linux.git mainline
cd mainline
git checkout -b `cat ${MAINLINE}/COMMIT`
git am ${MAINLINE}/????-*

Verifying the mainline build binaries

In order to allow verification that the published builds are the builds made by the mainline build system, the individual files are checksummed and the results of that published as CHECKSUMS in the same directory. This file is in turn signed by the mainline builder using the GPG key below which can be obtained from the Ubuntu Keyserver:

pub   2048R/17C622B0 2008-05-01
      Key fingerprint = 60AA 7B6F 3043 4AE6 8E56  9963 E50C 6A09 17C6 22B0
uid                  Kernel PPA <kernel-ppa@canonical.com>

The verification can be done by running the following commands:

  1. Import the above public key to your keyring (if you haven't already done that):
    $ gpg --keyserver hkps://pgp.mit.edu --recv-key "60AA7B6F30434AE68E569963E50C6A0917C622B0"
  2. Download the CHECKSUMS and CHECKSUMS.gpg files from the build directory and verify if the CHECKSUMS is signed with the above key:
    $ gpg --verify CHECKSUMS.gpg CHECKSUMS
    gpg: Signature made .... using RSA key ID 17C622B0
    gpg: Good signature from "Kernel PPA <kernel-ppa@canonical.com>"
    gpg: WARNING: This key is not certified with a trusted signature!
    gpg:          There is no indication that the signature belongs to the owner.
  3. Verify the checksums of downloaded deb files:
    $ shasum -c CHECKSUMS 2>&1 | grep 'OK$'
    You should get a line ending with "OK" for each of downloaded deb file and each type of checksums that are given in the CHECKSUMS file.

Upstream kernels in detail

We currently build five sets of upstream kernels. All formal tags from Linus' tree and from the stable trees, plus:

  1. the daily tip of Linus' linux kernel source tree,

  2. the tip of the drm-next head of Dave Airlie's linux repository daily,

  3. the tip of the drm-intel-next head of Keith Packard's linux repository daily until 2012, after which it has been taken over by Daniel Vetter at http://cgit.freedesktop.org/drm-intel/, and in particular, the drm-intel-next branch,

  4. the tip of the master branch of the debloat-testing tree daily,
  5. tags from the combined v2.6.32.x.y tree (by StefanBader) which is v2.6.32.x with DRM from 2.6.33.y.

This makes these kernels closer to the Lucid kernels which are based on 2.6.32 kernels with DRM backported from the 2.6.33 series.

The tagged releases (as made by Linus and the stable maintainers) are found under a directory matching their tag name and which kernel configuration they were built with (<tag>-<series>).

Daily tip of the tree builds are found in the daily sub-directory named for the date they were made.

Each build directory contains the header and image .deb files for the generic flavour i386 and amd64 architectures, as well lowlatency.

Can I install and then immediately use a kernel in a live environment?

No. One has two choices to use a mainline kernel:

  1. Install the mainline kernel in an installed environment, restart, and choose this newly installed kernel.
  2. Build a live environment with the new kernel in it. Given the amount of effort involved in doing this, it is easiest to use an installed OS to test the mainline kernel.

Kernel/MainlineBuilds (last edited 2021-10-15 16:20:16 by roguescholar)