GitKernelBuild

Differences between revisions 1 and 42 (spanning 41 versions)
Revision 1 as of 2007-12-13 21:30:18
Size: 5288
Editor: c-67-160-148-53
Comment: Add GitKernelBuild Wiki and document upstream bugzilla bug submission process.
Revision 42 as of 2012-11-23 10:13:23
Size: 7643
Editor: anthonywong
Comment: point the guildelines of reporting upstream bugs to the correct link
Deletions are marked like this. Additions are marked like this.
Line 1: Line 1:
||<tablestyle="float:right; font-size: 0.9em; width:40%; background:#F1F1ED; margin: 0 0 1em 1em;" style="padding:0.5em;"><<TableOfContents>>||

Also refer to https://help.ubuntu.com/community/Kernel/Compile for building Ubuntu kernels.
Line 3: Line 7:
The following document should help users build their own kernel from the upstream mainline kernel to help verify if a bug still exists or not. If a bug is still present in the upstream kernel, it is encouraged that the bug be reported to http://bugzilla.kernel.org . The bug submission process for http://bugzilla.kernel.org will be outlined at the end of the document. Please note that the following steps are targeted towards Ubuntu users and will focus on building the mainline kernel from the git repository at http://git.kernel.org . The following document should help users build their own kernel from the upstream mainline kernel to help verify if a bug still exists or not. If a bug is still present in the upstream kernel, it is encouraged that the bug be reported to http://bugzilla.kernel.org. The bug submission process is outlined at
[[KernelTeam/KernelTeamBugPolicies#Overview_on_Reporting_Bugs_Upstream|this page]]. Please note that the following steps are targeted towards Ubuntu users and focuses on building the mainline kernel from the git repository at
http://git.kernel.org.
Line 7: Line 12:
There are a few tools that are necessary in order to build your own kernel(s). The 'git-core' package provides the git revision control system which will be used to clone the mainline git repository. The 'kernel-package' provides the make-kpkg utility which automatically build your kernel and generate the linux-image and linux-header .deb files which will get installed. Please install both of these packages. There are a few tools that are necessary in order to build your own kernel(s). The 'git-core' package provides the git revision control system which will be used to clone the mainline git repository. The 'kernel-package' provides the make-kpkg utility which automatically build your kernel and generate the linux-image and linux-header .deb files which can be installed. You will need to install both of these packages.
Line 10: Line 15:
sudo apt-get install git-core kernel-package sudo apt-get install git-core kernel-package fakeroot build-essential ncurses-dev
Line 13: Line 18:
Note that, for newer versions of Ubuntu, you would replace "ncurses-dev" with "libncurses5-dev".
Line 14: Line 21:
 1. Change to the directory where you want to clone the git tree. In this example we will use /usr/src
  {{{ cd /usr/src
 1. Change to the directory where you want to clone the git tree. In this example we will use $HOME
  {{{
cd $HOME
Line 18: Line 26:
  {{{ sudo git clone git://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux-2.6.git   {{{
git clone git://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git
Line 20: Line 29:
 1. Change to linux-2.6 directory.
  {{{ cd linux-2.6
 1. Change to linux directory.
  {{{
cd linux
Line 24: Line 34:
  {{{ sudo cp /boot/config-`uname -r`.config .   {{{
cp /boot/config-`uname -r` .config
Line 26: Line 37:
 1. Bring the config file up to date.  (Answer any questions that get prompted).
  {{{ sudo make oldconfig
 1. Bring the config file up to date. Answer any questions that get prompted. Unless you know you are interested in a particular feature, accepting the default option by pressing Enter should be a safe choice.
  {{{
make oldconfig
Line 29: Line 41:
 In cases where your kernel source is significantly newer than the existing config file, you'll be presented with all of the new config options for which there is no existing config file setting. You can either sit there and keep hitting Enter to take the default (generally safe), or you can just run
  {{{
yes '' | make oldconfig
}}}
 which emulates exactly the same thing and saves you all that time.
Line 30: Line 48:
  {{{ sudo make menuconfig   {{{
make menuconfig
Line 32: Line 51:
 Note that Ubuntu kernels build with debugging information on, which makes the resulting kernel modules (*.ko files) much larger than they would otherwise be (linux-image*.deb will be 200-300 MB instead of 20-30 MB). To turn this off, go into "Kernel hacking"; then, under "Kernel debugging", turn OFF "Compile the kernel with debug info". Cf. [[https://bugs.launchpad.net/ubuntu/+source/kernel-package/+bug/90283|bug 90283]]
 1. Edit scripts/setlocalversion and comment out the line "echo "+"". If this is not done, make-kpkg will fail to build the packages with an error "package linux-image-[kernel version]-custom+ not in control info".
  {{{
sed -rie 's/echo "\+"/#echo "\+"/' scripts/setlocalversion
}}}
 In addition, if you want to build from a git non-HEAD version, also comment out the lines extra plus at about line 168:
  {{{
- if test "${LOCALVERSION+set}" != "set"; then
- scm=$(scm_version --short)
- res="$res${scm:++}"
- fi
+ #if test "${LOCALVERSION+set}" != "set"; then
+ # scm=$(scm_version --short)
+ # res="$res${scm:++}"
+ #fi
}}}
 Cf. [[https://bugs.launchpad.net/ubuntu/+source/kernel-package/+bug/58307/comments/25|bug 58307]]
Line 33: Line 70:
  {{{ sudo make-kpkg clean   {{{
make-kpkg clean
Line 35: Line 73:
 1. Build the linux-image and linux-header .deb files.
  {{{ sudo fakeroot make-kpkg --initrd --append-to-version=-custom kernel_image kernel_headers
 1. Build the linux-image and linux-header .deb files (CONCURRENCY_LEVEL can also be set manually to how many CPUs/cores to use to build the kernel). This process takes a lot of time.
  {{{
CONCURRENCY_LEVEL=`getconf _NPROCESSORS_ONLN`
fakeroot make-kpkg --initrd --append-to-version=-custom kernel_image kernel_headers
Line 38: Line 77:
 With this command the package names will be something like linux-image-2.6.24-rc5-custom and linux-headers-2.6.24-rc5-custom, and in that case the version will be 2.6.24-rc5-custom-10.00.Custom. You may change the string custom into something else by changing the `--append-to-version` option. You may also change the default 10.00.Custom into something else by using the `--revision` option.
Line 39: Line 79:
  {{{ cd ..   {{{
cd ..
Line 41: Line 82:
 1. Now install the .deb files. In this example the files are linux-image-2.6.24-rc5-custom_2.6.24-rc5-custom-10.00.Custom_i386.deb and linux-headers-2.6.24-rc5-custom_2.6.24-rc5-custom-10.00.Custom_i386.deb and
  {{{ sudo dpkg -i linux-image-2.6.24-rc5-custom_2.6.24-rc5-custom-10.00.Custom_i386.deb
 1. Now install the .deb files. In this example, the files are linux-image-2.6.24-rc5-custom_2.6.24-rc5-custom-10.00.Custom_i386.deb and linux-headers-2.6.24-rc5-custom_2.6.24-rc5-custom-10.00.Custom_i386.deb. You may receive warnings about '/lib/firmware/2.6.24-rc5-custom/' - this is expected and will only be problematic if the driver you are trying to test requires firmware.
  {{{
sudo dpkg -i linux-image-2.6.24-rc5-custom_2.6.24-rc5-custom-10.00.Custom_i386.deb
Line 45: Line 87:
 1. You are now ready to boot into your new kernel. Just to make sure you select the new kernel when you boot.
  {{{ sudo reboot
 1. You now need to make the new kernel bootable.
  Get your module names.
  {{{
cd /lib/modules
ls
}}}
  Update.
  {{{
sudo update-initramfs -ck module-name-for-new-kernel (in our case "update-initramfs -ck 2.6.24-rc5-custom_2.6.24-rc5-custom-10.00")
sudo update-grub
}}}
 1. You are now ready to boot into your new kernel. Just make sure you select the new kernel when you boot.
  {{{
sudo reboot
Line 49: Line 103:
== Reporting Bugs Upstream ==
Unfortunately it is sometimes the case that bugs can still exist in the mainline kernel. The mainline kernel has their own bugtracking system at http://bugzilla.kernel.org . It would be helpful if you would also report your bug there. It is often the case that once a bug is escalated upstream there is a quick resolution through the help and support of the mainline kernel community. Bug reports in Launchpad can also be set up to monitor bugs reported in other bugtracking systems. The following steps should help you report your bug upstream and monitor it in Launchpad:
== Using Ubuntu Kernel Configuration ==
The basic instructions provided above work well if you are building your own custom kernel. However, if you want to build a kernel that matches the official Ubuntu kernel package configuration as much as possible a few extra steps are needed. Note that if you are simply trying to build the ubuntu kernel, you should be following the https://help.ubuntu.com/community/Kernel/Compile guide instead of this one.
Line 52: Line 106:
 1. Go to http://bugzilla.kernel.org
 1. Verify your bug does not already exist in the upstream bugtracking system.
  i. If it does exist, please add any useful information that may be lacking in the current report.
   * Link your Launchpad report to the upstream kernel bug report (see step 5).
  i. If your bug does not exist, proceed to the next step.
 1. Enter your bug report at http://bugzilla.kernel.org/enter_bug.cgi . Follow the instructions on that page.
 1. When creating your bug report be sure to _attach_ any relevant information. This will usually include:
  i. dmesg output.
  i. lspci -vvnn output.
  i. If the bug is about interrupts, then /proc/interrupts is also helpful.
  i. If a BIOS issue is suspected, dmidecode output is also helpful.
  i. If the bug is regarding memory oddities, /proc/mtrr info is also helpful.
  i. Most importantly, if this is a regression, the best information you can provide the kernel developers is the information from doing a git bisect. This will hopefully narrow down the exact patch that is causing the regression.
   * http://www.reactivated.net/weblog/archives/2006/01/using-git-bisect-to-find-buggy-kernel-patches/
 1. After you've reported your bug to the kernel bugzilla, remember to set up your Launchpad bug report to monitor the upstream bugzilla report.
  i. On your Launchpad bug report page, look in the "Actions" area on the left hand side of the page.
  i. Click the "Also affects project" link.
  i. Enter the URL of your bugzilla report.
  i. Click the "Add to Bug Report" button.
  i. Your Launchpad report is not monitoring your upstream bugzilla report.
 1. Perform steps 1-7 above, use the Ubuntu kernel config in step 4
 2. Override the kernel-package default packaging scripts with the Ubuntu packaging scripts:
  {{{
cd $HOME
git clone git://kernel.ubuntu.com/ubuntu/ubuntu-lucid.git
cp -a /usr/share/kernel-package ubuntu-package
cp ubuntu-lucid/debian/control-scripts/{postinst,postrm,preinst,prerm} ubuntu-package/pkg/image/
cp ubuntu-lucid/debian/control-scripts/headers-postinst ubuntu-package/pkg/headers/
}}}
 3. Build packages using overlay directory:
  {{{
cd $HOME/linux
CONCURRENCY_LEVEL=`getconf _NPROCESSORS_ONLN` fakeroot make-kpkg --initrd --append-to-version=-custom --overlay-dir=$HOME/ubuntu-package kernel_image kernel_headers
}}}
 4. Perform steps 9-11 above

Note: The "--overlay-dir" option is only available in Lucid or later. If you need to build a kernel on a previous distribution, either install a backport of kernel-package if available, or manually edit /usr/share/kernel-package as needed.

Also refer to https://help.ubuntu.com/community/Kernel/Compile for building Ubuntu kernels.

Many times patches for bugs are committed upstream but have yet to make their way down to the Ubuntu kernel. It is often helpful if users are able to verify if the upstream patches do indeed resolve the issue they are witnessing. Likewise, in the opposite situation, it is useful to know if a bug may still exist upstream.

The following document should help users build their own kernel from the upstream mainline kernel to help verify if a bug still exists or not. If a bug is still present in the upstream kernel, it is encouraged that the bug be reported to http://bugzilla.kernel.org. The bug submission process is outlined at this page. Please note that the following steps are targeted towards Ubuntu users and focuses on building the mainline kernel from the git repository at http://git.kernel.org.

Prerequisites

There are a few tools that are necessary in order to build your own kernel(s). The 'git-core' package provides the git revision control system which will be used to clone the mainline git repository. The 'kernel-package' provides the make-kpkg utility which automatically build your kernel and generate the linux-image and linux-header .deb files which can be installed. You will need to install both of these packages.

sudo apt-get install git-core kernel-package fakeroot build-essential ncurses-dev

Note that, for newer versions of Ubuntu, you would replace "ncurses-dev" with "libncurses5-dev".

Kernel Build and Installation

  1. Change to the directory where you want to clone the git tree. In this example we will use $HOME
    • cd $HOME
  2. Clone the mainline kernel git tree.
    • git clone git://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git
  3. Change to linux directory.
    • cd linux
  4. Copy the kernel config file from your existing system to the kernel tree.
    • cp /boot/config-`uname -r` .config
  5. Bring the config file up to date. Answer any questions that get prompted. Unless you know you are interested in a particular feature, accepting the default option by pressing Enter should be a safe choice.
    • make oldconfig
    In cases where your kernel source is significantly newer than the existing config file, you'll be presented with all of the new config options for which there is no existing config file setting. You can either sit there and keep hitting Enter to take the default (generally safe), or you can just run
    • yes '' | make oldconfig
    which emulates exactly the same thing and saves you all that time.
  6. (optional) If you need to make any kernel config changes, do the following and save your changes when prompted:
    • make menuconfig

    Note that Ubuntu kernels build with debugging information on, which makes the resulting kernel modules (*.ko files) much larger than they would otherwise be (linux-image*.deb will be 200-300 MB instead of 20-30 MB). To turn this off, go into "Kernel hacking"; then, under "Kernel debugging", turn OFF "Compile the kernel with debug info". Cf. bug 90283

  7. Edit scripts/setlocalversion and comment out the line "echo "+"". If this is not done, make-kpkg will fail to build the packages with an error "package linux-image-[kernel version]-custom+ not in control info".
    • sed -rie 's/echo "\+"/#echo "\+"/' scripts/setlocalversion
    In addition, if you want to build from a git non-HEAD version, also comment out the lines extra plus at about line 168:
    • -       if test "${LOCALVERSION+set}" != "set"; then
      -               scm=$(scm_version --short)
      -               res="$res${scm:++}"
      -       fi
      +       #if test "${LOCALVERSION+set}" != "set"; then
      +       #       scm=$(scm_version --short)
      +       #       res="$res${scm:++}"
      +       #fi

    Cf. bug 58307

  8. Clean the kernel source directory.
    • make-kpkg clean
  9. Build the linux-image and linux-header .deb files (CONCURRENCY_LEVEL can also be set manually to how many CPUs/cores to use to build the kernel). This process takes a lot of time.
    • CONCURRENCY_LEVEL=`getconf _NPROCESSORS_ONLN` fakeroot make-kpkg --initrd --append-to-version=-custom kernel_image kernel_headers

    With this command the package names will be something like linux-image-2.6.24-rc5-custom and linux-headers-2.6.24-rc5-custom, and in that case the version will be 2.6.24-rc5-custom-10.00.Custom. You may change the string custom into something else by changing the --append-to-version option. You may also change the default 10.00.Custom into something else by using the --revision option.

  10. Change to one directory level up (this is where the linux-image and linux-header .deb files were put)
    • cd ..
  11. Now install the .deb files. In this example, the files are linux-image-2.6.24-rc5-custom_2.6.24-rc5-custom-10.00.Custom_i386.deb and linux-headers-2.6.24-rc5-custom_2.6.24-rc5-custom-10.00.Custom_i386.deb. You may receive warnings about '/lib/firmware/2.6.24-rc5-custom/' - this is expected and will only be problematic if the driver you are trying to test requires firmware.
    • sudo dpkg -i linux-image-2.6.24-rc5-custom_2.6.24-rc5-custom-10.00.Custom_i386.deb
      sudo dpkg -i linux-headers-2.6.24-rc5-custom_2.6.24-rc5-custom-10.00.Custom_i386.deb
  12. You now need to make the new kernel bootable.
    • Get your module names.
      cd /lib/modules
      ls
      Update.
      sudo update-initramfs -ck module-name-for-new-kernel (in our case "update-initramfs -ck 2.6.24-rc5-custom_2.6.24-rc5-custom-10.00")
      sudo update-grub
  13. You are now ready to boot into your new kernel. Just make sure you select the new kernel when you boot.
    • sudo reboot

Using Ubuntu Kernel Configuration

The basic instructions provided above work well if you are building your own custom kernel. However, if you want to build a kernel that matches the official Ubuntu kernel package configuration as much as possible a few extra steps are needed. Note that if you are simply trying to build the ubuntu kernel, you should be following the https://help.ubuntu.com/community/Kernel/Compile guide instead of this one.

  1. Perform steps 1-7 above, use the Ubuntu kernel config in step 4
  2. Override the kernel-package default packaging scripts with the Ubuntu packaging scripts:
    • cd $HOME
      git clone git://kernel.ubuntu.com/ubuntu/ubuntu-lucid.git
      cp -a /usr/share/kernel-package ubuntu-package
      cp ubuntu-lucid/debian/control-scripts/{postinst,postrm,preinst,prerm} ubuntu-package/pkg/image/
      cp ubuntu-lucid/debian/control-scripts/headers-postinst ubuntu-package/pkg/headers/
  3. Build packages using overlay directory:
    • cd $HOME/linux
      CONCURRENCY_LEVEL=`getconf _NPROCESSORS_ONLN` fakeroot make-kpkg --initrd --append-to-version=-custom --overlay-dir=$HOME/ubuntu-package kernel_image kernel_headers
  4. Perform steps 9-11 above

Note: The "--overlay-dir" option is only available in Lucid or later. If you need to build a kernel on a previous distribution, either install a backport of kernel-package if available, or manually edit /usr/share/kernel-package as needed.

KernelTeam/GitKernelBuild (last edited 2021-04-10 11:22:56 by pmaydell)