Ubuntu Wiki

This wiki describes the Ubuntu Kernel Stable Release Updates Maintenance Procedure with an example.

The following steps in brief shall help in SRU maintenance:

Say you are releasing a stable update for 2.6.31.10

1. Get the source of the upstream stable releases

    git clone git://git.kernel.org/pub/scm/linux/kernel/git/stable/linux-2.6.31.y.git

   Note: If you got a clone of Linus tree somewhere local, you should use --reference to minimize the space required for this new repository.

2. See what has changed since the last release. Since, in this document we talk about the 2.6.31.10 Ubuntu SRU kernel we do the following:

    git log v2.6.31.9..v2.6.31.10

   v2.6.31.9 and v2.6.31.10 are tags applied to the git tree. We are trying to find the git log between these tags. Should v2.6.31.10 be the latest update, it does not need to be specified. So the following is giving the same result:

    git log v2.6.31.9..

3. Now that you have a listing of the patches, you need to get the actual patches.

    git format-patch [-s] [-o <dir>] v2.6.31.9..v2.6.31.10

   The -s adds your Signed-off-by: to every patch that is generated and the -o lets you select the directory where those patches are stored. The default is the current directory. You can of course use any convenient location to store your patches generated. These patches will be numbered as 0001* and so on.

4. Now start looking at the patch from 0001* .. look at the code and review it. You could use the following as a guideline for your review:
   1. why is the patch applied ?
   2. could you categorize it as follows: (please change/add any other category you feel more appropriate)

      1)	Memory corruption: is the patch applied because something else was writing to a memory location unintentionally? Was it fixing something that was writing to a wrong location? Or was it fixing something wrong (variable) writing to the right location?
      2)	Userspace interaction: Does the patch fix what was seen by userspace? Were wrong values being passed ? Was the kernel data unintentionally exposed?
      3)	Ooops- Thats a straight one.. was the kernel oops fixed. Generally this will also have some bugzilla link or a launchpad link
      4)	Enablement: Was this patch enabling some hardware? Was this patch fixing some wrong values in a driver code which leads to the correct functioning of a feature in a driver?
      5)	Error Handling: Was this patch fixing the return values (eg unsigned to signed) or was freeing the memory/ releasing locks before returning?
      6)	Fixup: All patches are some fixup :D But was this patch fixing a wrong logic? (eg: accidental exchange of variables..leading to a wrong result)
      7)	Regression: Was this patch a bandaid for some patch released before? Often a feature addition or a bug fix introduces a bug. If this patch is a fixing such a bug then this is a regression patch.
      8)	Correctness: Was this patch correcting some initial values used for the driver?
      9)	Data Loss: Was this patch dealing with data loss because of wrong memory length or other such reason?
      10)	Locking: was this patch dealing with addition or deletion of locks in a code which did not consider locking before? This could also be coupled with the error handling category in which the patch corrects error handling and in doing so releases memory or locks.
      11)	Enhancement: Does this patch beautify/enhance some code? This patch could be offering the same functionality but making the code more readable/robust.
      12)	Data corruption: Was this patch dealing with wrong register values being written accidentally ? Was this patch dealing with filesystem corrupting the data in files on disk ?
      13)	Quirk: Is this patch adding a simple vendor id related structure in some driver to enable it too ? This patch can be categorized as a Quirk and Enablement.

   While looking at the patches, watch for a patch that might cause the ABI to bump. Such a patch adds/modifies/deletes non static functions. Confirm this by compiling the branch (see later) We need to mention this in the review email.

   Some tips:

   • Look in kernel bugzilla for bugs which are mentioned in the patch description. Read the comments by other kernel developers to get more insight in the development of a patch. If the patch is a regression, look at the previous patch which introduced the regression. look at user comments.
   • Sometimes when you are reviewing the patchset in 2.6.31.10 for the Ubuntu kernel, the upstream review of 2.6.31.11/12 may be over. Look out for this. Look if any regression was introduced in 2.6.31.10 and is fixed in the forward releases. If so, withhold this patch and apply it along with the fix. Comment this in your review mail.

   • Look at launchpad for bugs which this patch might fix. Note this bug number and its link. Note that you could look at launchpad bugs with their associated upstream bug number too. Have a look at https://bugs.edge.launchpad.net/bugs/bugtrackers/linux-kernel-bugs and sort it on the upstream bug number. Then you could see if the upstream bug number is linked against a launchpad bug.

5. Now that your personal review of the stable release patches is over, you need to send it across to other people on the Ubuntu kernel list, so that they can review it and ack it or give further insight. Before doing this do the following:

   Open a bug on Launchpad indicating that 2.6.31.10 (in our example) has been released upstream and you would like to see these updates in Ubuntu release-name (Karmic in this case). Look at an example tracking bug. I looked at the one Leann made. Here is a link: https://bugs.edge.launchpad.net/ubuntu/karmic/+source/linux/+bug/496816 Make a similar bug that tracks the updates. This bug is called the "tracking bug" for an update. The tracking bug should list the patches that were introduced by the upstream stable kernel release. This should be the output of:

    git log --reverse --pretty=%s v2.6.31.9..v2.6.31.10

   You should also mention what launchpad bugs this update might fix. Also if you are reverting any patch in favor of an upstream patch, then mention this in the tracking bug on launchpad. However this you will be able to find out only when you apply the patches. The patches can be applied only after the next step. So you could update this part later.

6. Now you need to include this bug link in all your patches. This indicates, that you are applying this patch to the Ubuntu kernel for resolving this launchpad bug. This can be done by using the maint-modify-patch written by Stefan (git://kernel.ubuntu.com/smb/maint-tools.git).

   You configure the maint-tools by looking at the maint-tools/doc/example-maintscript.cfg. Copy the file to $HOME/.maintscripts.cfg. Open the new file and create an alias with your name and replace the signer: field with your alias. Finally add the path to the script to the PATH variable in .bashrc to make it available from any place (this has to be done only once). Then

    cd <dir-where-you-have-your-patches>
    maint-modify-patch -b <bugnumber> *.patch

7. You now need to sign these patches with your name. This is also done using the maintools.

    maint-modify-patch -s *.patch

   Alternatively you could sign the patches when you acquire them (see git format-patch).

8. Now open the patches and verify that the correct bug link is inserted (should start with BugLink:) and that all patches are signed by your name appropriately.

9. Besides the buglink above, you also should add the link to any bug on launchpad that the patch resolves in particular. Use the maintools to do so. Again verify the BugLink:

10. Now you are ready to apply these patches. So create a new branch on which you will apply these patches.

     git fetch
     git checkout -b stable-2.6.31.10 origin/master
     git am <location where your patches are>/00*.patch

    This will apply all the patches that you created using git format. If the above command fails, then most probably the reason is that you already had a similar patch in your Ubuntu tree before. Identify why the patch was already present in your Ubuntu tree. Was it because of any security reasons or was it a [SAUCE] patch that enabled hardware or one which was accepted upstream but not yet released as a part of stable updates? Unless its a security patch, revert the existing patch in favor of the upstream patch. The idea is that you want the most updated patch and the upstream patch is most likely to be that one. The security patch is however not reverted. You keep the already existing security patch and skip the upstream patch. However you need to mention this in your review email. Also mention this revert or skip in your tracking bug on launchpad.

    1. If the patch has been applied by a security release, simply skip over that patch with

         git am --skip

    2. If we were carrying that patch as a [SAUCE] patch or have been pulled it to resolve a bug report, we revert the old patch and replace it with the upstream patch. First find the commit SHA-ID of the previously applied patch, then revert it with

         git revert -e -s <commit-id>

       When you revert a patch in favor of an upstream patch note this in the git message for revert - "This patch is reverted in favor of an upstream patch in stable update 2.6.31.<y>". When the revert is done, apply the patch from upstream. As the git am failed, it is still in .git/rebase-apply.

         patch -p1 <.git/rebase-apply/patch

       Add the files changed by the patch with git add (to find out which files have changed git status can be used) and then resume applying it (this also will proceed with the remaining patches) with

         git am --resolved

11. Now compile the branch with all these patches applied. (You can compile you kernel using Stefan's buildscripts for remote compiling). See if any patch needs to bump the ABI. Revert that patch, see if the kernel compiles. If so, then you have identified which patch needs to bump the ABI. Note this patch. Mention this patch and ABI bump, in your review email. The ABI bump can be done as follows:

    1. Up to Hardy (or before abstracted debian)

       Edit debian/changelog and increment the ABI number (e.g. 2.6.24.1.2 becomes 2.6.24.2.2). Then

         touch debian/control.stub.in
         fakeroot debian/rules clean
         git add debian
         git commit -s -F debian/commit-templates/bumabi

    2. For newer releases (with abstracted debian) Edit debian.master/changelog and increment the ABI by 1. (e.g. 2.6.31-18.56 would become 2.6.31-19.56). Then

         git add debian.master/changelog
         git commit -s -F debian/commit-templates/bumpabi
         fakeroot debian/rules clean

       The clean step here ensures that all generated files are updated.

12. Now that your patches are applied, you need to either commit these changes if you have the permission or post them to your public repository. As an example: if you have a branch name stable-2.6.31.10 and you want to push it to your public personal repository on http://kernel.ubuntu.com, then one trick to do the push in a quick and easier manner is described below with an example. Say you always want to push your repositories to kernel.ubuntu.com:/srv/kernel.ubuntu.com/git/<username>ubuntu-karmic.git and say you want to push a branch named stable-2.6.31.10 then do this:

     git remote add <local name> <username>@kernel.ubuntu.com:/srv/kernel.ubuntu.com/git/<username>/ubuntu-karmic.git

    You do this once and this creates a remote entry that allows you to refer to that repository by the name you choose (for example mykarmic). Then you can push your working branch to it with the following command:

     git push mykarmic +stable-2.6.31.10

    You do this every time you want to update your remote repository with your local branch. The + allows you to completely replace the current state of your remote branch. Without that, only fast-forward changes are allowed.

13. Now write an email to kernel-team@lists.ubuntu.com indicating the tracking bug number and the SRU update number for your Ubuntu kernel and release so and so. Indicate what bugs on launchpad this update might fix. Also indicate the location to the public repository where your branch is kept. Also indicate the location of the upstream changelog and patchset for this stable update. This shall generally be:

    • http://kernel.org/pub/linux/kernel/v2.6/ChangeLog-2.6.<x>.<y>

    • http://kernel.org/pub/linux/kernel/v2.6/patch-2.6.<x>.<y>.bz2

    and then append the review of those patches. The review message should contain the list of patches generated by the following command:

     git log --reverse --pretty=short v2.6.31.9..v2.6.31.10

    After every patch description, insert your comments. It helps other people to have then categorized, though that is not strictly required. The format is more or less free-form, just keep in mind that other people will read over those and want to understand/grasp the information quickly. An example comment section could be:

     Type (or Category): Memory corruption
     Limits the end condition of a loop to stay within array limits.

    In general the comment should be a one line description of what the patch does. Whether any patch was reverted for applying this patch. Whether this patch causes regression, the fix of which is released in a forward stable release, due to which we shall apply the patch with that fix. Whether this patch is an ABI bumper.

Thats it !.. press the send button ! and you are good to go