StableKernelMaintenanceExample

Differences between revisions 4 and 5
Revision 4 as of 2010-01-27 13:36:40
Size: 12940
Editor: csurbhi
Comment:
Revision 5 as of 2010-01-27 14:20:57
Size: 12866
Editor: csurbhi
Comment:
Deletions are marked like this. Additions are marked like this.
Line 18: Line 18:
      git log --reverse --pretty=%s v2.6.31.9..v2.6.31.10 will give you a short listing of the patches with the subject lines only and the first subject line will be the one which was applied immediately after the tag 2.6.31.9 was applied.        3. Now that you have a listing of the patches, you need to get the actual patches.
Line 20: Line 21:
    whereas:
      git log --reverse --pretty=oneline v2.6.31.9..v2.6.31.10 will you give you the subject line and the commit id. In the next step, you can use this commit id for the last patch which will be application of the tag 2.6.31.10

3. Now that you have a listing of the patches, you need to get the actual patches. This can be done by: find out the commit id for the patch that applies 2.6.31.9 by doing a git log
and searching for 2.6.31.9. copy the commit id for this patch. We shall need it now to generate the patch set. Commit id for the tag 2.6.31.10 can be found from the step above

      git format-patch <commit-id-for-tag-2.6.31.9> .. <commit-id-of-tag-2.6.31.10> -o ./patches/
      git format-patch v2.6.31.9..v2.6.31.10 -o ./patches/
     
Line 51: Line 46:
     14), ABI Bumper - generally this patch adds/modifies/delete non static functions. Keep a watch on this patch. ( Confirm this by compiling the branch - see later)
     
Line 55: Line 49:
     While looking at the patches, watch for a patch that might cause the ABI to bump. Such a patch adds/modifies/deletes non static functions. Confirm this by compiling the branch (see later) We need to mention this in the review email.
Line 57: Line 52:
    Some tips for beginners:
    I. Look in kernel bugzilla for bugs which are mentioned in the patch description. Read the comments by other kernel developers to get more insight in the development of a patch. If the patch is a regression, look at the previous patch which introduced the regression. look at user comments.

   II.
Sometimes when you are reviewing the patchset in 2.6.31.10, the review of 2.6.31.11 may be over upstream. Look out for this. Look if any regression was introduced in 2.6.31.10 an is fixed in the forward releases. If so, withhold this patch and apply it along with the fix. Comment this in your review mail.

   III.
Look at launchpad for bugs which this patch might fix. Note this bug number and its link. Note that you could look at launchpad bugs with their associated upstream bug number too. Look at this: https://bugs.edge.launchpad.net/bugs/bugtrackers/linux-kernel-bugs and sort it on the upstream bug number. Then you could see if the upstream bug number is linked against a launchpad bug.
    Some tips:
   * Look in kernel bugzilla for bugs which are mentioned in the patch description. Read the comments by other kernel developers to get more insight in the development of a patch. If the patch is a regression, look at the previous patch which introduced the regression. look at user comments.
   * Sometimes when you are reviewing the patchset in 2.6.31.10 for the Ubuntu kernel, the upstream review of 2.6.31.11/12 may be over. Look out for this. Look if any regression was introduced in 2.6.31.10 and is fixed in the forward releases. If so, withhold this patch and apply it along with the fix. Comment this in your review mail.
   * Look at launchpad for bugs which this patch might fix. Note this bug number and its link. Note that you could look at launchpad bugs with their associated upstream bug number too. Have a look at https://bugs.edge.launchpad.net/bugs/bugtrackers/linux-kernel-bugs and sort it on the upstream bug number. Then you could see if the upstream bug number is linked against a launchpad bug.
Line 65: Line 58:
    Now that your personal review of the stable release patches is over, you need to send it across to other people on the ubuntu kernel list, so that they can review it and ack it or give further insight. Before doing this do the following:     Now that your personal review of the stable release patches is over, you need to send it across to other people on the Ubuntu kernel list, so that they can review it and ack it or give further insight. Before doing this do the following:
Line 68: Line 61:
   Make a similar bug that tracks the updates. This bug is called the "tracking bug" for an update. You should also mention what launchpad bug this update fixes. Also if you are reverting any patch in favour of an upstream patch, then mention this in the tracking bug on launchpad. However this you will find out only when you apply the patches. The patches can be applied only after the next step. So you could update this part later.    Make a similar bug that tracks the updates. This bug is called the "tracking bug" for an update. The tracking bug should list the patches that were introduced by the upstream stable kernel release. This should be the output of: 'git log --reverse --pretty=%s v2.6.31.9..v2.6.31.10'. You should also mention what launchpad bugs this update might fix. Also if you are reverting any patch in favor of an upstream patch, then mention this in the tracking bug on launchpad. However this you will be able to find out only when you apply the patches. The patches can be applied only after the next step. So you could update this part later.
Line 82: Line 75:
      * Trick: you could add the maint-modify-patch in your .bashrc or add the path to it in $PATH.
Line 83: Line 78:
        5) $tool-dir/maint-modify-patch -s *.patch
   Now open the patches and verify that the correct bug link is inserted and that all patches are signed by your name appropriately. Alternatively you could add the maint-modify-patch in your .bashrc or add the path to it in $PATH.
7. You now need to sign these patches with your name. This is also done using the maintools.
        1) $tool-dir/maint-modify-patch -s *.patch
Line 86: Line 81:
7. Besides the buglink above, you also should add the link to any bug on launchpad that the patch resolves in particular. Use the maintools to do so.    * Alternatively you could sign the patches when you acquire them. This could be done by git format-patch -s (mentioned in step 3)
        
   Now open the patches and verify that the correct bug link is inserted (should start with BugLink:) and that all patches are signed by your name appropriately.
   
8. Besides the buglink above, you also should add the link to any bug on launchpad that the patch resolves in particular. Use the maintools to do so. Again verify the BugLink:
Line 88: Line 87:
8. You now need to sign these patches with your name. This is done using the maintools and is already described above. Alternatively you could sign the patches when you aquire them. This could be done by git format-patch -s (mentioned in step 3)
Line 99: Line 97:
   would apply all the patches that you created using git format. f the above command fails, then most probably the reason is that you already had a similar patch in your ubuntu tree before. Do the following in this case. Identify why the patch is already present in your ubuntu tree. was it because of security or was it a [sauce] patch that enable hardware or one which was accepted upstream but not yet released as a part of stable updates ? Unless its a security patch, revert the existing patch in favour of the upstream patch. The idea is that you want the most updated patch in such a
case and the upstream patch is most likely to be that one. The security patch is however not reverted. You keep the security patch and skip the upstream patch. However you need to mention this in your review email. So, then reset the head of the tree to where it was before applying your patches:
   would apply all the patches that you created using git format. If the above command fails, then most probably the reason is that you already had a similar patch in your Ubuntu tree before. Identify why the patch was already present in your Ubuntu tree. Was it because of any security reasons or was it a [sauce] patch that enabled hardware or one which was accepted upstream but not yet released as a part of stable updates ? Unless its a security patch, revert the existing patch in favor of the upstream patch. The idea is that you want the most updated patch and the upstream patch is most likely to be that one. The security patch is however not reverted. You keep the already existing security patch and skip the upstream patch. However you need to mention this in your review email. Also mention this revert or skip in your tracking bug on launchpad. In this case, to apply the patches reset the head of the tree to where it was before applying your patches:

        git am --abort
Line 106: Line 105:
   When you revert a patch in favor of an upstream patch note this in the git message for revert -    When you revert a patch in favor of an upstream patch note this in the git message for revert - "this patch is reverted in favor of an upstream patch with commit id released in a stable update 2.6.31.<y> now again apply the patches in the stable tree: "
Line 108: Line 107:
        git revert -e -s <commit-id-that-you-want-to-revert> "this patch is reverted in favour of an upstream patch with commit id released in a stable update 2.6.31.<y> now again apply the patches in   the stable tree:         git revert -e -s <commit-id-that-you-want-to-revert>

   R
eapply the patches again:
Line 112: Line 113:
     * Some tricks - to avoid resetting the head and restarting the git am you could do the following: If applying fails for a security patch, you skip it with 'git am --skip', the next patch in the sequence gets applied. If one patch application fails and you need to revert a corresponding patch, then this can be done while the am failure is active. Immediately do the ''git revert''. After that you can apply the patch which is stored in .git/rebase-apply/patch using 'patch -p1', then do a 'git add' to add the modified files (git status helps to find those) and if all is looking good, then do a 'git am --resolved'( This also continues the 'git am' sequence )      * Some tricks - to avoid resetting the head and restarting the git am you could do the following: If applying fails for a security patch, you skip it with 'git am --skip', the next patch in the sequence gets applied. If one patch application fails and you need to revert a corresponding patch, then this can be done while the am failure is active. Immediately do the 'git revert'. After that you can apply the patch which is stored in '.git/rebase-apply/patch' using 'patch -p1', then do a 'git add' to add the modified files (git status helps to find those) and if all is looking good, then do a 'git am --resolved'( This also continues the 'git am' sequence )
Line 114: Line 115:
10. Now compile the branch with all these patches applied. (You can compile you kernel using Stefan's build-scripts for remote compiling). See if any patch needs to bump the ABI. Rever that patch, see if the kernel compiles. If so, then you have identified which patch needs to bump the abi. Note this patch. Mention this in your review email. The ABI bump can be done as follows: 10. Now compile the branch with all these patches applied. (You can compile you kernel using Stefan's build-scripts for remote compiling). See if any patch needs to bump the ABI. Revert that patch, see if the kernel compiles. If so, then you have identified which patch needs to bump the ABI. Note this patch. Mention this patch and ABI bump, in your review email. The ABI bump can be done as follows:
Line 131: Line 132:
11. Now that your patches are applied, you need to either commit these changes if you have the permission or post them to your public repository. As an example: if you have a branch name stable-2.6.31.10 and you want to push it on kernel.ubuntu.com, then one trick to do the push in a quick and easier manner is as follows
    1. git remote add zinc-karmic surbhi@kernel.ubuntu.com:/srv/kernel.ubuntu.com/git/surbhi/ubuntu-karmic.git (do this once)
11. Now that your patches are applied, you need to either commit these changes if you have the permission or post them to your public repository. As an example: if you have a branch name stable-2.6.31.10 and you want to push it on kernel.ubuntu.com, then one trick to do the push in a quick and easier manner is described below with an example.
Say you always want to push your repositories to kernel.ubuntu.com:/srv/kernel.ubuntu.com/git/<username>ubuntu-karmic.git and say you want to push a branch named stable-2.6.31.10 then do this:
 
    1.
git remote add zinc-karmic <username>@kernel.ubuntu.com:/srv/kernel.ubuntu.com/git/<username>/ubuntu-karmic.git (do this once)
Line 136: Line 139:
12. Now write an email to kernel-team@lists.ubuntu.com indicating the tracking bug number and the sru update number for your ubuntu kernel and release so and so. Indicate what bugs on launchpad this update might fix. Also indicate the location to the public repository where your tree is kept. Also indicate the location of the upstream changelog and patchset for this stable update. This shall generally be: 12. Now write an email to kernel-team@lists.ubuntu.com indicating the tracking bug number and the SRU update number for your Ubuntu kernel and release so and so. Indicate what bugs on launchpad this update might fix. Also indicate the location to the public repository where your branch is kept. Also indicate the location of the upstream changelog and patchset for this stable update. This shall generally be:

This wiki describes the Ubuntu Kernel Stable Release Updates Maintenance Procedure with an example

The following steps in brief shall help in SRU maintenance:

Say you are releasing a stable update for 2.6.31.10

1. Get the source of the upstream stable releases

  • git clone git://git.kernel.org/pub/scm/linux/kernel/git/stable/linux-2.6.31.y.git

2. See what has changed since the last release. Since, in this document we talk about the 2.6.31.10 ubuntu SRU kernel we do the following:

  • git log 2.6.31.9..2.6.31.10
  • 2.6.31.9 and 2.6.31.10 are tags applied to the git tree. We are trying to find the git log between these tags.

3. Now that you have a listing of the patches, you need to get the actual patches.

  • git format-patch v2.6.31.9..v2.6.31.10 -o ./patches/
  • You can of course use any convenient location to store your patches generated. These patches will be numbered as 0001* and so on.

4. Now start looking at the patch from 0001* .. look at the code and review it. You could use the following as a guideline for your review:

  1. why is the patch applied ? b. could you categorize it as follows: (please change/add any other category you feel more appropriate)
  • 1) Memory corruption: is the patch applied because something else was writing to a memory location unintentionally? Was it fixing something that was writing to a wrong location? Or was it fixing something wrong (variable) writing to the right location?
    2) Userspace interaction: Does the patch fix what was seen by userspace? Were wrong values being passed ? Was the kernel data unintentionally exposed?
    3) Ooops- Thats a straight one.. was the kernel oops fixed. Generally this will also have some bugzilla link or a launchpad link
    4) Enablement: Was this patch enabling some hardware? Was this patch fixing some wrong values in a driver code which leads to the correct functioning of a feature in a driver?
    5) Error Handling: Was this patch fixing the return values (eg unsigned to signed) or was freeing the memory/ releasing locks before returning?
    6) Fixup: All patches are some fixup :D But was this patch fixing a wrong logic? (eg: accidental exchange of variables..leading to a wrong result)
    7) Regression: Was this patch a bandaid for some patch released before? Often a feature addition or a bug fix introduces a bug. If this patch is a fixing such a bug then this is a regression patch.
    8) Correctness: Was this patch correcting some initial values used for the driver?
    9) Data Loss: Was this patch dealing with data loss because of wrong memory length or other such reason?
    10) Locking: was this patch dealing with addition or deletion of locks in a code which did not consider locking before? This could also be coupled with the error handling category in which the patch corrects error handling and in doing so releases memory or locks.
    11) Enhancement: Does this patch beautify/enhance some code? This patch could be offering the same functionality but making the code more readable/robust.
    12) Data corruption: Was this patch dealing with wrong register values being written accidentally ? Was this patch dealing with filesystem corrupting the data in files on disk ?
    13) Quirk: Is this patch adding a simple vendor id related structure in some driver to enable it too ? This patch can be categorized as a Quirk and Enablement.
    • While looking at the patches, watch for a patch that might cause the ABI to bump. Such a patch adds/modifies/deletes non static functions. Confirm this by compiling the branch (see later) We need to mention this in the review email.


  • Some tips:
  • Look in kernel bugzilla for bugs which are mentioned in the patch description. Read the comments by other kernel developers to get more insight in the development of a patch. If the patch is a regression, look at the previous patch which introduced the regression. look at user comments.
  • Sometimes when you are reviewing the patchset in 2.6.31.10 for the Ubuntu kernel, the upstream review of 2.6.31.11/12 may be over. Look out for this. Look if any regression was introduced in 2.6.31.10 and is fixed in the forward releases. If so, withhold this patch and apply it along with the fix. Comment this in your review mail.
  • Look at launchpad for bugs which this patch might fix. Note this bug number and its link. Note that you could look at launchpad bugs with their associated upstream bug number too. Have a look at https://bugs.edge.launchpad.net/bugs/bugtrackers/linux-kernel-bugs and sort it on the upstream bug number. Then you could see if the upstream bug number is linked against a launchpad bug.


  • Now that your personal review of the stable release patches is over, you need to send it across to other people on the Ubuntu kernel list, so that they can review it and ack it or give further insight. Before doing this do the following:

5. Open a bug on Launchpad indicating that 2.6.31.10 (in our eg) has been released upstream and you will like to see these updates in ubuntu release-name( karmic in our eg). Look at an example tracking bug. I looked at the one Leann made. Here is a link: https://bugs.edge.launchpad.net/ubuntu/karmic/+source/linux/+bug/496816

  • Make a similar bug that tracks the updates. This bug is called the "tracking bug" for an update. The tracking bug should list the patches that were introduced by the upstream stable kernel release. This should be the output of: 'git log --reverse --pretty=%s v2.6.31.9..v2.6.31.10'. You should also mention what launchpad bugs this update might fix. Also if you are reverting any patch in favor of an upstream patch, then mention this in the tracking bug on launchpad. However this you will be able to find out only when you apply the patches. The patches can be applied only after the next step. So you could update this part later.

6. Now you need to include this bug link in all your patches. This indicates, that you are applying this patch to the ubuntu kernel for resolving this launchpad bug. This can be done by using the maintools written by Stefan. git://kernel.ubuntu.com/smb/maint-tools.git

  • You configure the maint-tools by looking at the maint-tools/doc/example-maintscript.cfg. Change the signer: field in this and mark your name there and make an alias with your name. Then put this modified maintscript.cfg in your ~/.maintscript.cfg Now execute the following:
    • 1) cd <dir-where-you-have-your-patches>

      2) tool-dir= <location-where-you-have-your-git-mainttools> 3) export tool-dir

      4) $tool-dir/maint-modify-patch -b <bugnumber> *.patch

    Note that you should not give a '=' before the bugnumber!
  • Trick: you could add the maint-modify-patch in your .bashrc or add the path to it in $PATH.

7. You now need to sign these patches with your name. This is also done using the maintools.

  • 1) $tool-dir/maint-modify-patch -s *.patch
  • Alternatively you could sign the patches when you acquire them. This could be done by git format-patch -s (mentioned in step 3)

    Now open the patches and verify that the correct bug link is inserted (should start with BugLink:) and that all patches are signed by your name appropriately.

8. Besides the buglink above, you also should add the link to any bug on launchpad that the patch resolves in particular. Use the maintools to do so. Again verify the BugLink:

9. Now you are ready to apply these patches. So create a new branch on which you will apply these patches.

  • git checkout master git pull git checkout -b stable-2.6.31.10

    git am <location where your patches are>/00*.patch

  • would apply all the patches that you created using git format. If the above command fails, then most probably the reason is that you already had a similar patch in your Ubuntu tree before. Identify why the patch was already present in your Ubuntu tree. Was it because of any security reasons or was it a [sauce] patch that enabled hardware or one which was accepted upstream but not yet released as a part of stable updates ? Unless its a security patch, revert the existing patch in favor of the upstream patch. The idea is that you want the most updated patch and the upstream patch is most likely to be that one. The security patch is however not reverted. You keep the already existing security patch and skip the upstream patch. However you need to mention this in your review email. Also mention this revert or skip in your tracking bug on launchpad. In this case, to apply the patches reset the head of the tree to where it was before applying your patches:
    • git am --abort

      git reset --hard <commit-id> now do the revert:

    When you revert a patch in favor of an upstream patch note this in the git message for revert - "this patch is reverted in favor of an upstream patch with commit id released in a stable update 2.6.31.<y> now again apply the patches in the stable tree: "

    • git revert -e -s <commit-id-that-you-want-to-revert>

    Reapply the patches again:
    • git am <location where your patches are>/00*.patch

    • Some tricks - to avoid resetting the head and restarting the git am you could do the following: If applying fails for a security patch, you skip it with 'git am --skip', the next patch in the sequence gets applied. If one patch application fails and you need to revert a corresponding patch, then this can be done while the am failure is active. Immediately do the 'git revert'. After that you can apply the patch which is stored in '.git/rebase-apply/patch' using 'patch -p1', then do a 'git add' to add the modified files (git status helps to find those) and if all is looking good, then do a 'git am --resolved'( This also continues the 'git am' sequence )

10. Now compile the branch with all these patches applied. (You can compile you kernel using Stefan's build-scripts for remote compiling). See if any patch needs to bump the ABI. Revert that patch, see if the kernel compiles. If so, then you have identified which patch needs to bump the ABI. Note this patch. Mention this patch and ABI bump, in your review email. The ABI bump can be done as follows:

  1. vim debian.master/changelog - increment the abi by 1. eg 2.6.31-18.56 would be 2.6.31-19.56
  2. touch debian.master/rules.d/control.stub.in
  3. fakeroot debian/rules clean
  4. git add debian.master/changelog
  5. git commit -s -F debian/commit-templates/bumpabi

11. Now that your patches are applied, you need to either commit these changes if you have the permission or post them to your public repository. As an example: if you have a branch name stable-2.6.31.10 and you want to push it on kernel.ubuntu.com, then one trick to do the push in a quick and easier manner is described below with an example. Say you always want to push your repositories to kernel.ubuntu.com:/srv/kernel.ubuntu.com/git/<username>ubuntu-karmic.git and say you want to push a branch named stable-2.6.31.10 then do this:

  1. git remote add zinc-karmic <username>@kernel.ubuntu.com:/srv/kernel.ubuntu.com/git/<username>/ubuntu-karmic.git (do this once)

  2. git push zinc-karmic +stable-2.6.31.10 (every time you want to synchronize your remote branch with your local branch, do this)

12. Now write an email to kernel-team@lists.ubuntu.com indicating the tracking bug number and the SRU update number for your Ubuntu kernel and release so and so. Indicate what bugs on launchpad this update might fix. Also indicate the location to the public repository where your branch is kept. Also indicate the location of the upstream changelog and patchset for this stable update. This shall generally be:

  • http://kernel.org/pub/linux/kernel/v2.6/ChangeLog-2.6.<x>.<y>

    http://kernel.org/pub/linux/kernel/v2.6/patch-2.6.<x>.<y>.bz2 and then append the review of those patches. The review message should be as follows: output of: git log --pretty=short v2.6.31.9..v2.6.31.10 after every patch description (from the above command), should be the following:

    Category: One line description of what the patch does. Whether any patch was reverted for applying this patch. Whether this patch causes regression, the fix of which is released in a forward stable release, due to which we shall apply the patch with that fix. Whether this patch is an ABI bumper.


Thats it !.. press the send button ! and you are good to go Smile :)

KernelTeam/StableKernelMaintenanceExample (last edited 2010-01-29 14:42:21 by p5B2E72FC)