CVE Work Flow

CVEs in Ubuntu are represented via entries in the Ubuntu CVE tracker. Each active CVE is represented by a file in the active directory of tracker git repository.

CVE status is mirrored into the kernel CVE Matrix which provides the primary list of CVEs affecting all kernel packages. See "Kernel CVE Matrix" below for more details.

CVEs are also exposed as bugs against the linux packages in launchpad. There should be an open bug for each CVE tracker entry. These bugs are created by the Ubuntu security team as they create the CVEs. The bug numbers are listed in the CVE summary found by clicking through the CVE number in the kernel CVE matrix. See "CVE Bugs" below for more details.

Kernel CVE Work Flow

New CVEs are identified and added to the lp:ubuntu-cve-tracker git repo by the Ubuntu security team. They create both tracker entries and new CVE bugs. These then propagate to the kernel CVE matrix and to the kernel CVE bug list.

The kernel team is responsible for tracking and applying upstream fixes for these CVEs, where no fix is forthcoming the CVE will be pushed back to the Ubuntu security team for resolution.

Set up your local ubuntu-cve-tracker bzr repo

                .../ubuntu-cve-tracker/
                        branches/
                                trunk/
                                kernel-team/
                        work/

Ensure that bzr is configured to use your launchpad-login before creating your repo (or else you won't be able to push changes):

• bzr launchpad-login {yourlaunchpadlogin}

To create your local copy of the repo, the kernel-team branch, and a work branch, cut-n-paste this:

• bzr init-repo --format=pack-0.92 --no-trees ubuntu-cve-tracker
  cd ubuntu-cve-tracker
  mkdir branches
  bzr branch lp:ubuntu-cve-tracker branches/trunk
  bzr branch lp:~canonical-kernel-team/ubuntu-cve-tracker/kernel-team branches/kernel-team
  bzr checkout --lightweight branches/kernel-team work

  • (That use of --format=pack-0.92 avoids slow "Doing on-the-fly conversion from RemoteRepositoryFormat").

Your branch layout will look like this:

• .../ubuntu-cve-tracker$ ( cd work && bzr branches )
      * branches/kernel-team
        branches/trunk

ubuntu-cve-tracker triage session workflow

1. Update your trunk, kernel-team, and work branches before each work session:
   

   ...$ cd .../ubuntu-cve-tracker
   .../ubuntu-cve-tracker$  bzr pull -d branches/trunk
   .../ubuntu-cve-tracker$  bzr pull -d branches/kernel-team
   .../ubuntu-cve-tracker$ cd work
   .../work$  bzr update
   

   (Alternately, you can rm -r your work directory and checkout a fresh copy.)

2. Merge the kernel CVE autotriager repo into your work branch:
   

   .../work$  bzr merge --pull http://people.canonical.com/~kernel/cve-tracker/

     All changes applied successfully.
     Now on revision 9632.

3. Inspect the newly merged "autotriage" commit(s) for reasonableness.
   

   .../work$  bzr qlog .. &
   

   • (or)
     

   .../work$  bzr log -r submit:
   

4. Make other triage commits if desired (e.g. modify active/CVE-xxxx-xxxx files, as described below).
   

5. Push to the new commit(s) to the ~canonical-kernel-team repo
   

   .../work$  bzr push :parent
   

     Using saved push location: bzr+ssh://bazaar.launchpad.net/~canonical-kernel-team/ubuntu-cve-tracker/kernel-team/
     Pushed up to revision 9632.

Triage of CVEs

The first responsibility involves identifying any upstream commits which may resolve the issue. Where these are known at the time the CVE enters the system the Ubuntu security team will have added the break-fix: information to the tracker. It is common however for the fix to be known but not yet applied, for example they will supply a pointer to the upstream email discussions. When these resolve and we have upstream commits identified the kernel team adds these to the tracker, see "Break Fix Notation" below. If the upstream discussion does not result in upstream commits the CVE should be referred back to the Ubuntu security team.

Entries which do not yet have upstream commits identified will show up as needs-triage in the matrix. Once the break-fix: information has been added the CVE matrix will update indicating which kernel branches need fixes applying. See "CVE Status" below for descriptions of the meanings of the various states.

Once triage is complete any updates to the Ubuntu CVE tracker should be pushed up to the kernel CVE updates branch for review and merging by the Ubuntu CVE team.

• bzr push lp:~canonical-kernel-team/cve-tracker

Fixing CVEs

Once we have identified the upstream fixes these fixes should be applied to the various affected branches. Branches marked needed require the fix, though it should be noted that rebase branches will receive theirs via their associated master branch. See "Rebase Branches" below for more details.

When taking ownership of a CVE you should assign the main tasks of the associated launchpad bug to yourself to prevent two people from trying to handle the same CVE. See "CVE Bugs" below.

The CVE fixes should be cherry-picked back to the appropriate branch where at all possible. Where that is not possible some backporting may be appropriate. In all cases the fix should be clearly identified with the upstream commit id in one of the forms below in the signed-of-by: section. See "CVE Patch Formatting" for more details:

• (cherry-picked from commit 1234567890123456789012345678901234567890)

or

• (backported from commit 1234567890123456789012345678901234567890)

Where a simple backport is not possible then it is appropriate to pass the CVE back to the Ubuntu security team for further review.

CVE Bug Status

Each CVE has a separate status for each affected package. Some statuses may also carry additional version information which tells us where a fix was allied. The statuses are interpreted as below:

needs-triage -- we have insufficient information to know whether the CVE affects this package, typically this is as the upstream fix has yet to be identified and until it is we do not know how far back the issue occurs.

needed -- the branch is affected and a fix has been identified, so far it has not been applied. Action is required.

pending (without version) -- the branch is affected and the fix has been identified and applied. Essentially the fix is Fix Committed in the git repositories but has not yet been uploaded.

pending (with version) -- the branch is affected and the fix has been identified and applied. The branch also has had a release cut such that a known version number has been associated with it.

not-affected (without version) -- the branch does not need a fix for this issue. For example an x86 only fix would not require fixing in an ARM only package.

not-affected (with version) -- in development releases prior to the first official release, any CVE fix will be closed not-affected with a version indicating that no 'released' version was affected. CVEs do not really apply in development until the official release date.

released (with version) -- the branch was affected but the fix has now been officially releases _and_ announced via a USN. The kernel team never places entries into this state, this occurs as part of the issuance of the USN.

Kernel CVE Matrix

The kernel CVE Matrix provides a simple tabulated view of the currently active CVEs against any kernel package. There is a row for each CVE showing the individual status for each branch. The CVE name links through to the main CVE tracker description which contains a link to the launchpad bug for that CVE. The matrix makes use of colour to identify those items needing work, and to indicate the priority of that work (the nearer to red the most important). The kernel team workflow aims to move those identified needs-triage and needed (yellow and orange) to pending (green).

When all of the CVE entries are dark green the whole row will be removed and the bug moves to retired.

The CVE matrix is generated live from the Ubuntu CVE tracker, the kernel CVE tracker updates branch (See "Triage of CVEs" above) and from the the git repositories representing the kernel packages. It is updated hourly at around 45 minutes past the hour.

CVE Bugs

The outstanding CVEs listed on the Kernel CVE matrix also have launchpad bugs associated with them. These bugs should be used when working on a specific CVE, you should assign at least the master branch tasks to yourself to identify who is working on that CVE. Once the CVE is successfully triaged the bug should only have open tasks where patches are required. See the CVE Bug List for a full list of outstanding CVEs.

Where a branch does not require a fix this may be noted by moving that tasks Invalid. If a task already has the fix committed (in master-next) from upstream, then set that task Fix Committed. Other branches may be left In Progress and the tasks will be closed automatically as the fixes are applied and uploaded.

Break Fix Notation

When triaging a fix you will identify one or more upstream fixes which resolve the CVE. These should be added to the Ubuntu CVE tracker via the break-fix: notation. A kernel CVE has a linux package stanza similar to the below:

• Patches_linux:
   break-fix: ef14f0c1578dce4b688726eb2603e50b62d6665a fa8b18edd752a8b4e9d1ee2cd615b82c93cf8bba
   break-fix: ef14f0c1578dce4b688726eb2603e50b62d6665a 093019cf1b18dd31b2c3b77acce4e000e2cbc9ce
  upstream_linux: needed
  hardy_linux: not-affected
  lucid_linux: pending
  maverick_linux: needed
  natty_linux: needed
  oneiric_linux: pending
  devel_linux: not-affected (3.2.0-9.16)

Each break-fix: line identifies a SHA1 pair, saying "If you have this first SHA1 then we need the second SHA1 applied". If there is no introducing commit you may simply use a '-' and the fix is unconditionally required.

After updating the tracker you should push your changes to the Kernel CVE tracker updates branch (See "Triage of CVEs" above for details).

Rebase Branches

Some branches receive the majority of their fixes from another branch. This occurs via a rebase of that branch onto the latest version of the parental branch. For example the Lucid EC2 branch is based off of the Lucid master branch. When fixes are applied they need only be applied to the parental branches, the associated branches will gain the fix indirectly in the next upload. This appears in the matrix as a pending status even though the fix was not yet applied directly to that branch.

For a list of rebase branches see XXX.

CVE Patch Formatting

It is very important that patches destined to fix CVEs are formatted correctly of particular importance is the maintenance of any upstream SHA1s. These allow the triager to confirm that the CVE fixes have indeed been applied to the appropriate branches. It is also very important that the CVE number and the CVE bug number is correctly identified. Normally this will be added at the bottom of the upstream commit message, looking similar to the example below:

• [...]
  Signed-off-by: David S. Miller <davem@example.net>
  
  (cherry picked from commit c237899d1f8c5bfcfc9d6204052e0e065827ff75)
  CVE-2011-3359
  BugLink: http://bugs.launchpad.net/bugs/905060
  Signed-off-by: Andy Whitcroft <apw@example.net>

The maint-modify-patch script in the kteam-tools repo will handle this with the correct formatting. Example usage is:

• maint-modify-patch --bugid=<bug number> --cve=<YYYY-nnnn> --sob=<alias> <patchfilename>

It is also vital that you correctly identify the versions to which a patch applies, as commonly the applier will neither know which branches are unaffected and which patches apply to each. Use of subjects similar to the below is recommended:

• Subject: [natty,natty/ti-omap4 CVE 1/1] inetpeer: reduce stack usage