This document is intended to document the processes involved in the maintenance of stable (after release) kernel trees. Those should be independent of a specific release. If not, this should be noted on the specific description.
Stable release updates will be done after the initial release. To prevent regressions there are a more formal process involved in getting new patches included. The process is described in general at StableReleaseUpdates with a kernel specific part described KernelUpdates (Note the proposal below).
Every SRU needs a bug. Patches are sent to kernel mailing list and have to get at least one ACK (maybe 2 with the current team size).
- SRU justification is added to bug and nomination for releases are used to track progress for each release (if appropriate).
- ubuntu-sru has to get subscribed
[TBD] milestone is set for ??? (patches that are important, or just generally if we think it should get into next release).
Proposal: Always add fixes from -stable tree.
Since the process for stable kernel updates (LWN article) is very similar to our SRU process and there have been quite a number of bugs that have been solved by finally finding out that there has been a fix in the stable tree, there has been a proposal to add those automatically to our stable tree (discussion was on kernel team mailing list). The updated process could look like this:
- Stable release updates from the kernel.org stable kernel that matches the kernel version will qualify as SRU updates. Possible import process could be:
- one LP bug for each release
- one LP bug for each patch of each release
Those updates also should be ACKed by at least one (two) members of the kernel team.
- Updates resulting from stable tree imports should be kept at least 2 weeks in -proposed before moving to -updates. Any change or regression in the stable tree resets the counter.
Current contact: Kees Cook
CVE updates usually are done in batches (except for very critical problems). There is a list of CVEs and all currently maintained releases have to be checked for possible vulnerabilities.
The patchsets are prepared by one member of the stable maintenance team but not pushed if any of the patches still is embargoed.
- test builds must be done at least for i386 and amd64
- test packages and upload sources are provided on an internal server (e.g. chinstrap).
- boot tests should be done for i386 (and amd64)
[TBD] verification of the vulnerability (if possible)
this would be good but requires automated tests
Proposed work flow for security updates with unreleased or pending -proposed updates:
- create a branch based on the last official (-updates/-security) release
- add fixes and do the release from this branch
- after release (and after gathering the ABI files) merge back to main tree.
This makes sure the version of the security release can be checked out and based upon.
Note: Had to use "git push origin master" to get merges visible.
Updates to custom binaries (virtual/openvz/...)
Custom binary builds are usually community maintained and not as critical as the main kernel flavours. Nevertheless updates should follow a certain pattern to be clearly documented.
- Get triggered by a community member (usually kernel-team mailing list)
- Open a LP bug for SRU documentation
- Create a branch and pull/cerry-pick
- Do a test build
- Publish the SRU request on the kernel-team list. References to patches as gitweb links since patches might be big.
Either in a PPA or on the porter machines. HPPA is a problem due to hardware access.
A new upload is prepared by editing/creating the changelog (KernelMaintenance)
- Use chroots to create the uploads (because scripts depend on certain environments)
The insertchanges target does not always work properly, especially on older releases. Always make sure to check the generated file.
- Are there duplicate/unwanted entries
- Has every entry a bug reference
Ubuntu patches (UBUNTU keyword prefixed) automatically this if they contained a Bug keyword in the description.
Patches from upstream did not (but do with updated scripts and if they include a Bug keyword.
- CVEs should have the CVE-xxxx-xxxx number in their description
- The changelog keeps order of uploads, git log the order of check ins but this cannot always be honoured. The make process cannot handle complicated forth and back of ABI releases.
On updates the command for generating the source package should include the -v option (and have the orig.tar.gz, to do only diff uploads)
dpkg-buildpackage -S -rfakeroot -I.git -I.gitignore -i'\.git.*' \ -v<version>
A note to <version>: this should be the last relevant version uploaded. So for uploads to -proposed this should be the last version uploaded to updates. Updates to security probably the same but most of the time those are both the same anyway.
The actual release
- The upload target differs, depending on the type of upload:
-security uploads go to security.ubuntu.com (jackass) while
-proposed uploads go to upload.ubuntu.com
The packages have to be signed by someone with the upload rights and the uploaded packages will be invisible until they are ACCEPTED
For -security uploads the contacts are kees or jdstrand. For those the status is even invisible after acceptance.
For -proposed uploads the packages have to be accepted by either slangasek or pitty (or cjwatson(?))
If the kABI changed note the reasoning on the stable kernel ABI wiki page.
- Propose tag for ABI changing checkings (BumpABI). This is manual but it allows a quick overview which changes caused ABI bumps.
- Somehow also keep track of the changes in supported hardware to gather this for releases.
As described in KernelMaintenance. Thoughts on that:
- The ABI files have to be fetched. Would it make sense to get them, then change the release tag to that state? Better would be to get those from test builds. Currently the only problem is HPPA (no host).
The creation of the new release uses whoami and stuff. Might be better to take DEBNAME and DEBEMAIL...