StableReleaseCadence

Differences between revisions 5 and 6
Revision 5 as of 2010-11-15 18:14:53
Size: 6300
Editor: user-69-73-1-154
Comment:
Revision 6 as of 2010-11-15 18:30:33
Size: 6387
Editor: user-69-73-1-154
Comment:
Deletions are marked like this. Additions are marked like this.
Line 81: Line 81:
 * At the end of the first week, any bug fixes which have not been verified as fixed by the reporter or by internal testing will have the patches reverted, and the associated bugs set to "incomplete". A tag of "stable-reverted-<release>" will be added at this time (where <release> is [lucid | maverick . . .]).  * At the end of the first week, any bug fixes which have not been verified as fixed by the reporter or by internal testing will have the patches reverted, and the associated bugs set to "incomplete". A tag of "stable-reverted-<release>" will be added at this time (where <release> is [lucid | maverick . . .]). A comment will be added to the bug describing how to request reapplication of the fix.

Stable Kernel Release Cadence

Goals

The Kernel Stable Cadence was arrived at during discussions before and during Natty Narwhal Developer Summit in Orlando. The Blueprint from the UDS sessions may be found here. The new process has a number of goals which are designed to improve some problems with the old process. These goals are:

  1. Reduce the maximum time between valid fixed are applied and verified and their delivery to the end user.
  2. Assure that every bug fix is verified to solve the problem while the kernel is in -proposed testing.
  3. Increase kernel quality by applying comprehensive regression and certification testing to all stable kernels before release.
  4. Reduce the time until non-critical and non-embargoed security fixes are applied and released.
  5. Reduce the impact that critical and embargoed security updates have on the normal stable release schedule.
  6. Reduce the amount of time it takes for upstream stable kernel updates to reach users.
  7. Avoid kernel updates which contain a very large number of updates.
  8. Provide more predictability of when a fix that's been applied to the source repository will be released.

Implementation

Whenever possible, Stable kernels will now be released on a two-week cadence. The cycle is two weeks long, and repeats every two weeks, with the exception of holidays, UDS weeks, and other considerations. An attempt will be made to synchronize the schedule with the the Ubuntu platform schedule.

Patches will be divided into the following classes:

Non-Security Patches

  • Stable Upstream Patches
    • Stable updates from upstream (GregKH's repos) will be accepted without testing of individual functionality.
  • Other Patches
    • No patch will be accepted for a bug if we don't have the hardware to test it or high confidence that it will be tested by the community. At the end of the first week of baking in -proposed, if these patches have not been verified as fixing the specific bugs the patch will be reverted.

Security (CVE) Patches

  • Non-Critical
    • Non-critical CVE patches will be applied directly to the master branch of the effected git repository and will be part of the next -proposed upload. A special -security upload will not be required for these patches.
  • Critical
    • Critical CVE patches will be applied to the master branch of the effected git repository and will be uploaded as a -security release which will be released imediately.

Testing

  • There are 3 types of required testing: verification, certification, and regression. The testing of a -proposed kernel must be completed within 14 working days begining when the upload has been accepted to the -proposed pocket.
  • Verification testing is the assurance that a bug is actually fixed by the proposed kernel, and is the responsibility of the bug reporter.
  • Certification is assurance that the kernel passes certification tests on specific hardware, and is the responsibility of the Platform Services, certification QA team.
  • Regression testing is testing to assure that no bugs have been re-introduced by the proposed kernel, and is the responsibility of the Platform QA team.
  • A tracking bug will be created and associated with each stable kernel in order to track status and completion of testing
    • This tracking bug will be listed in the changelog for the upload
    • This tracking bug will be assigned to (TBD - probably QA team)
    • This tracking bug number will be included in announcements sent when a new kernel is ready for testing after reverts have been made
    • Setting the tag "verification-done" on this tracking bug will signify that the kernel has passed testing and may be released

Details

  • Changes from bug reports will only be considered if we have the hardware and resources to test the fix, or a very high confidence that the bug reporter will test the proposed release.
  • At the beginning of the two-week cycle, bug fixes, upstream stable patches, and non-embargoed security fixes will be applied, and the kernel uploaded to proposed.
  • At the end of the first week, any bug fixes which have not been verified as fixed by the reporter or by internal testing will have the patches reverted, and the associated bugs set to "incomplete". A tag of "stable-reverted-<release>" will be added at this time (where <release> is [lucid | maverick . . .]). A comment will be added to the bug describing how to request reapplication of the fix.

  • During the second week, certification and regression testing will be performed to insure that the kernel is functional. Minimum testing will insure that the kernels boots, networking operates, X starts (for workstations), and updates can be performed. Certification tests will not include the entire set of manually executed tests, but will include an automated set of tests. Testing will be a joint effort by the QA and certification teams. The exact testing to be performed is To Be Determined.
  • Proposed releases will be tracked with a tracking bug opened by the stable kernel team, and linked via an entry in the changelog. This bug will be assigned to the QA team (or a launchpad team - TBD), and tracked on the kernel bug dashboard. The bug will be set to verified when all QA and Cert testing has passed.
  • Security kernels which are critical or embargoed will also receive the same tests required for proposed kernels.

Reconsideration of reverted fixes

When fixes are reverted due to lack of verification, they may be proposed for reconsideration by performing the following:

  • Update the bug with a firm commitment to verify the fix during the next stable kernel release cycle (usually this will begin one week after the fix is reverted). Change the "stable-reverted-<release>" tag to "stable-reapply-<release>" (where <release> is [lucid | maverick . . .]).

Initial Implementation Schedule

The initial cycles will adhere as much as possible to the two-week cadence, with adjustments for holidays.

Kernel/StableReleaseCadence (last edited 2019-03-14 10:47:54 by anthonywong)