Ubuntu Wiki

• Launchpad Entry: jaunty-kernel-team-process

• Created: PeteGraner

• Contributors: JimLieb

• Packages affected: N/A

Summary

Review of the current kernel team processes for all stages of the release cycle.

Release Note

There are no end user impacts in this process definition work. The intent of this proposal is to improve internal processes.

Rationale

The Intrepid development cycle exposed a number of areas where the kernel team could improve the process. For example, regressions were introduced because patches or drivers were not carried foreward from the previous release. Processes, particularly decision processes, must be documented and published during the cycle. The kernel team and the work expected of it continues to expand and the current process is not scaling well.

Assumptions

The intent is to make no assumptions about "How Things are/were Done". Many new members of the team have little experience of the Debian/Ubuntu history.

Development Cycle

The development cycle starts with the opening of a new source tree and ends when support for the release expires.

Initial Planning

The initial planning has two goals. First, it defines and documents exactly what the kernel for this development cycle includes and, more importantly, what it drops. Each change will include a rationale for the change. Second, the result is published to all interested parties.

A key part of planning is to report the status of this work to the UDS where it will be reviewed.

The result of this work is not cast in stone. It will change as development proceeds but any changes in direction will also be published. The final version, including the rationale for each change will be published in the release documentation.

Kernel Re-base

The initial part of this work is more or less mechanical once the baseline kernel is chosen from the upstream. The final step is to isolate the differences between the current release tree and the mainline tree used for the re-base. These differences are Ubuntu specific changes that must be triaged.

SAUCE Patch Triage

Patches isolated during the re-base fit into the following categories:

• Upstream features, such as the RT patches, may not in the mainline yet. These patches will either have to be re-based to current version as well or forward ported to the new version.
• Some patches are backports of upstream fixes. Where possible, they will be replaced by their upstream equivalent.
• Other patches are Ubuntu specific bug fixes. These patches will have to be forward ported to the new version.
• There are Ubuntu specific configurations and code that are necessary for the distribution but will not meet the upstream requirements for mainline inclusion. These may or may not be forward ported.
• Some upstream features/options will be disabled in Ubuntu kernels. Disabled features are not supported.

This is a notification point that follows the process defined below. The documentation sent out in the notification shall document each change from the "vanilla" kernel, including a rationale for the change. Note that each change does not mean every changeset. The granularity of this is at the level of user visible subsystem or feature.

• Upstream versions of patches are preferred if their functional behavior is identical to the Ubuntu patch they replace. The update of the old patch to a mainline version is recorded.
• Any patch that is dropped without an upstream replacement is documented with both a rationale and an indication of what other parts of the distribution would be affected, for example, dropping support for a device or system.
• Additions that come from the new upstream that may impact other packages in the distribution.

Firmware Load Triage

• This is a notification point

Configuration (re)Definition

• This is a notification point

Development

Freeze Milestones

Update Process

SRU Release Points

Retirement

Review and Notification

At various stages of the development cycle, formal, documented reviews of certain items such as patches or firmware take place. The results of these reviews and the decisions made must then be communicated to the appropriate people and groups.

Review Steps

• Identify components
• Decision criteria for inclusion/removal, i.e. ubuntu patch superceded by upstream
• Licensing. Defined review process including legal.
• Stakeholders. Who is interested? Who will be inconvenienced/incensed?

Notification Process

• Who gets notified?
• Communications path: email, release notes, wiki page
• When? How often?

Kernel Specific Issues

Kernel Version Choice

The release cycle is long enough that it is possible that at least one, if not two kernel releases from upstream could occur within the development cycle. This brings up the following issues:

• Early in the cycle is better than later. What are the bounds?
• A decision to re-base depends on the value of new upstream work.
• The impact on the rest of the distribution must be known. Possible regressions in user-land packages should be identified and documented.

Feature Configuration Management

Kernel configuration for the various flavors of kernel are somewhat ad hoc and could use some structure.

Some embedded distributions have broken the kernel configuration file into generic, board/arch specific options, and "feature" options. They also have tools to merge and "layer" these configuration fragments into a final configuration for the build. This may be a useful model for re-factoring server/generic/desktop/notebook etc. configurations into something manageable.

Kernel Subsystem and User Land Dependencies

In the past, kernel features were added/enabled but there was no tracking of the user land part of the feature. The user land development fell behind, making the kernel component unusable. There should be a process in place to track the kernel and user land tasks. One option is to create open general release bugs in Launchpad with tasks to track both the kernel and user land work.

ABI Management

ABI changes can cause breakage in 3rd party packages. A process for identifying ABI changes and notifying impacted 3rd parties is defined.

Vanilla Kernel Packaging

The often requested Vanilla Kernel is an Ubuntu packaging of what is the current Linux release. This is not a supported kernel but it is useful for the isolation of reported bugs and the testing of upstream patches and features. Given the volatile nature of the Linux 2.6 HEAD, the packaging has to be controlled.

• The packaging process should be as automatic as possible from the git pull to the push to the PPA archive.

• There must be a process in place to decide when to generate a package. For example, daily packaging of potentially unstable or unbuildable kernel trees is wasteful and packaging very late in the release cycle limits testing availability.
• The package list and kernel configuration variations may not be as extensive as for the supported kernels.

Unresolved issues

BoF agenda and discussion

pgraner's notes

• Tree Mgt & Process & Kernel Packaging

  • New release tasks
    • Patch review
      • Need to be sure that when we drop a patch, it's an explicit decision, and coordinated with developers who might be depending on it.
      • next rebase to include a public review of all patches and their fate (e.g. on ubuntu-devel@lists)?
      • Upstreaming of Sauce patches.
  • Kernel Subsystem updates

    • Matching of Kernel -> Userspace components

• Notification?
  • Review
    • Firmware review
      • What can be deprecated, removed and whats new? License OK.
      • License review procedures.
      • Who is involved? Tech Board? Legal? sabdfl?
• Kernel Version
  • Checkpoints thought the cycle to ensure we don't hit the 2.6.27 late cycle issue
• New kernel features
  • Diffing of config
  • What constitutes us turning on a feature?
  • Approval
  • HWDb
  • New 3rd party code/firmware.
  • License review
  • Tracking files
• ABI (yet one more time)
  • List of symbols that vendors need?
    • VMWare
    • Parallels
    • Others?
• SRU Release Points
  • Predictable release of SRUs (time baesed)
  • Security as needed
  • Try and schedule ABI security bumpers and bugs at same time
• Upstream Vanilla Kernel Package
  • Is ths something we can start providing?
  • Beneficial for bug reporters to test the upstream kernel
  • Will also improve the upstream bug reporting process

References

• https://wiki.ubuntu.com/KernelTeam/KnowledgeBase (See the top list of links)

CategorySpec