CDImagePortsConsolidation

Summary

Currently, each flavour has a ports/ directory, separating some architectures from others. This is sometimes confusing to users looking to test various images, and sometimes confusing to adminsistrators looking to add or remove various images. Flavour coordinators often forget for which architectures their images are built, leading to gaps in ISO verification.

Consolidation of the /ports/ directories into the regular flavour directories would help provide a more accurate overview of the images made avialable for each flavour.

Mirror administrators are already expected to identify which architectures they prefer, so there should be little impact.

Release Note

Ubuntu now delivers all images available for each flavour in one convenient location.

Rationale

Flavour developers are often unaware which images are built, especially for powerpc or sparc, and may not be able to suitable support them. By showing them all together, the relevant test teams can better ensure that only testable images are produced, and that all produced images are tested.

User stories

  • Alice is a Xubuntu tester who discovers that powerpc images are available. She dusts off an old powerpc laptop, and helps test to ensure these images can be used to install Xubuntu successfully.
  • Bob wants to make a local mirror of the Ubuntu images available, but only for armel and amd64. Bob's users find the resulting combined pages much easier to read, and are not confused by alternate paths.
  • Chris is on the CDImage team, and is concerned about disk space usage. A quick look at a few pages highlights the set of images produced. Comparison to the ISO test tracker suggests which images may be suitable for removal.

Assumptions

  • Mirror administrators are using ubumirror to mirror cdimage
  • Flavour developers care about which architectures are produced and tested
  • Users may have a variety of hardware and may choose to install Ubuntu more widely when all images are presented

Design

  • Collapse all architectures, rather than having separate /ports/ trees for all new image builds (natty and later).
  • Modify cdimage release scripts to push all architectures to common /releases/ on release (continuing subset restrictions on releases.ubuntu.com)

Implementation

Code Changes

  • Modify cdimage scripts to not push to /ports/ anymore
  • Modify release scripts to copy appropriate release images to releases.ubuntu.com
  • Modify examples in ubumirror to not reference /ports/ anymore
  • Modify ubuntu-qa-tools to not pull from /ports/ anymore

Migration Plan

TBD

Test/Demo Plan

  • Build natty images using the new structure
  • Attempt to mirror using ubumirror
  • Attempt to grab images using ubuntu-qa-tools

Unresolved issues

BoF agenda and discussion

  • Review of rationale
  • Raise additional complications
  • Detailed breakdown of code changes
  • Migration Plan preparation

{{{currently we look at cdimage and each flavour has a ports directory, with some arches. We have to look in places to find out what we have per flavour. We forget to clean up various bits at times and space on cdimage is a big issue/limited resource. When we can't tell what we are looking at, there is a problem.

background: the reason why they are separate directories is essentially that they are built in separate clusters. when we are releaswing, then, its useful to do the important images ASAP. Since we initially layed out that split, there were some build changes to allow the ability to do partial rebuilds, to build for only some arches. One possibility is to build everything in a single pass, and in a single directory, but if we only want to build x96, we can pass the arches flag. Other options is putting links back and forward.

Not all flavours choose all arches. Such flavours are particularly poorly served by such a policy of separate directory.

The recommend solution to mirror everything, is to use umirror. With umirror, you have to identify which architectures you want, and which one you don't.

One thing to pay attention to, is how images are listed in the UI. The arches most used should be at the top. This may want to be different per flavour. Likely i386, amd64, armel, and powerpc. There are 2 flavours that support powerpc, and 2 that support arm. There ay be more powerpc users, but in terms of how we do things effectively, the above arch order is likely preferable.

Code changes: Most of the changes are in cdimage, with other changes in various mirroring tools. The proposal is that we ort to centralise all the images, per flavour. To take care of this for releases.ubuntu.com, we may need to use redirects. Future releases could be hosted in releases.ubuntu.com as well. The HTMl for the image dirs is automatically generated. This could be altered to indicate the level of support for an image, depending on the flavour.

These changes

Work items: cjwatson: Migrate cdimage to the above discussed directory structure: cjwatson: Fixing the release publication scripts, as there are some not only in cdimage, but scripts related to publishing to do with the iso tracker. -tools. Persia: Review the ubumirror defaults, and make sure they comply with the new policy. Other to be examined are testdrive, and ubuntu-qa-tools. persia: Need to talk to mirroring people, to make sure that the private mirroring scripts won't blow. cjwatson: Put redirects on cdimage, to point the old ports directories to the new locations. Persia: Make sure the spec indicates releases.ubuntu.com is unaffected by these changes.}}}


CategorySpec

Specs/CDImagePortsConsolidation (last edited 2010-10-27 19:43:30 by host194)