ARMDeveloperEnvironment

Differences between revisions 3 and 24 (spanning 21 versions)
Revision 3 as of 2010-05-18 13:03:05
Size: 3646
Editor: 5ac884b8
Comment: correct spec link
Revision 24 as of 2010-05-27 14:48:32
Size: 5808
Editor: bd7a3fd5
Comment:
Deletions are marked like this. Additions are marked like this.
Line 3: Line 3:
 * '''Launchpad Entry''': [[https://blueprints.launchpad.net/ubuntu-arm/+spec/arm-m-development-tools]]  * '''Launchpad Entry''': [[https://blueprints.launchpad.net/ubuntu/+spec/arm-m-development-tools]]
Line 5: Line 5:
 * '''Contributors''':  * '''Contributors''': GuilhermeSalgado
Line 10: Line 10:
This should provide an overview of the issue/functionality/change proposed here. Focus here on what will actually be DONE, summarising that so that other people don't have to read the whole spec. See also CategorySpec for examples.

== Release Note ==

This section should include a paragraph describing the end-user impact of this change. It is meant to be included in the release notes of the first release in which it is implemented. (Not all of these will actually be included in the release notes, at the release manager's discretion; but writing them is a useful exercise.)

It is mandatory.
We want to provide a simple yet powerful set of tools to allow ARM developers to easily create/manage archives (including package uploads/builds) and generate images.
Line 20: Line 14:
This should cover the _why_: why is this change being proposed, what justifies it, where we see this justified. Organizations need to generate their own images for testing
and evaluation purposes, possibly including software that supports unreleased hardware or software with restricted redistribution rights. To generate such images they need to maintain an integrated set of software packages that can be installed on to their devices.

== Definitions ==

See the Definitions section of [[Specs/M/ARMArchiveBranching|ARMArchiveBranching]] and [[Specs/M/ARMArchiveFeatures|ARMArchiveFeatures]]
Line 24: Line 23:
 1. Ted wants to generate an image for marvel doves, which require non-free software packages that are not included in Ubuntu but instead are packaged in a marvel PPA. Ted must be able to create a new archive (by [[Specs/M/ARMArchiveBranching|branching]] the Ubuntu archive and marvel's PPA) and generate the image using the new archive.

 1. A partner is experimenting with a custom netbook UI but doesn't want to include it in their main archive until they've done some more testing, so they use the tools to create a slim archive and do any UI changes (or add new packages) there. Once they decide the UI has had enough testing, they push the changes from the slim archive back to their main one. (The slim archive could be hosted on LP, like a [private] PPA, as soon enough we'll be able to upload packages through sftp).

 1. YAP (Yet Another Partner) is working on optimizing their new (not-yet-released) chip, but for that they need a version of GCC newer than the one on the Ubuntu archive. They want to create a new (private) archive where they'll upload the new GCC version, but upgrading to that new version of GCC is known to break binary compatibility, so it must be possible for them to easily rebuild all packages using the new GCC and generate images out of the new binaries to ensure the resulting system works as expected.

 1. YAP also has a separate team working on the UI for a device which will use their new chip, so they want to have yet another archive, based on the one containing the new GCC, where they'll make their UI changes without affecting other users of the archive containing the new GCC. The tools should allow them to do that as well.

 1. During the development of YAP's latest device, it should be possible for them to easily see the changes done to the upstream archive since both archives diverged. They should also be able to review those changes and pull the ones they want into their archive.

 1. YAP has finished development of a new device and want to upstream the changes they've done when developing. They need to be able to see what are those changes, decide which ones should go upstream and submit them. Similarly, they should be able to easily pull some/all changes from the upstream archive.

 1. At the end of the development of their new device, YAP's engineers want to freeze their archive so that all package uploads have to be reviewed by their release team before they're accepted. They run 'larc freeze http://archives.yap.com/new-device' and it's done.

Some of the features described here will depend on [[Specs/M/DerivedArchiveRebuild|DerivedArchiveRebuild]]
Line 25: Line 40:

 * Archives can be refered to by their URLs. (I think this is how we're going to tell the tools the archives they'll operate on)
Line 28: Line 45:
You can have subsections that better describe specific parts of the issue. The goal here is to provide a high-level interface with sensible defaults,
abstracting some of the low-level complexity of other utilities/frameworks that require more knowledge to be operated.

Users must be able to run any of the tools on either a Desktop or a Server. Also, the tools must not be tied to Launchpad, although they should probably take advantage of anything provided by Launchpad whenever desirable.

The tools should be able to operate on both local and remote archives, for both read and write operations. When writing they'll probably use the archive's HTTP RESTFUL API, and sftp to upload packages. Also they should be able to deal with archives hosted either on Launchpad or on [[Specs/M/ARMArchiveBranching|{{{vostok}}}]].
Line 32: Line 54:
This section should describe a plan of action (the "how") to implement the changes discussed. Could include subsections like:
Line 36: Line 57:
Should cover changes required to the UI, or specific UI that is required to implement this

=== Code Changes ===

Code changes should include an overview of what needs to change, and in some cases even the specific details.

=== Migration ===

Include:
 * data migration, if any
 * redirects from old URLs to new ones, if any
 * how users will be pointed to the new way of doing things, if necessary.
All tools will probably have no UI other than their command-line arguments.
Line 57: Line 67:
This should highlight any issues that should be addressed in further specifications, and not problems with the specification itself; since any specification with problems cannot be approved.  * A way of enforcing version number rules in a particular archive would be good. this is because we'll have to use fairly odd version numbering in downstream archives, so it'd be nice to have archives enforce their own rules for version numbering.
  * I assume this is to be enforced server-side, so should probably be moved to another spec? -- GuilhermeSalgado
Line 59: Line 70:
== BoF agenda and discussion ==
=== Goals ===
 * Goal: branch subset of ubuntu packages and manage changes
  * experiments - short/adhoc / long running
  * multiple parents: e.g. integrate goodies from multiple PPAs
  * hierarchy: common archive -> project archive -> project variant archive etc.
   * automatic superseding and merging
=== Releasing ===
 * Release/Freezes/ACLs
   * have automatic merged release branch owned by releaes team during development
     period that gets set to manual mode during freezes.
   * at release another branch is auto created that is not changeable??
=== Getting Started ===
 * getting started: command line tool to branch some archive; by default it starts
   by copying the binaries;
 * managing changes: webtool that visualizes relationship to parent archive:
   * changes in downstream archive
  * changes in upstream archive (merge o matic'ish)
 * some changes like gcc would require move a binary copy archive to a source/rebuild
   everything mode? is that true? gcc might just have changed for a bug fix/crash etc.
   * tracking build dependencies may be relevant here
 * a way of enforcing version number rules in a particular archive would be good
 * automatic superseding and merging, to keep the archive in sync with its upstream. Must also be able to stop the automatic superseding/merging at any point (e.g. when freezing for a release).
  * This also sounds like server-side to me. -- GuilhermeSalgado

Summary

We want to provide a simple yet powerful set of tools to allow ARM developers to easily create/manage archives (including package uploads/builds) and generate images.

Rationale

Organizations need to generate their own images for testing and evaluation purposes, possibly including software that supports unreleased hardware or software with restricted redistribution rights. To generate such images they need to maintain an integrated set of software packages that can be installed on to their devices.

Definitions

See the Definitions section of ARMArchiveBranching and ARMArchiveFeatures

User stories

  1. Ted wants to generate an image for marvel doves, which require non-free software packages that are not included in Ubuntu but instead are packaged in a marvel PPA. Ted must be able to create a new archive (by branching the Ubuntu archive and marvel's PPA) and generate the image using the new archive.

  2. A partner is experimenting with a custom netbook UI but doesn't want to include it in their main archive until they've done some more testing, so they use the tools to create a slim archive and do any UI changes (or add new packages) there. Once they decide the UI has had enough testing, they push the changes from the slim archive back to their main one. (The slim archive could be hosted on LP, like a [private] PPA, as soon enough we'll be able to upload packages through sftp).
  3. YAP (Yet Another Partner) is working on optimizing their new (not-yet-released) chip, but for that they need a version of GCC newer than the one on the Ubuntu archive. They want to create a new (private) archive where they'll upload the new GCC version, but upgrading to that new version of GCC is known to break binary compatibility, so it must be possible for them to easily rebuild all packages using the new GCC and generate images out of the new binaries to ensure the resulting system works as expected.
  4. YAP also has a separate team working on the UI for a device which will use their new chip, so they want to have yet another archive, based on the one containing the new GCC, where they'll make their UI changes without affecting other users of the archive containing the new GCC. The tools should allow them to do that as well.
  5. During the development of YAP's latest device, it should be possible for them to easily see the changes done to the upstream archive since both archives diverged. They should also be able to review those changes and pull the ones they want into their archive.
  6. YAP has finished development of a new device and want to upstream the changes they've done when developing. They need to be able to see what are those changes, decide which ones should go upstream and submit them. Similarly, they should be able to easily pull some/all changes from the upstream archive.
  7. At the end of the development of their new device, YAP's engineers want to freeze their archive so that all package uploads have to be reviewed by their release team before they're accepted. They run 'larc freeze http://archives.yap.com/new-device' and it's done.

Some of the features described here will depend on DerivedArchiveRebuild

Assumptions

  • Archives can be refered to by their URLs. (I think this is how we're going to tell the tools the archives they'll operate on)

Design

The goal here is to provide a high-level interface with sensible defaults, abstracting some of the low-level complexity of other utilities/frameworks that require more knowledge to be operated.

Users must be able to run any of the tools on either a Desktop or a Server. Also, the tools must not be tied to Launchpad, although they should probably take advantage of anything provided by Launchpad whenever desirable.

The tools should be able to operate on both local and remote archives, for both read and write operations. When writing they'll probably use the archive's HTTP RESTFUL API, and sftp to upload packages. Also they should be able to deal with archives hosted either on Launchpad or on Specs/M/ARMArchiveBranching.

Implementation

UI Changes

All tools will probably have no UI other than their command-line arguments.

Test/Demo Plan

It's important that we are able to test new features, and demonstrate them to users. Use this section to describe a short plan that anybody can follow that demonstrates the feature is working. This can then be used during testing, and to show off after release. Please add an entry to http://testcases.qa.ubuntu.com/Coverage/NewFeatures for tracking test coverage.

This need not be added or completed until the specification is nearing beta.

Unresolved issues

  • A way of enforcing version number rules in a particular archive would be good. this is because we'll have to use fairly odd version numbering in downstream archives, so it'd be nice to have archives enforce their own rules for version numbering.
    • I assume this is to be enforced server-side, so should probably be moved to another spec? -- GuilhermeSalgado

  • automatic superseding and merging, to keep the archive in sync with its upstream. Must also be able to stop the automatic superseding/merging at any point (e.g. when freezing for a release).


CategorySpec

Specs/M/ARMDeveloperEnvironment (last edited 2010-06-08 10:57:12 by fw-unat)