ARMDeveloperEnvironment

Differences between revisions 11 and 13 (spanning 2 versions)
Revision 11 as of 2010-05-20 06:05:44
Size: 6374
Editor: mwhudson
Comment:
Revision 13 as of 2010-05-20 19:59:28
Size: 6263
Editor: bd7a3fd5
Comment:
Deletions are marked like this. Additions are marked like this.
Line 10: Line 10:
 * provide a way to create new archives by branching a subset of ubuntu packages
 * make it easy to change these archives
 * visualize (through a web tool, probably) deltas between archives and their (direct or no?) upstreams/downstreams
 * manage these deltas by pulling/pushing changes from/to upstreams
We want to provide a simple yet powerful set of tools to allow ARM developers to easily create/manage archives and generate images. These tools will be implemented as a layer on top of utilities/frameworks that work on a lower level and thus require more knowledge to be operated.
Line 15: Line 12:
There are two kinds of new archive that can be created: a 'full' archive which starts out containing every package in its parent archive at the version at the time of creation (the debian/ubuntu relationship is a bit like this) or an archive that starts out empty and expects that a user will have another archive to satisfy most package requests (the relationship between the primary archive and a PPA is like this) These tools will have a high-level interface with sensible defaults so that developers don't have to learn the specifics of each utility/framework in order to get started.
Line 19: Line 16:
- ?
Line 22: Line 20:
We need to allow ARM developers to easily create/maintain derivative archives for use during the development of new devices. They should be able to branch archives, make changes to them and manage deltas. We need to allow ARM developers to easily create/maintain derivative archives for use during the development of new devices. They should be able to branch archives, make changes to them and manage deltas without having to learn the myriad of tools that are currently needed to perform such tasks.
Line 26: Line 24:
 * Ted wants to package the latest version of the Foo project, which depends on the latest version of Bar and Baz, which in turn are not yet available in Ubuntu. Instead of packaging Bar and Baz, Joe can branch off of two existing PPAs that already have packages for the latest version of Bar and Baz. (This use case was invented to try and explain why we might want multiple parents, as discussed during the UDS session)  * Ted wants to package the latest version of the Foo project, which depends on the latest version of Bar and Baz, which in turn are not yet available in Ubuntu. Instead of packaging Bar and Baz, Joe can use these dev tools to easily branch off of two existing PPAs that already have packages for the latest version of Bar and Baz. (This use case was invented to try and explain why we might want multiple parents, as discussed during the UDS session)
Line 28: Line 26:
 * A partner is developing a custom netbook UI but need to keep it private until it's finalized, so they create a slim archive off of the main Ubuntu archive and do any UI changes (or add new packages) on their new archive, which is only accessible by themselves. The slim archive is made possible (instead of a full one) because it should be binary compatible with the main Ubuntu archive. (Not yet clear whether or not the new archive should be hosted on LP, like a private PPA; probably need to allow it to live in LP or outside)  * A partner is developing a custom netbook UI but need to keep it private until it's finalized, so they use the tools to create a slim archive off of the main Ubuntu archive and do any UI changes (or add new packages) on their new archive, which is only accessible by themselves. The slim archive is made possible (instead of a full one) because it should be binary compatible with the main Ubuntu archive. (Not yet clear whether or not the new archive should be hosted on LP, like a private PPA; probably need to allow it to live in LP or outside)
Line 32: Line 30:
 * YAP also has a separate team working on the UI for a device which will use their new chip, so they want to have yet another archive, based on the one containing the new GCC, where they'll make their UI changes without affecting other users of the archive containing the new GCC.  * YAP also has a separate team working on the UI for a device which will use their new chip, so they want to have yet another archive, based on the one containing the new GCC, where they'll make their UI changes without affecting other users of the archive containing the new GCC. The tools should allow them to do that as well.
Line 36: Line 34:
 * YAP has finished development of a new device and want to upstream the changes they've done when developing. They need to first see what are those changes, decide which ones should go upstream and submit them.  * YAP has finished development of a new device and want to upstream the changes they've done when developing. They need to first see what are those changes, decide which ones should go upstream and submit them.  Similarly, they should be able to easily pull some/all changes from the upstream archive.
Line 42: Line 40:
 * archives can be refered to by their URLs. (I think this is how we're going to tell the tools the archives they'll operate on)  * Archives can be refered to by their URLs. (I think this is how we're going to tell the tools the archives they'll operate on)

 * There are two kinds of new archive that can be created: a 'full' archive which starts out containing every package in its parent archive at the version at the time of creation (the debian/ubuntu relationship is a bit like this) or an archive that starts out empty and expects that a user will have another archive to satisfy most package requests (the relationship between the primary archive and a PPA is like this)
Line 46: Line 46:
You can have subsections that better describe specific parts of the issue. Goals:
 * provide a way to create new archives by branching a subset of ubuntu packages
 * make it easy to change these archives
 * visualize (through a web tool, probably) deltas between archives and their (direct or no?) upstreams/downstreams
 * manage these deltas by pulling/pushing changes from/to upstreams
 * Must allow either short/adhoc (e.g. upload a new upstream version of gstreamer, rebuild things against it and test that everything still works) or long running experiments (e.g. hardware enablement and the like, where you may take months to get things how you like before dropping it all in to Ubuntu at once when the hardware is announced)
 * deep hierarchies: common archive -> project archive -> project variant archive etc.
 * automatic superseding and merging, to keep the archive in sync with its upstream. Must also be able to stop the automatic superseding/merging at any point (e.g. when freezing for a release).
 * Need to be able to freeze an archive so that all uploads are held for approval by the release team.
 * a way of enforcing version number rules in a particular archive would be good. this is because we'll have to use fairly odd version numbering in downstream archives, so it'd be nice to have archives enforce their own rules for version numbering.
Line 50: Line 59:
This section should describe a plan of action (the "how") to implement the changes discussed. Could include subsections like:
Line 54: Line 62:
Should cover changes required to the UI, or specific UI that is required to implement this Most tools will probably have no UI other than their command-line arguments, except for showing deltas between archives, in which case we'll probably want a new UI in Launchpad. That UI should also allow to push/pull part of the delta upstream/downstream.
Line 56: Line 64:
=== Code Changes ===

Code changes should include an overview of what needs to change, and in some cases even the specific details.
Line 68: Line 73:
This should highlight any issues that should be addressed in further specifications, and not problems with the specification itself; since any specification with problems cannot be approved.
Line 70: Line 74:
== BoF agenda and discussion ==
=== Goals ===
  * Must allow either short/adhoc (e.g. upload a new upstream version of gstreamer, rebuild things against it and test that everything still works) or long running experiments (e.g. hardware enablement and the like, where you may take months to get things how you like before dropping it all in to Ubuntu at once when the hardware is announced)

  * deep hierarchies: common archive -> project archive -> project variant archive etc.

  * automatic superseding and merging, to keep the archive in sync with its upstream. Must also be able to stop the automatic superseding/merging at any point (e.g. when freezing for a release).

  * Need to be able to freeze an archive so that all uploads are held for approval by the release team.

 * a way of enforcing version number rules in a particular archive would be good. this is because we'll have to use fairly odd version numbering in downstream archives, so it'd be nice to have archives enforce their own rules for version numbering.

=== Releasing (side discussion which should probably not be covered here) ===
 * Release/Freezes/ACLs
   * have automatic merged release branch owned by releaes team during development
     period that gets set to manual mode during freezes.
   * at release another branch is auto created that is not changeable??
<james_w> the "Release/Freezes/ACLs" was a bit of a side discussion that I don't think we need to cover too much

Summary

We want to provide a simple yet powerful set of tools to allow ARM developers to easily create/manage archives and generate images. These tools will be implemented as a layer on top of utilities/frameworks that work on a lower level and thus require more knowledge to be operated.

These tools will have a high-level interface with sensible defaults so that developers don't have to learn the specifics of each utility/framework in order to get started.

Release Note

?

Rationale

We need to allow ARM developers to easily create/maintain derivative archives for use during the development of new devices. They should be able to branch archives, make changes to them and manage deltas without having to learn the myriad of tools that are currently needed to perform such tasks.

User stories

  • Ted wants to package the latest version of the Foo project, which depends on the latest version of Bar and Baz, which in turn are not yet available in Ubuntu. Instead of packaging Bar and Baz, Joe can use these dev tools to easily branch off of two existing PPAs that already have packages for the latest version of Bar and Baz. (This use case was invented to try and explain why we might want multiple parents, as discussed during the UDS session)
  • A partner is developing a custom netbook UI but need to keep it private until it's finalized, so they use the tools to create a slim archive off of the main Ubuntu archive and do any UI changes (or add new packages) on their new archive, which is only accessible by themselves. The slim archive is made possible (instead of a full one) because it should be binary compatible with the main Ubuntu archive. (Not yet clear whether or not the new archive should be hosted on LP, like a private PPA; probably need to allow it to live in LP or outside)
  • YAP (Yet Another Partner) is working on optimizing their new (not-yet-released) chip, but for that they need a version of GCC newer than the one on the Ubuntu archive. They want to create a new (private) archive where they'll upload the new GCC version, but upgrading to that new version of GCC is known to break binary compatibility, so it must be possible for them to easily rebuild all packages using the new GCC.
  • YAP also has a separate team working on the UI for a device which will use their new chip, so they want to have yet another archive, based on the one containing the new GCC, where they'll make their UI changes without affecting other users of the archive containing the new GCC. The tools should allow them to do that as well.
  • During the development of YAP's latest device, it should be possible for them to easily see the changes done to the upstream archive since both archives diverged. They should also be able to review those changes and pull the ones they want into their archive.
  • YAP has finished development of a new device and want to upstream the changes they've done when developing. They need to first see what are those changes, decide which ones should go upstream and submit them. Similarly, they should be able to easily pull some/all changes from the upstream archive.

Some of the features described here will depend on DerivedArchiveRebuild

Assumptions

  • Archives can be refered to by their URLs. (I think this is how we're going to tell the tools the archives they'll operate on)
  • There are two kinds of new archive that can be created: a 'full' archive which starts out containing every package in its parent archive at the version at the time of creation (the debian/ubuntu relationship is a bit like this) or an archive that starts out empty and expects that a user will have another archive to satisfy most package requests (the relationship between the primary archive and a PPA is like this)

Design

Goals:

  • provide a way to create new archives by branching a subset of ubuntu packages
  • make it easy to change these archives
  • visualize (through a web tool, probably) deltas between archives and their (direct or no?) upstreams/downstreams
  • manage these deltas by pulling/pushing changes from/to upstreams
  • Must allow either short/adhoc (e.g. upload a new upstream version of gstreamer, rebuild things against it and test that everything still works) or long running experiments (e.g. hardware enablement and the like, where you may take months to get things how you like before dropping it all in to Ubuntu at once when the hardware is announced)
  • deep hierarchies: common archive -> project archive -> project variant archive etc.

  • automatic superseding and merging, to keep the archive in sync with its upstream. Must also be able to stop the automatic superseding/merging at any point (e.g. when freezing for a release).
  • Need to be able to freeze an archive so that all uploads are held for approval by the release team.
  • a way of enforcing version number rules in a particular archive would be good. this is because we'll have to use fairly odd version numbering in downstream archives, so it'd be nice to have archives enforce their own rules for version numbering.

Implementation

UI Changes

Most tools will probably have no UI other than their command-line arguments, except for showing deltas between archives, in which case we'll probably want a new UI in Launchpad. That UI should also allow to push/pull part of the delta upstream/downstream.

Test/Demo Plan

It's important that we are able to test new features, and demonstrate them to users. Use this section to describe a short plan that anybody can follow that demonstrates the feature is working. This can then be used during testing, and to show off after release. Please add an entry to http://testcases.qa.ubuntu.com/Coverage/NewFeatures for tracking test coverage.

This need not be added or completed until the specification is nearing beta.

Unresolved issues


CategorySpec

Specs/M/ARMDeveloperEnvironment (last edited 2010-06-08 10:57:12 by fw-unat)