ARMevaluationAELALIP

Differences between revisions 1 and 36 (spanning 35 versions)
⇤ ← Revision 1 as of 2010-05-04 13:51:14 →
Size: 2662
Editor: fw-tnat
Comment:
← Revision 36 as of 2010-09-15 16:33:23 → ⇥
Size: 9585
Editor: fw-tnat
Comment:
Deletions are marked like this. Additions are marked like this.
Line 3: Line 3:
 * '''Launchpad Entry''': https://blueprints.launchpad.net/ubuntu-arm/+spec/arm-m-ael-alip-evaluation  * '''Launchpad Entry''': UbuntuSpec:arm-m-ael-alip-evaluation
Line 9: Line 9:

This should provide an overview of the issue/functionality/change proposed here. Focus here on what will actually be DONE, summarising that so that other people don't have to read the whole spec. See also CategorySpec for examples.		 ARM uses various linux file system images. This spec explores how these might be supplied from Ubuntu.
Line 13: Line 12:

This section should include a paragraph describing the end-user impact of this change. It is meant to be included in the release notes of the first release in which it is implemented. (Not all of these will actually be included in the release notes, at the release manager's discretion; but writing them is a useful exercise.)

It is mandatory.		 An archive of the ALIP-AEL software stack can now be built for ARM architectures ARMv5 & ARMV6 using Ubuntu packages and associated tools. This method can be extended for the remaining packages of the full distribution, provided any dependency loops are broken and packages are well behaved under cross compilation.
Line 19: Line 15:

This should cover the _why_: why is this change being proposed, what justifies it, where we see this justified.		 If Ubuntu is used from the very beginning of involvement with ARM it's more likely to be used for further activities. All initial examples of linux based systems provided by ARM Ltd. would tend to be Ubuntu based systems.
Line 23: Line 18:
 * New Architecture: ARM designers design a new architecture. They package a GNU cross compilation toolchain for the architecture. They can now use the available tools to cross build a complete, or partial, system image. This both exercises the toolchain, provides a basis for demonstrating the architectures benefits and provides the build-essential packages required to build (bootstrap) the complete archive for the new architecture.
Line 24: Line 20:
 * Development: ARM Ltd. developers want to develop a new BSP, tool or other software package. Pulling an original ALIP-AEL archive from Ubuntu they can use xdeb to quickly put their package into this & iterate builds for development.

 * uClinux To demonstrate an MMUless core ARM Ltd. developers pull a minimal uClinux system image from Ubuntu and boot it on their target.

 * Compiler Validation: To validate non-standard compiler options (including architectures) a partial, or complete, archive is rebuilt with a compiler defaulting to those options.
  
 * Demo: User supplied binary packages can be inserted into the ALIP-AEL software stack image with minimal effort.
  
 * Ship: ARM supplied development boards can be shipped with an installer or debootstrap style image which boots an ALIP-AEL stack with the ability to install further packages from the Ubuntu or related archives. Reduced images are flashed into boards such that they boot to a GUI out of box.

In the current cycle only the Compiler Validation use case will be satisfied, using xdeb. This method will be used rather than repeated native builds for the separate architectures since this aids development of solutions for the other cases. Those cases - New Architecture, Development, Demo may be considered for inclusion in future cycles.

The ucLinux case will not be considered further as it is one example of the New Architecture use case.
 
Line 25: Line 35:
Required tools are provided, in a timely fashion, by
 https://blueprints.launchpad.net/ubuntu/+spec/arm-m-cross-compilers <<BR>>
 https://blueprints.launchpad.net/ubuntu/+spec/arm-m-xdeb-cross-compilation-environment <<BR>>
 https://blueprints.launchpad.net/ubuntu/+spec/arm-m-development-tools <<BR>>
 https://blueprints.launchpad.net/ubuntu/+spec/arm-m-image-building-tool <<BR>>
Cross compilation GNU toolchain supplied by user in standard GNU/Ubintu packaged form for use by the Ubuntu tools.
Line 27: Line 43:

You can have subsections that better describe specific parts of the issue.		 The hardest use case is that of a complete new architecture.
A roadmap to achieve this can be constructed:-
 * Compiler Validation
 * Development
 * Demo
 * Ship
 * New Architecture.
Line 31: Line 52:

This section should describe a plan of action (the "how") to implement the changes discussed. Could include subsections like:

=== UI Changes ===

Should cover changes required to the UI, or specific UI that is required to implement this

=== Code Changes ===

Code changes should include an overview of what needs to change, and in some cases even the specific details.

=== Migration ===

Include:
 * data migration, if any
 * redirects from old URLs to new ones, if any
 * how users will be pointed to the new way of doing things, if necessary.

== Test/Demo Plan ==

It's important that we are able to test new features, and demonstrate them to users. Use this section to describe a short plan that anybody can follow that demonstrates the feature is working. This can then be used during testing, and to show off after release. Please add an entry to http://testcases.qa.ubuntu.com/Coverage/NewFeatures for tracking test coverage.

This need not be added or completed until the specification is nearing beta.		 === Compiler Validation ===
 * Package toolchain with desired architecture as default into local archive.
 * Pull alip-ael source packages (list to be attached) from ubuntu into local archive.
 * Build a chroot by hand
 * Set up high level dependencies (seeds) to construct alip-ael
 * Use xdeb, in that chroot, on the local archive to rebuild the alip-ael stack
   * Break any dependency loops locally
   * Fix any packages not cross compilation compliant (pushed upstream)
 * Build images from the local archive using the image building tool.
=== Development ===
Satisfied by the above.
=== Demo ===
Satisfied by the above.
=== Ship ===
Modify the ubuntu installer, if required, to access archives produced above
=== New Architecture ===
 * Produce build-essential packages using xdeb
 * Modify buildd to use xdeb
 * Move to native buildds as hardware becomes available.
Line 56: Line 73:
Does the ubuntu installer require adapting to use incomplete archives, in the manner described above.
Line 57: Line 75:
This should highlight any issues that should be addressed in further specifications, and not problems with the specification itself; since any specification with problems cannot be approved. == Research on packages in AEL/ALIP ==
Peter Pearse researched the list of packages currently in AEL/ALIP. See [[https://docs.google.com/document/edit?id=1k3_daUoOqFnS8vqF32meaq8Abk7FP6NKRHcoPWeGRPY&hl=en|Original alip-ael packages]]

Tom Gall has proposed an possible alip as listed in

[[https://spreadsheets.google.com/ccc?key=0AnPR4S1Uev7KdDhHM2RxVGFFY2VQM01MVEJXbTZ3TkE&hl=en|alip Package Status]]
Line 61: Line 84:
Use this section to take notes during the BoF; if you keep it in the approved spec, use it for summarising what was discussed and note any options that were rejected. Agenda:
* AEL/ALIP introduction
* Discussion
 * use-cases
 * procedures
 * delivery formats
 * cross compiling

=== AEL information and resources ===

  * http://www.arm.com/linux

=== ALIP Information and Resources ===

  * http://linux.onarm.com/index.php/Main_Page

=== Discussion ===

Packages and filesystem
 * When pulling updates into ALIP, generally choose the latest upstream versions
 * Filesystem tarballs distributed per architecture baseline
  * ARMv5
  * ARMv6
  * ARMv7 +NEON
  * ARMv7 +-Thumb2 +NEON
  * Generally, compiler options enable as many processor features as possible.
     -> this is an explicit intention to exercise build scenarios and processor features
  * System is flexible so the user can do a build with the right compiler options to target their use case / platform.
  * about 200 packages
  * Don't necessarily need to produce builds for all possible scenarios - this can be left to the user if they have specific requirements
    
Packaging
  * Git repository based on upstream
  * Some arm-specific patches held in git - but these are proactively pushed
    upstream also.
  * The AEL kernel typically contains some patches which are unlikely to be merged upstream.

Do people really need to rebuild everything?
  * People find the flexibility useful
  * If the user cannot rebuild, they are restricted with the common subset of the architecture.
  * In debian packages, copmiler options are often overridden during the build process, making it difficult to do a clean build with user-specified options
  * Currently the default compiler flags are set in the toolchain scratchbox
  * Currently, different compilers are used - gcc , armcc (RVCT).
  * Supporting multiple toolchains can create packaging complications e.g., dependencies on libgcc don't make sense between different toolchains. Proper dependencies are needed if we want to use the Debian/Ubuntu tools to construct filesystems.
  * Build-dependencies can be addressed by a static chroot which provides the functionality of build-essential.
  * An options is to base off Debian source packages, without using prebuild binary packages.
  * Info on switching toolchains in scratchbox: http://www.scratchbox.org/wiki/ForeignToolchains
  * A service for building packages (such as Ubuntu PPAs) is conceivable as a way to simplify life for users/developers, but not a priority for now?

Testsuites
  * Can consider running build-time tests ("make check") but not currently done by default. Does not work with true cross-compiling.

Use cases:
 * rootfs for different architectures to do validation/testing/demoes/benchmarks
   - armv5
   - armv6
   - armv7 neon
   - armv7 thumb2 neon
   - uclinux and eglibc versions for armv6 onwards
 * contains ~200 packages to provide a small minimum demo image
 * need to allow rebuild of the rootfs for different architecture opts
   - 4 hours to rebuild ~200 packages in QEMU on a desktop PC
 * set of packages hosted in git repos:
   - light: busybox for the small fs use case; our busybox would need a new flav
or based on uclinux
   - medium: xserver/x11 test environment
   - full: allows running a browser, webkit or mozilla, gtk or qt; need to provi
de both of them
 * packages have patches which are being pushed upstream
 * rebuild with different compiler or compiler flags
 * in practice, it's hard to support uClinux without a different toolchain (though it's possible in principle); it might be possible for the prebuilt images, but needs to be investigated
 * Should be really easy and really solid as to allow newbies to do rebuilds
 * A key problem is in package dependencies with outside toolchains: when building with armcc, we want packages to pickup dependencies on the toolchain runtime libs
   - We need to take the assumption that the armcc toolchain is packaged
 * ARM did a Debian rebuild using armcc; it would be highly interesting to know how they solved that; Will Deacon to check this out :-)
 * We want to be able to test with new toolchains
   - We need to take the assumption that the armcc toolchain is packaged

 * Problems:
   * packaging and updating to new major toolchain is hard to do
   * building with armcc might require packages - binaries only is OK -> not trying to solve the toolchain bootstrapping problem

=== Actions ===
 * Will Deacon to get details on the ARM internal Debian build using armcc
 * Ubuntu folks to try out ALIP! :-) arm.com/linux

Summary

ARM uses various linux file system images. This spec explores how these might be supplied from Ubuntu.

Release Note

An archive of the ALIP-AEL software stack can now be built for ARM architectures ARMv5 & ARMV6 using Ubuntu packages and associated tools. This method can be extended for the remaining packages of the full distribution, provided any dependency loops are broken and packages are well behaved under cross compilation.

Rationale

If Ubuntu is used from the very beginning of involvement with ARM it's more likely to be used for further activities. All initial examples of linux based systems provided by ARM Ltd. would tend to be Ubuntu based systems.

User stories

  • New Architecture: ARM designers design a new architecture. They package a GNU cross compilation toolchain for the architecture. They can now use the available tools to cross build a complete, or partial, system image. This both exercises the toolchain, provides a basis for demonstrating the architectures benefits and provides the build-essential packages required to build (bootstrap) the complete archive for the new architecture.

  • Development: ARM Ltd. developers want to develop a new BSP, tool or other software package. Pulling an original ALIP-AEL archive from Ubuntu they can use xdeb to quickly put their package into this & iterate builds for development.

  • uClinux To demonstrate an MMUless core ARM Ltd. developers pull a minimal uClinux system image from Ubuntu and boot it on their target.
  • Compiler Validation: To validate non-standard compiler options (including architectures) a partial, or complete, archive is rebuilt with a compiler defaulting to those options.
  • Demo: User supplied binary packages can be inserted into the ALIP-AEL software stack image with minimal effort.
  • Ship: ARM supplied development boards can be shipped with an installer or debootstrap style image which boots an ALIP-AEL stack with the ability to install further packages from the Ubuntu or related archives. Reduced images are flashed into boards such that they boot to a GUI out of box.

In the current cycle only the Compiler Validation use case will be satisfied, using xdeb. This method will be used rather than repeated native builds for the separate architectures since this aids development of solutions for the other cases. Those cases - New Architecture, Development, Demo may be considered for inclusion in future cycles.

The ucLinux case will not be considered further as it is one example of the New Architecture use case.

Assumptions

Required tools are provided, in a timely fashion, by

Cross compilation GNU toolchain supplied by user in standard GNU/Ubintu packaged form for use by the Ubuntu tools.

Design

The hardest use case is that of a complete new architecture. A roadmap to achieve this can be constructed:-

  • Compiler Validation
  • Development
  • Demo
  • Ship
  • New Architecture.

Implementation

Compiler Validation

  • Package toolchain with desired architecture as default into local archive.
  • Pull alip-ael source packages (list to be attached) from ubuntu into local archive.
  • Build a chroot by hand
  • Set up high level dependencies (seeds) to construct alip-ael
  • Use xdeb, in that chroot, on the local archive to rebuild the alip-ael stack
    • Break any dependency loops locally
    • Fix any packages not cross compilation compliant (pushed upstream)
  • Build images from the local archive using the image building tool.

Development

Satisfied by the above.

Demo

Satisfied by the above.

Ship

Modify the ubuntu installer, if required, to access archives produced above

New Architecture

  • Produce build-essential packages using xdeb
  • Modify buildd to use xdeb
  • Move to native buildds as hardware becomes available.

Unresolved issues

Does the ubuntu installer require adapting to use incomplete archives, in the manner described above.

Research on packages in AEL/ALIP

Peter Pearse researched the list of packages currently in AEL/ALIP. See Original alip-ael packages

Tom Gall has proposed an possible alip as listed in

alip Package Status

BoF agenda and discussion

Agenda: * AEL/ALIP introduction * Discussion

  • use-cases
  • procedures
  • delivery formats
  • cross compiling

AEL information and resources

ALIP Information and Resources

Discussion

Packages and filesystem

  • When pulling updates into ALIP, generally choose the latest upstream versions
  • Filesystem tarballs distributed per architecture baseline
    • ARMv5
    • ARMv6
    • ARMv7 +NEON
    • ARMv7 +-Thumb2 +NEON
    • Generally, compiler options enable as many processor features as possible.

      • -> this is an explicit intention to exercise build scenarios and processor features

    • System is flexible so the user can do a build with the right compiler options to target their use case / platform.
    • about 200 packages
    • Don't necessarily need to produce builds for all possible scenarios - this can be left to the user if they have specific requirements

Packaging

  • Git repository based on upstream
  • Some arm-specific patches held in git - but these are proactively pushed
    • upstream also.
  • The AEL kernel typically contains some patches which are unlikely to be merged upstream.

Do people really need to rebuild everything?

  • People find the flexibility useful
  • If the user cannot rebuild, they are restricted with the common subset of the architecture.
  • In debian packages, copmiler options are often overridden during the build process, making it difficult to do a clean build with user-specified options
  • Currently the default compiler flags are set in the toolchain scratchbox
  • Currently, different compilers are used - gcc , armcc (RVCT).
  • Supporting multiple toolchains can create packaging complications e.g., dependencies on libgcc don't make sense between different toolchains. Proper dependencies are needed if we want to use the Debian/Ubuntu tools to construct filesystems.
  • Build-dependencies can be addressed by a static chroot which provides the functionality of build-essential.
  • An options is to base off Debian source packages, without using prebuild binary packages.

  • Info on switching toolchains in scratchbox: http://www.scratchbox.org/wiki/ForeignToolchains

  • A service for building packages (such as Ubuntu PPAs) is conceivable as a way to simplify life for users/developers, but not a priority for now?

Testsuites

  • Can consider running build-time tests ("make check") but not currently done by default. Does not work with true cross-compiling.

Use cases:

  • rootfs for different architectures to do validation/testing/demoes/benchmarks
    • - armv5 - armv6 - armv7 neon - armv7 thumb2 neon - uclinux and eglibc versions for armv6 onwards
  • contains ~200 packages to provide a small minimum demo image
  • need to allow rebuild of the rootfs for different architecture opts
    • - 4 hours to rebuild ~200 packages in QEMU on a desktop PC
  • set of packages hosted in git repos:
    • - light: busybox for the small fs use case; our busybox would need a new flav

or based on uclinux

  • - medium: xserver/x11 test environment - full: allows running a browser, webkit or mozilla, gtk or qt; need to provi

de both of them

  • packages have patches which are being pushed upstream
  • rebuild with different compiler or compiler flags
  • in practice, it's hard to support uClinux without a different toolchain (though it's possible in principle); it might be possible for the prebuilt images, but needs to be investigated
  • Should be really easy and really solid as to allow newbies to do rebuilds
  • A key problem is in package dependencies with outside toolchains: when building with armcc, we want packages to pickup dependencies on the toolchain runtime libs
    • - We need to take the assumption that the armcc toolchain is packaged

  • ARM did a Debian rebuild using armcc; it would be highly interesting to know how they solved that; Will Deacon to check this out Smile :-)

  • We want to be able to test with new toolchains
    • - We need to take the assumption that the armcc toolchain is packaged
  • Problems:
    • packaging and updating to new major toolchain is hard to do

    • building with armcc might require packages - binaries only is OK -> not trying to solve the toolchain bootstrapping problem

Actions

  • Will Deacon to get details on the ARM internal Debian build using armcc

  • Ubuntu folks to try out ALIP! Smile :-) arm.com/linux

CategorySpec

Specs/M/ARMevaluationAELALIP (last edited 2011-01-21 16:09:41 by fw-tnat)