Specs/M/ARMCrossCompilers

Launchpad Entry: arm-m-cross-compilers
Created: 2010/04/27
Contributors: Marcin Juszkiewicz, LoicMinier
Packages affected: gcc, eglibc, binutils, (cross-toolchain), (linux)

Summary

This specification describes the requirements for cross compilers packages.

Release Note

If we have ARM specific release notes: Ubuntu 10.10 now provides a cross-compiler for armel on i386 and amd64.

Rationale

Provide cross-compilers for various use cases such as building the kernel for kernel developers, building bootloaders on i386 into arch: all packages, or to cross-build any particular package.

User stories

Amit would like to cross-compile the kernel for ARM, he would like a cross-compiler as close to the native armel toolchain as possible, but just an apt-get install away. He simply installs the armel cross-compiler packages from the archive and voila!
Nicolas would like to cross-build a single package with xdeb (see the xdeb spec), he wants the results to be identical to a native build and doesn't want to have to build the toolchain by hand. He install the armel cross-compiler packages which were built using the same sources as the native armel toolchain and he can already start cross-building packages.
Deepti would like to ship a cross-compiler disguised as a native compiler in the new ARM SDK she is working on. She can rely on the pre-build cross-compiler binaries for her SDK.
Srinivasa would like to build his own cross-compiler packages for a new architecture which doesn't exist yet and which needs to be bootstraped. He can easily create a new source package which will him a full cross-compiler as .debs for his new architecture.

Assumptions

Cross-compilers are built using the same rules as the native toolchain, extending the logic to cover for the cross-compiler use case rather than duplicating it.
Cross-compilers currently target only Debian architectures and their corresponding GNU triplet.
Only the default upstream languages except objective-C are guaranteed to be supported by the cross-compiler, that is C, C++, and fortran. (Objective-C is being considered for demotion from the default set.)

Design

Change the standard rules in the toolchain packages (and their runtime deps) to allow creation of runtime cross-toolchain binary packages. The build needs to be parametrized to pass at least the target architecture of the toolchain and optionally the build stage in case multiple stages are needed.
Extend the -source binary packages provided by gcc-4.4/binutils/eglibc etc. to include the whole packaging, not just the source and optionally the patches but also the full build logic as to be able to call this logic.
Create a new source package which build-depends on these sources and will call the builds of all the toolchain components in order.
This new cross-toolchain source package should be able to call into local copies of the gcc-4.4/binutils/eglibc/etc. sources, as to allow custom cross-compilers. Also, some pedants claim that the cross-compiler wasn't built from source if it's not rebuilt on every toolchain source upload, this would allow silencing these complaints. This is why gcj-4.4 and gnat-4.4 have forked sources intead of a build-dep on gcc-4.4-source.
Refactor/simplify gcc packaging (rules) when/where needed to have more maintainable rules and to allow anybody to swap in his own sources; changes will be reviewed by Matthias Klose.

Implementation

Import gcc packaging in bzr to have full history while doing these extensive changes.
Merge the 'binary-cross' target into 'binary' for toolchain components.
Use the same interface to set the target architecture of the toolchain across all packages which need to be aware of it.
Allow building the intermediate stages of gcc and eglibc. Probably needs a parameter to set the stage which should be built.
Probably need to provide -source packages for build dependencies of the cross compiler, would be nice if they could all be built the same way.
Extend gcc/binutils to use -sysroot instead of --with-libs= --with-headers, the latter are deprecated and -sysroot is needed to build a full cross-toolchain using system pathes without installing the intermediate results (e.g. using an uninstalled binutils to build gcc); gcc-4.5 apparently already switched to sysroot, so this should be backported to 4.4. Right now, multiarch still requires --with-libs= --with-headers, but we will cross this bridge when we get there. It's likely we need to talk to upstream to see how multiarch pathnames and -sysroot can work together since --with-headers/-libs are deprecated upstream.
Provide the full package sources in the -source binary packages, e.g. copy the .dsc and .diff.gz into the -source binary package.
Create a new source package creating a cross-toolchain from the above
Check packages which build-depend on -source packages in Debian and fix them to work with -source packages which contain .dsc / .diff.gz etc.
Replace dh_movefiles (with dh_install?) in the gcc packaging to allow moving to new compat levels. Ideally, the packages should remain buildable on dapper (debhelper 5).

Test/Demo Plan

Cross-build the kernel and a couple of packages with the new cross-compilers.

Unresolved issues

BoF agenda and discussion

Improve packaging of cross-compilers

Overall goal: try to script creation of cross-compiler packages and allow in archive cross-compilers (and not out of archive + manually built)

- Improve support for -sysroot - Merge the binary-cross target into binary for e.g. gcc - Other simplifications of gcc/binutils etc. packaging to allow replacing these - Provide .dsc + .diff.gz etc. in the -source packages.

First goal:

- have x-compiler packages of the same version and targeting the same toolchain as the native compilers - compiler targeting arm running on x86 obvious first goal - also can target e.g. uclibc

existing approach: dpkg-cross, other highly manual approaches.

Can't run it in build environment because you can't build-depend on arm versions of packages.

Bootstrap from source: linux binutils gcc eglibc gcc eglibc gcc

Emdebian bootstrap: dpkg-cross <long list> binutils gcc binary-cross eglibc binary-cross

Bootstrap of cross-compiler requires building glibc-initial, then gcc-cross-initial, glibc, gcc-cross.

Could build e.g. glibc-arm package that could be installable on x86, but it would be ugly to provide an arm package as arch: all

Check how gcc-mingw32 is done? - support is split into few packages:

mingw32-runtime (headers + libraries)
mingw32-binutils
mingw32 (cross compiler)

Want to have the native and cross- packaging for gcc as similar as possible.

- perhaps don't need to mangle the binary names and have things like gcc-defaults
- - but maybe we want to anyway

Use case for build-depending on x-compiler: targeting nolibc

Need/opportunity to refactor/simplify gcc packaging because we want the rules to build the cross compiler and native compiler to be similar.

We currently have binary packages gcc-4.4-source which contain the debianized source tree of the gcc build, but not the dsc. If we provide the .dsc we know that building the package is easier. Need to provide -source packages for build dependencies of the cross compiler, would be nice if they could all be built the same way.

Need to move to using sysroot rather than configuring the cross toolchain at build time and using dpkg -i

sysroot or multiarch?

gcc 4.5 packaging uses sysroot already OpenEmbedded moved to sysroot too

What languages to build cross compilers for?

- demand driven, or keep things equivalent? - some things easy (obj-c) some things hard (java) - really only needs C and C++ - could target everything which is in main: C, C++, Fortran

also see: discussion of how to break build-dependency loops

https://blueprints.launchpad.net/ubuntu-arm/+spec/arm-m-automated-bootstrap

pbuilder? apt-cross?

Actions

lool to check reverse-build-deps in Debian of the tools source packages, to verify whether they'll be ok to transition to a common approach of shipping full Debian source packages (.dsc) in the binary package
lool to look into replacing dh_movefiles in the gcc packaging