MultiarchCross

Revision 5 as of 2009-10-23 13:52:13

Clear message

Summary

Extensions to the MultiarchSpec necessary for automated cross compilation and toolchain builds.

Motivation

The emdebian project aims to provide precompiled binaries for several embedded architectures, for some of which it is not possible or sensible to compile natively. For these, we need to build cross compilers and subsequently compile individual packages using these.

Current Status

The binutils package can be told to build a cross variant by passing a special environment variable. As there are no dependencies on target packages, this works for any GNU triplet supported by upstream, and can be automated easily.

The gcc package has a mechanism in place that rewrites various files in debian/ so that a set of cross compiler packages is built. The rewritten control file then declares build dependencies on several packages whose names end in a dash, the target Debian triplet and "-cross". These must be generated with the "dpkg-cross" tool from target packages. Automated bootstrap of new architectures is not possible this way, as a C library package compiled for the target is necessary to start building the packaged compiler.

Individual packages can be cross-compiled by passing a "-a" option to dpkg-buildpackage, which presets the environment variables accordingly. The package is responsible for honoring these variables, which most of the time can be achieved by passing --build and --host parameters to the configure script. The package's build dependencies need to be split into host and target dependencies manually, which precludes automation. An extended control file syntax that uses a "Build-Depends-Tools" field to list build dependencies that should not be translated is used in emdebian, but this solution is generally not considered scalable.

Utilizing Multiarch

The layout for libraries used by Multiarch is similar to that used by dpkg-cross in converted packages (except for having two path components swapped), and both have the goal of having libraries for different architectures co-installable, so it makes sense to integrate both, and drop dpkg-cross in the long run.

Multiarch already goes most of the way by specifying new paths where libraries are to be found; while the MultiarchSpec lists library -dev packages as unresolved, the transition plan is already pretty specific on making binutils and gcc look into multiarch directories, which is only needed for -dev packages.

Some extra work will be required to handle pkg-config .pc files (by teaching pkg-config about the new directories) and architecture-dependent headers (which are installed below /usr/lib by several packages). These can be dealt with by patching individual packages.

As it is for the (current) Multiarch approach, only libraries and library development files are interesting for the cross-compilation case. dpkg-cross strips all other files from the generated packages, so there is no change required there either.

we would like to take up the opportunity to introduce a host/target distinction in build dependencies in Debian at the same time, even if Debian itself would not use that information. This would be cheap to do at this stage, as the dependency handling code needs to be adapted anyway to handle architecture-qualified dependencies; we propose that in build dependencies, the special qualifiers ":build" and ":host" can be appended to package names. When not cross-compiling, these can and should be treated as if they were not present. The ":any" qualifier would have the same meaning as for regular dependencies, i.e. it would allow the dependency to be met by any package with an architecture that can be executed on the host, regardless of whether that is the same architecture as the current DEB_BUILD_ARCH.

Cross compiler generation is currently being changed so it can be automated in the current environment, by utilizing the "binutils-source", "gcc-source", "glibc-source"/"uclibc-source" and "gdb-source" binary packages together with a framework package containing build scripts and several small helper packages that can be fed to autobuilders. Later on, when Multiarch enabled apt is able to resolve dependencies on packages of a specified architecture, toolchain builds can be switched from a full bootstrap to building individual packages separately.

Transition

Phase 1 (before Debian squeeze)

  • dpkg is taught to ignore architecture qualifiers in build dependencies

This should happen before the release, so qualification can start afterwards without introducing compatibility problems.

Phase 2 (after Debian squeeze)

  • emdebian tools and dpkg are taught to handle "build"/"host"-qualified build dependencies
  • emdebian converts "Build-Depends-Tools" based host/target build dependency distinction to new format, submits patches.

At this point, emdebian can cross-compile all up-to-date Debian packages on their autobuilders, but toolchains still need some manual attention, which is acceptable.

Phase 3 (not really urgent)

  • apt gains support for installing target build dependencies
  • apt and dpkg learn about full architecture qualification

This will allow firmware files and boot blocks to be generated on any host, eliminating the need for architecture-independent files that can be built only on a single host

Unresolved Issues

Arch Qualification vs Toolchain Builds

Toolchain builds require going back and forth between several source trees, most notably gcc and libc:

Step

Source

Prerequisite

Action

0

linux

none

build linux-libc-dev headers

1

binutils

none

build binutils

2

gcc

none

build "freestanding" gcc

3

gcc

1,2

build static libgcc

4

libc

0,1,2,3

build intermediate libc, statically linked against libgcc

5

gcc

1,2,4

build shared libgcc, linked against intermediate shared libc

6

libc

0,1,2,5

build final libc

7

gcc

1,2,5,6

build final "hosted" gcc

8

gcc

1,5,6,7

build language support libraries

For this, architecture qualification of build dependencies is not really useful as the users are caught in the middle of the dependency loop. There are some ideas how to handle this in the autobuilders, but these are outside of the scope of this document.