DistCompilerFlags
Please check the status of this specification in Launchpad before editing it. If it is Approved, contact the Assignee or another knowledgeable person before making changes.
Launchpad Entry: dist-compiler-flags
Packages affected: dpkg, and all packages built using a compiler targeting native code.
Summary
Building a package archive with a consistant set of compiler and linker options currently requires modification of each single package or the use of wrapper scripts. This specification defines a way to pass/inject these options in the package build process.
Release Note
No specific release note; this will be used as base work for stuff like hardening or optimization for a specific processor.
Rationale
The set of compiler/linker flags used for building packages is defined in Debian policy:
Packages should be compiled using -g -O2 unless DEB_BUILD_OPTIONS doesn't specify noopt in which case packages should be built using -g -O0.
Packages should be stripped, unless nostrip is passed in DEB_BUILD_OPTIONS.
Currently two approaches are used to build the archive with modified options:
- Changing the compiler specs to modify the default options (either by patching the compiler or by adding custom spec files).
- Using scripts which wrap the invocations of the compiler and the linker.
Changing the behaviour at this level has a few disadvantages:
- The default behaviour of the development tools is changed unconditinally for all uses of the tools (not just package building. -fstack-protector), or for the most use cases (i.e. let build-essential depend on a package providing wrapper scripts or spec files).
- Packages can hardly work around problems in the development tools (for example lowering the optimization level, overriding a default optimization level).
- It is errorprone to combine more than one wrapper package.
- The injected/changed options do not show up in the build log (in the case of spec files), or require extra output (in the case of wrapper scripts).
The injection of compiler/linker options should be done in the basic package build tool (dpkg-buildpackage); the debian/rules file should handle passing/overriding the options to the upstream build system.
Similiar implementations do exist for rpm (Fedora, OpenSuse) and the Gentoo build system.
Use Cases
- Optimize a distribution to build for a spcific processor (as currently done with pentium-builder)
- Build a distribution with a common set of security/hardening options.
- Test build a set of packages using a set of non-standard set of compiler/linker options.
Assumptions
Design
We define a set of macros which are passed from dpkg-buildpackage to the debian build system (debian/rules). The injected flags are collected and passed to the upstream build system (LoicMinier: is this only done via the environment or via command line args as well?):
DEB_HOST_CFLAGS: Optimization options which are passed to the debian build system and can/should be overriden by the package build if needed (default value: -g -O2, or -g -O0 if noopt is specified. LoicMinier: would be nice to clarify what overrides are allowed, for example the package could filter our "-O" if it needs to, or it could append "-O3" to the flags to force a higher level of opts; it might be best to never clear the contents of this parameter
DEB_HOST_CFLAGS_APPEND: Optimization options appended to the compiler flags, which must not be overwritten by the package (mostly used to for test builds). Default value: empty. LoicMinier: why not simply require DEB_HOST_CFLAGS to be honored?
DEB_HOST_CPPFLAGS: Preprocessor flags which are passed to the debian build system and can/should be overriden by the package build if needed (default: empty). This macro is seldom used (most build systems just use CFLAGS instead of CPPFLAGS).
DEB_HOST_CPPFLAGS_APPEND: Preprocessor flags appended to the preprocessor flags, which must not be overwritten by the package (mostly used to for test builds). Default value: empty.
DEB_HOST_CXXFLAGS: Same as DEB_HOST_CFLAGS for C++ sources.
DEB_HOST_CXXFLAGS_APPEND: Same as DEB_HOST_CFLAGS_APPEND for C++ sources.
DEB_HOST_FFLAGS: Same as DEB_HOST_FFLAGS for Fortran sources.
DEB_HOST_FFLAGS_APPEND: Same as DEB_HOST_FFLAGS_APPEND for Fortran sources.
Compiler flags for other languages not mentioned above (Extend the specification as needed). LoicMinier: would it make sense to have libtool flags (LT_)? And configure args? What about setting the CC?
DEB_HOST_LDFLAGS: Options passed to the compiler when linking executables or shared objects (if the linker is called directly, then -Wl and , have to be stripped from these options. Default: empty.
Note: The DEB_HOST_ prefix is used to define these options for cross builds as well. For the case of a native build DEB_HOST == DEB_BUILD. We don't care about DEB_BUILD options for cross builds.
dpkg-buildpackage sets these flags to the default values unless the flags are already defined in the environment.
TODO: Identify other build systems not using dpkg-buildpackage and adopt these as well (autopkgtst).
TODO: Make build systems like cdbs comply with this spec.
Implementation
The implementation itself is trivial, but needs to be done in every source package.
TODO: Examples for debian/rules
LoicMinier: would be nice to have a transition plan; what happens if the parameters are not set? Does the package have the responsability to set them properly? Also, is the package/compiler required to support some specific flags or a specific optimization level like "-g -O2" and what other flags are allowed in these parameters?
Test/Demo Plan
The implementation status of a package can be checked by (automatically) inspecting the debian/rules file for every package which builds architecture dependent packages, or by inspecting the build dependencies of a source package (in the case that a build system supporting the spec is used).
The correctness of the implementation can be checked by inspecting the build logs.
Outstanding Issues
This should highlight any issues that should be addressed in further specifications, and not problems with the specification itself; since any specification with problems cannot be approved.
BoF agenda and discussion
Discussed at UDS/Boston.