LTO

Differences between revisions 2 and 3
Revision 2 as of 2021-01-28 16:01:07
Size: 3685
Editor: doko
Comment:
Revision 3 as of 2021-01-28 17:06:44
Size: 3689
Editor: doko
Comment:
Deletions are marked like this. Additions are marked like this.
Line 29: Line 29:
This is least ugly workaround we came up with, without having to fix build failures at the same time, when we enable LTO. It will go away at some time. This is the least ugly workaround we came up with, without having to fix build failures at the same time, when we enable LTO. It will go away at some time.

This page describes the approach to use link time optimizations for package builds starting with Ubuntu 21.04.

Summary

Link time optimization (LTO) is a way to run optimizations across multiple translation units, enabling more opportunities for optimizations at link time. Each source file is translated to an intermediate representation, which is then used at link to build an executable or a shared object/library. The optimizations allow for faster code and smaller files.

Compiler flags for LTO are injected in dpkg-buildflags, and can be overridden in the package build. LTO will be enabled on amd64, arm64, ppc64el and s390x.

Implementation

LTO is enabled on all 64bit architectures, except for riscv64, aiming to not slowing down the riscv64 builds further.

On architectures where LTO is disabled, it can be enabled in the rules file by

  • export DEB_BUILD_MAINT_OPTIONS=optimize=+lto

It can be disabled by

  • export DEB_BUILD_MAINT_OPTIONS=optimize=-lto

This adds the flags -flto=auto -ffat-lto-objects to CFLAGS, CXXFLAGS, OBJCFLAGS ... and the flag -flto=auto to LDFLAGS. -ffat-lto-objects is used to generate both the intermediate representation and the object code in the object files, the latter needed to ship .o and .a files in packages. All intermediate code is stripped out by dh_strip.

Building with -ffat-lto-objects slows down the build a bit, however we have around 2000 packages shipping .o and .a files. If a package doesn't ship these object files, then a package can remove the -ffat-lto-objects option.

LTO exclusion list

This is the least ugly workaround we came up with, without having to fix build failures at the same time, when we enable LTO. It will go away at some time.

The package lto-disabled-list contains a text file, listing source packages and architectures where LTO should not be enabled, allowing the package to build without modifying it. When adding a package to this list, please make sure that either a Debian or a Launchpad issue is on file.

Testing

Two archive test rebuilds were done to watch for regressions introduced by adding the LTO flags.

First test rebuild (using GCC 9):

Second test rebuild (using GCC 10):

Regressions are build failures failing in the LTO build, but not in the reference build.

Known issues

  • Link time optimization lets the compiler use more memory than a normal link step. Work arounds are reducing the parallelism, or serialization of different link time optimizations.
  • C++ symbols files differ, usually because some symbols for template instantiations are optimized out.
  • GCC and clang use different LTO options. The injected flags use the GCC variants.
  • The feature area optimize is not yet implemented in Debian dpkg. It will appear in dpkg 1.21.x. Patches using this area can be safely forwarded to Debian. The area will just be ignored.

References

Link time optimizations are already enabled in other Linux distributions, such as:

Plans for Debian:

ToolChain/LTO (last edited 2021-08-13 10:07:51 by ginggs)