Notes about enabling PIE by default in gcc for amd64

The following is my notes about landing PIE in 16.04.

In the gcc-5, there are two additional patches added to enable this, the first is applying the patch H.J. Lu landed in gcc trunk (https://gcc.gnu.org/viewcvs/gcc?view=revision&revision=223796), which allows gcc to be configured with -pie on by default (and adds the disabling option -no-pie). The second patch changes the arguments passed to the linker (ld) to enable -z now (aka Immediate Binding) when -pie is enabled on amd64.

Patches are available in https://code.launchpad.net/~sbeattie/+junk/pie-amd64-patches .

Build Testing

A first round of building testing was done in the gcc-pie-amd64 PPA, attempting to build most of main that has architecture specific components (i.e. build architecture 'all' or 'amd64'). The vast majority of packages succeeded to build.

(I also enabled the ppa in a xenial desktop VM and basic testing showed no problems.)

I'll capture the failures I see and the solutions to them

PIE+ASLR issue for kernels in vivid (3.19) and older

In build testing, one of the issues discovered is an (as yet unknown) issue with the kernel's handling of PIE+aslr binaries for kernels older than 4.2 (i.e. vivid and older). This manifests when bash is built with pie enabled and then is used to build some other packages; frequently an error like:

   bash: xmalloc: .././locale.c:81: cannot allocate 2 bytes (0 bytes allocated)

is seen in the build logs. Unfortuantely, the buildds are running a mix of 3.13 and 3.19 kernels. Building with bash reverted to non-pie works around the issue. I verified that it's a kernel issue by reproducing the issue locally in an sbuild with bash+pie on a host running trusty, reproducing the build failure and then rebooting into the linux-lts-wily kernel, redoing the build, and seeing it succeed.

This issue affects the following package build failures (from below)

• aalib
• cdebconf-terminal
• cloog-ppl
• cpio
• cwidget
• dbus-c++
• ecryptfs-utils
• elfutils
• evolution-data-server
• firefox
• git
• glade
• libmnl
• p11-kit
• shadow
• util-linux

This has been filed as LP: #1518483

A workaround that I'm testing that at least succeeded for cpio is rebuild bash with pie disabled, and then rebuild the failing package. But it leaves the possibility that other build programs may trip over it in perhaps less obvious ways.

Incompatible relocation R_X86_64_32

• camlp5
• findlib
• netcfg

These are due to the static libraries they're linking against (/usr/lib/ocaml/libcamlrun.a for campl5 and findlib, /usr/lib/gcc/x86_64-linux-gnu/5/../../../x86_64-linux-gnu/libcheck.a for netcfg) not being built in PIC mode. To resolve, build the package that provides the static library with -pie first, then rebuild the failing package.

incremental linking via ld -r (incompatible with -pie)

Often shows up with go modules and for other oddballs.

• grub2
• lxd
• qemu

miscellaneous test case failures and other oddities

A few packages have tests that check that they either identify a binary executable properly, or verify that the did some operation on an executable that does not mess it up. These will need to be patched:

• cmake (amd64)

  • looks like a test case fails that tries to ensure that it bundled an executable properly

  • Upstream patch: https://cmake.org/gitweb?p=cmake.git;a=commit;h=fe558718b30da989db8b880374012a0e580574e6

• qtbase-opensource-src (amd64)

  • Fails in mimetype test because it's looking for an executable type, but the pie binaries look like shared libraries

Hardening-wrapper needs to be taught about -pie and -no-pie

• hardening-wrapper (amd64)

  • fails in no-hardening testcase, needs to know about pie by default and no-pie

Others have random failures due to -pie and likely need a patch to disable it.

• emacs24

  • builds fine with -no-pie; known issue with emacs

• linux
• erlang (maybe? it's a documentation build failure, need to verify it succeeds with non-pie)

  • fop fails when building a piece of documentation with a java traceback of Exception in thread "main" java.lang.NoSuchMethodError: org.apache.xmlgraphics.xmp.Metadata.mergeInto(Lorg/apache/xmlgraphics/xmp/Metadata;)V

  • Could be the same issue as https://bugzilla.redhat.com/show_bug.cgi?id=1194369 ; may need to merge newer fop for erlang build (but not related to -pie

• golang-race-detector-runtime (amd64)

  • ==5011==ERROR: ThreadSanitizer failed to allocate 0x4000 (16384) bytes at address 1fc448bf40000 (errno: 12)

• keyutils (amd64)

  • unknown test failure (endianness test issue?)
  • new sync from debian addressed the issue

raw list of build failures (w/arches)

• checkbox (both)
  • non-pie related testcase failures
• click (both)
  • non-pie related testcase failures

• cmake (amd64)

  • lookslike a test case fails that tries to ensure that it bundled an executable properly
• corosync (i386)

  • /usr/bin/ld.bfd.real: BFD (GNU Binutils for Ubuntu) 2.25.51.20151113 assertion fail ../../bfd/elf32-i386.c:5297 o.0

• emacs24 (amd64)

  • unknown problem with pie (fails with wily build host, too)

  • rebuilt okay with -no-pie

• erlang (amd64)

  • unknown error occurs in documentation build
• fcitx-qt5 (both)

  • dpkg-gensymbols failure (both arches)

• golang (both)

  • https://github.com/golang/go/issues/13114

• golang-race-detector-runtime (amd64)

  • ==5011==ERROR: ThreadSanitizer failed to allocate 0x4000 (16384) bytes at address 1fc448bf40000 (errno: 12)

• gparted (both)

  • ../include/Win_GParted.h:28:31: fatal error: sigc++/class_slot.h: No such file or directory

• grantlee (both)

  • dpkg-gensymbols failure

• grep (i386)
  • non-pie testcase failure

• grub2 (amd64)

  • /usr/bin/ld: -r and -pie may not be used together

• hardening-wrapper (amd64)

  • fails in no-hardening testcase, needs to know about pie by default and no-pie
• ibus (both)

  • SyntaxError: invalid syntax Makefile:765: recipe for target 'org.freedesktop.ibus.gschema.xml.in' failed

• keyutils (amd64)

  • unknown test failure (endianness test issue?)
• libcmis (both)

  • depwait on a specific lib version Missing dependencies: libcppunit-dev (= 1.13.2-2.1) 

• libprelude (both)

  • perl/libpreludecpp-perl.i:1: Error: Unknown SWIG preprocessor directive: Exception (if this is a block of target language code, delimit it with %{ and %}) 

• libreoffice (both)
• libxdmcp (both)

  • dh_install: libxdmcp-dev missing files (usr/share/doc/libXdmcp/*.txt), aborting

• libxfont (both)

  • dh_install: libxfont-dev missing files (../../build-main/doc/*.txt), aborting

• libxi (both)

  • dh_install: libxi-dev missing files (usr/share/doc/libXi/*.txt), aborting

• lxd (amd64)

  • /usr/bin/ld: -r and -pie may not be used together during go module build

• phonon-backend-gstreamer (both)
  • cmake error
• procps (both)

  • Running ./pmap.test/pmap.exp ... FAIL: pmap X with unreachable process FAIL: pmap XX with unreachable process

• qemu (amd64)

  • kernel aslr issue

  • with non-pie bash, build gets farther, but hits /usr/bin/ld: -r and -pie may not be used together issue

• qtbase-opensource-src (amd64)

  • Fails in mimetype test because it's looking for an executable type, but the pie binaries look like shared libraries
• qtsvg-opensource-src (both)

  • dpkg-gensymbols: error: thunk is not a valid version

• sendmail (both)

  • *** debian/control was updated, aborting, please restart your build ***

• shim (amd64)

  • /usr/include/efi/x86_64/efibind.h:86:24: fatal error: stdint.h: No such file or directory ?

• sosreport (both)
  • non-pie testcase failures
• subversion (both)
  • unknown SWIG preprocess directives
• syslinux (both)

  • objcopy -O binary -S ldlinux.elf ldlinux.raw fails on both aches

• ubuntu-app-launch (both)
  • deprecated api warning with -Werror
• ubuntu-drivers-common (both)

  • test_system_device_drivers_detect_plugins test fail

• unity (i386)
  • still FTBFS after cmake dep is addressed
• whois (both)

  • Invalid version number in debian/changelog

build successful after static lib dependency was rebuilt with -pie (to get -fPIC)

• camlp5 (amd64)

  • /usr/bin/ld: /usr/lib/ocaml/libasmrun.a(roots.o): relocation R_X86_64_32 against `caml_frametable' can not be used when making a shared object; recompile with -fPIC

  • build succeeded after ocaml was rebuilt with -pie

• findlib (amd64)

  • /usr/bin/ld: /usr/lib/ocaml/libcamlrun.a(stacks.o): relocation R_X86_64_32 against `.rodata.str1.8' can not be used when making a shared object; recompile with -fPIC

  • build succeeded after ocaml was rebuilt with -pie

• netcfg (amd64)

  • /usr/bin/ld: /usr/lib/gcc/x86_64-linux-gnu/5/../../../x86_64-linux-gnu/libcheck.a(check.o): relocation R_X86_64_32 against `.rodata.str1.1' can not be used when making a shared object; recompile with -fPIC

  • not sure what package build fixed this, truthfully

kernel aslr failures, worked around by building with non-pie bash

• aalib (amd64)

• cdebconf-terminal (amd64))

• cloog-ppl (amd64)

• cpio(amd64)

• cwidget (amd64)

• dbus-c++ (amd64

• ecryptfs-utils (amd64)

• elfutils (amd64)

• evolution-data-server (amd64)

• firefox (amd64)

• git (amd64)

• glade (amd64)

• gnome-desktop3 (amd64)

• gnome-vfs (amd64)

• graphviz (amd64)

• gtk+2.0 (amd64)

• gtkmm2.4 (amd64)

• gtkspell (amd64)

• icu (amd64)

• indent (amd64)

• jack-audio-connection-kit (amd64)

• libbit-vector-perl (amd64)

• libhybris (amd64)

• liblangtag (amd64)

• libmnl (amd64)

• libreoffice

• librevenge (amd64)

• libunwind (amd64)

• libvigraimpex

• libxml-libxml-perl (amd64)

• mono (amd64)

• nmap (amd64)

• openipmi (amd64)

• p11-kit (amd64)

• pidgin (amd64)

• python-cryptography (amd64)

• qtgraphicaleffects-opensource-src (amd64)

• qtsensors-opensource-src (amd64)

• rpm (amd64)

• shadow (amd64)

• swedish (amd64)

• systemtap (amd64)

• util-linux (amd64)

• xfsprogs (amd64)

i386 failures due to amd64 FTBFS

Some i386 builds failed because they had a dependency on an arch all package built from a source that had FTBFS on amd64; after fixing the amd64 failure, rebuilding the i386 package succeeded.

cmake

• kde4libs (i386)
• libssh (i386)
• libvigraimpex
• mir (i386, depends on arch all package from cmake)
• oxide-qt (i386, depends on arch all package from cmake)
• unity (i386, depends on all arch from cmake)

gnome-desktop3

• evolution (i386, lib-gnome-desktop3-dev dependency)
• gnome-control-center (i386, dependency issue on libgnome-desktop-3-dev)
• gnome-settings-daemon (i386)
• nautilus (i386, depends on libgnome-desktop-3-dev)
• telepathy-glib (i386)
• totem (i386)