CompilerFlags

Differences between revisions 1 and 52 (spanning 51 versions)
Revision 1 as of 2008-05-02 21:18:34
Size: 1755
Editor: c-76-105-157-155
Comment: initial pass
Revision 52 as of 2009-04-09 23:20:42
Size: 11080
Editor: c-71-237-255-223
Comment:
Deletions are marked like this. Additions are marked like this.
Line 1: Line 1:
This page documents the Ubuntu-specific default compiler flags in the toolchain. Based on the work from GccSsp, [:Security/HardeningWrapper:], and DistCompiler. Please attempt to fix a source package's problems before disabling a given compiler feature. ||<tablestyle="float:right; font-size: 0.9em; width:30%; background:#F1F1ED; background-repeat: no-repeat; background-position: 98% 0.5ex; margin: 0 0 1em 1em; padding: 0.5em;"><<TableOfContents>>||

T
his page documents the Ubuntu-specific default compiler flags in the toolchain. Based on the work from GccSsp, [[Security/HardeningWrapper]], and DistCompilerFlags. Please attempt to fix a source package's problems before disabling a given compiler feature, and document the package and bug numbers in the Problems section below.

= Default Flags =

<<Anchor(stack-protector)>>
Line 5: Line 10:
First enabled in Ubuntu 6.10.
See GccSsp for further details.

Failure example: {{{
First enabled in Ubuntu 6.10. Enabled run-time stack overflow verification. See GccSsp for further details. Most problems are related to packages that do not use stdlib directly (kernel modules, certain libraries, etc).

Failure examples:
 {{{
Line 11: Line 16:

Disabled with {{{-fno-stack-protector}}} or {{{-nostdlib}}}.
  Indicates a program was compiled to expect to have the stdlib available, but did not find it at runtime.

 {{{
*** stack smashing detected ***
Aborted
}}}
  A function did not correctly maintain its stack variables. Usually indicates a stack buffer overflow.

Disabled with {{{-fno-stack-protector}}} or {{{-nostdlib}}} in {{{CPPFLAGS}}}.

<<Anchor(fortify-source)>>
Line 16: Line 29:
First enabled in Ubuntu 8.10. See [:Security/FortifySource:] for more details. First enabled in Ubuntu 8.10. Provides compile-time best-practices errors for certain libc functions, and provides run-time checks of buffer lengths and memory regions. Only activated when compiled with {{{-O2}}} or higher. Most problems are related to common unsafe uses of certain libc functions. (For implementation details, see [http://gcc.gnu.org/ml/gcc-patches/2004-09/msg02055.html]. Starting with Jaunty, [[https://lists.ubuntu.com/archives/ubuntu-devel/2009-March/027832.html|fwrite]] was [[http://launchpadlibrarian.net/24428002/glibc_2.9-4ubuntu3_2.9-4ubuntu4.diff.gz|removed]] from the list of functions that are marked with "warn_unused_result".)
Line 22: Line 35:
  The return value from {{{system()}}}, {{{write()}}}, and similar functions should be evaluated and handled appropriately. In cases where one absolutely must throw away the return value, it can be discarded with an empty test: {{{if (system("...")) { } }}}, though this is not recommended.
Line 26: Line 40:
  When using {{{open()}}} with {{{O_CREAT}}}, best-practice is to define a valid {{{mode}}} argument. For the least modes, try using {{{(S_IRUSR|S_IWUSR)}}} first. If that doesn't work as expected in the program, then start adding back perms. For example, user and group: {{{(S_IRUSR|S_IWUSR|S_IRGRP|S_IWGRP)}}}; user, group, and other: {{{(S_IRUSR|S_IWUSR|S_IRGRP|S_IWGRP|S_IROTH|S_IWOTH)}}}.
Line 30: Line 45:

Reduced checking via {{{-D_FORTIFY_SOURCE=1}}}. Disabled with {{{-U_FORTIFY_SOURCE}}} or {{{-D_FORTIFY_SOURCE=0}}}.
  The call to {{{read()}}} was done into a buffer with the wrong size. Double-check the size argument and the buffer size.

 {{{
warning: passing argument 1 of 'memcpy' discards qualifiers from pointer target type
warning: passing argument 1 of 'strcpy' discards qualifiers from pointer target type
}}}
  Code compiled with {{{-Werror}}} and using memcpy/strcpy/etc with qualifier overrides will fail. This is a bug in glibc 2.7. See [[https://launchpad.net/bugs/217481]].

 {{{
*** %n in writable segment detected ***
Aborted
}}}
  On x86, use of {{{"%n"}}} in a format string is limited to read-only memory (not stack or heap allocated strings).

 {{{
*** buffer overflow detected ***
Aborted
}}}
  A call to {{{sprintf}}} should be changed to use {{{snprintf}}}, or a too-small buffer was read into (see {{{read()}}} warnings above).

Reduced checking via {{{-D_FORTIFY_SOURCE=1}}} in {{{CPPFLAGS}}}. Disabled with {{{-U_FORTIFY_SOURCE}}} or {{{-D_FORTIFY_SOURCE=0}}} in {{{CPPFLAGS}}}.

<<Anchor(format-security)>>
Line 35: Line 70:
First enabled in Ubuntu 8.10. These options should only cause FTBFS if the package is compiling with {{{-Werror}}}. First enabled in Ubuntu 8.10. Enables compile-time warnings about misuse of format strings, some of which can have security implications. These options should only cause build failures if the package is compiling with {{{-Werror}}} or {{{-Werror=format-security}}}. For details on marking up source to gain these warnings, see the [[http://library.gnome.org/devel/glib/stable/glib-Miscellaneous-Macros.html#G-GNUC-PRINTF--CAPS|glib macro documentation]] about gcc's "format" attribute.
Line 39: Line 74:
warning: format ‘%s’ expects type ‘char *’, but argument 3 has type ‘int’
}}}
  For packages that aren't already building with {{{-Wall}}}, format character to argument types will be checked. Verify the correct variables for a given format string.

 {{{
Line 41: Line 81:

This is caused by code that fails to use {{{"%s"}}} for a {{{*printf}}} function. For example: {{{
printf(buf);
}}}
should be: {{{
printf("%s",buf);
}}}

Disabled with {{{-Wno-format-security}}} or {{{-Wformat=0}}}.
  This is caused by code that forgot to use {{{"%s"}}} for a {{{*printf}}} function. For example:
   {{{
f
printf(stderr,buf);
}}}
  should be:
   {{{
f
printf(stderr,"%s",buf);
}}}

Disabled with {{{-Wno-format-security}}} or {{{-Wformat=0}}} in {{{CPPFLAGS}}}.

<<Anchor(relro)>>
Line 53: Line 95:
First enabled in Ubuntu 8.10. First enabled in Ubuntu 8.10. Provides a read-only relocation table area in the final ELF. This option paves the way for using {{{-z now}}} which forces all relocations to be resolved at run-time (which would cause some additional initial load delay), providing an even higher level of protection to the relocation table -- it could then be entirely read-only which can be used to further harden long-running programs like daemons.
Line 58: Line 100:

<<Anchor(triage)>>
= FTBFS Triage =

== FORTIFY return value checking ==

When encountering "ignoring return value of ..." build problems due to the FORTIFY flag, it is best to follow this general approach (and to refer to the above individual sections on a case-by-case basis):

 0. If there are examples of error handlers in near-by code, emulate them. An example can be seen with [[http://launchpadlibrarian.net/19891829/fortify-warnings.patch.dpatch|conntrack]]. This is, obviously, the preferred method of dealing with it.
 0. If there isn't an obvious way to handle the error, then stubbing out an empty handler is the next way to go -- fundamentally, this doesn't improve (or weaken) the quality of the code -- ignoring the return code is what's already happening, so this doesn't change anything. However, what it _does_ do, is allows the FORTIFY option to still be enabled, which means the program will gain the run-time FORTIFY protections.
 0. If there isn't a way to work around the error, and the problem is isolated to a small area of the source code, I've disabled FORTIFY for _only_ [[http://launchpadlibrarian.net/17169346/sysklogd_1.5-2ubuntu5_1.5-2ubuntu6.diff.gz|those source files]], which can be a pain, depending on the package's build methods.
 0. If nothing works, and other people have looked at it, and no one has any ideas about how to work around a problem with FORTIFY, then it's time to disable it for the entire build using the -U_FORTIFY_SOURCE CFLAG.

Now, in all of these situations, upstream needs to be notified. Especially for the #2, where they need to make a larger design decision about how to deal with the unhandled error condition.

Also, in situations where you've had to disable FORTIFY (#3, #4), please document the issue in the "Problems" section below (preferably also with a bug).

=== Ignoring return code of write() ===

If the error is ignoring the return code of write() or a similar function then it indicates that the
code is not implementing retry support for writes. write(2) says

{{{
       If a write() is interrupted by a signal handler before any bytes are
       written, then the call fails with the error EINTR; if it is interrupted
       after at least one byte has been written, the call succeeds, and
       returns the number of bytes written.
}}}

So something like the following should be used to retry when there is a short write
(thanks to SteveLangasek)

{{{
  do {
      written = write(fd, buf, count);
      if (written == count)
          break;
      if (written > 0) {
          buf += written;
          count -= written;
      }
  } while (written >= 0 || errno == EINTR);
  if (written < 0) {
    /* Handle error */
  }
}}}

<<Anchor(extensions)>>
= Extending FORTIFY =
If you are writing libraries or other code where you want to provide mark-up in headers to yell loudly when a function is misused, you can add them yourself.

== warn_if_result_ignored ==

 * Add to your function declaration: {{{__attribute__ ((warn_unused_result))}}}

<<Anchor(problems)>>
= Problems =
If the upstream source cannot be reasonably fixed and a package must have compiler flags disabled or some other work-around, please open a launchpad bug, tag it with "hardening-ftbfs", and link to it here along with an explanation of what the problem is:

== General ==

=== all source ===
   There is a functional regression when using the (technically not valid) code: {{{sprintf(buf, "%sblah", buf)}}}. https://bugs.edge.launchpad.net/ubuntu/+source/glibc/+bug/305901

=== libxfont1 ===
   doesn't work with -Bsymbolic-functions (can't work, see https://launchpad.net/bugs/230460 for analysis).

=== sysklogd ===
  When built with FORTIFY_SOURCE, klogd misdirects calls to syslog() into glibc instead of into syslog.o. See https://bugs.launchpad.net/ubuntu/+source/sysklogd/+bug/255635

  Workaround: added -U_FORTIFY_SOURCE to klogd.o's Makefile line.

=== synergy ===
  Fails to build with FORTIFY_SOURCE, and when patched to solve the warnings, it seems to hang and behave strangely. See https://bugs.launchpad.net/bugs/250869 and http://bugs.debian.org/cgi-bin/bugreport.cgi?bug=488460

  Workaround: added -U_FORTIFY_SOURCE to debian/rules CXXFLAGS.

=== openjdk-6 ===

  Unknown -- possibly testcase failures? https://launchpad.net/bugs/330713

  Workaround: added -U_FORTIFY_SOURCE and -fno-stack-protector

== Valid Code, But Compiler Is Unhappy ==

=== bacula ===
   Upstream has strange unions. https://bugs.launchpad.net/ubuntu/+source/bacula/+bug/265102

=== zsnes ===
   https://bugs.launchpad.net/ubuntu/intrepid/+source/zsnes/+bug/250425

== Test Suites Not Updated ==

=== binutils ===
   Add -Wno-format-security to CFLAGS, CXXFLAGS, due to picky default hardening options.

=== binutils-avr ===
   Disabled -Wno-format-security and _FORTIFY_SOURCE to avoid FTBFS. https://bugs.edge.launchpad.net/ubuntu/+source/binutils-avr/+bug/301732

=== gdb ===
   Testsuite reports false failures due to warnings, etc.

= Notes =
 * To see a list of all the #define'd variables, use: {{{gcc -E -dM - </dev/null}}}

This page documents the Ubuntu-specific default compiler flags in the toolchain. Based on the work from GccSsp, Security/HardeningWrapper, and DistCompilerFlags. Please attempt to fix a source package's problems before disabling a given compiler feature, and document the package and bug numbers in the Problems section below.

Default Flags

-fstack-protector

First enabled in Ubuntu 6.10. Enabled run-time stack overflow verification. See GccSsp for further details. Most problems are related to packages that do not use stdlib directly (kernel modules, certain libraries, etc).

Failure examples:

  • '__stack_chk_fail' symbol not found
    • Indicates a program was compiled to expect to have the stdlib available, but did not find it at runtime.
    *** stack smashing detected ***
    Aborted
    • A function did not correctly maintain its stack variables. Usually indicates a stack buffer overflow.

Disabled with -fno-stack-protector or -nostdlib in CPPFLAGS.

-D_FORTIFY_SOURCE=2

First enabled in Ubuntu 8.10. Provides compile-time best-practices errors for certain libc functions, and provides run-time checks of buffer lengths and memory regions. Only activated when compiled with -O2 or higher. Most problems are related to common unsafe uses of certain libc functions. (For implementation details, see [http://gcc.gnu.org/ml/gcc-patches/2004-09/msg02055.html]. Starting with Jaunty, fwrite was removed from the list of functions that are marked with "warn_unused_result".)

Failure examples:

  • error: ignoring return value of 'int system(const char*)', declared with attribute warn_unused_result
    • The return value from system(), write(), and similar functions should be evaluated and handled appropriately. In cases where one absolutely must throw away the return value, it can be discarded with an empty test: if (system("...")) { } , though this is not recommended.

    error: call to '__open_missing_mode' declared with attribute error: open with O_CREAT in second argument needs 3 arguments
    • When using open() with O_CREAT, best-practice is to define a valid mode argument. For the least modes, try using (S_IRUSR|S_IWUSR) first. If that doesn't work as expected in the program, then start adding back perms. For example, user and group: (S_IRUSR|S_IWUSR|S_IRGRP|S_IWGRP); user, group, and other: (S_IRUSR|S_IWUSR|S_IRGRP|S_IWGRP|S_IROTH|S_IWOTH).

    warning: call to ‘__read_chk_warn’ declared with attribute warning: read called with bigger length than size of the destination buffer
    • The call to read() was done into a buffer with the wrong size. Double-check the size argument and the buffer size.

    warning: passing argument 1 of 'memcpy' discards qualifiers from pointer target type
    warning: passing argument 1 of 'strcpy' discards qualifiers from pointer target type
    *** %n in writable segment detected ***
    Aborted
    • On x86, use of "%n" in a format string is limited to read-only memory (not stack or heap allocated strings).

    *** buffer overflow detected ***
    Aborted
    • A call to sprintf should be changed to use snprintf, or a too-small buffer was read into (see read() warnings above).

Reduced checking via -D_FORTIFY_SOURCE=1 in CPPFLAGS. Disabled with -U_FORTIFY_SOURCE or -D_FORTIFY_SOURCE=0 in CPPFLAGS.

-Wformat -Wformat-security

First enabled in Ubuntu 8.10. Enables compile-time warnings about misuse of format strings, some of which can have security implications. These options should only cause build failures if the package is compiling with -Werror or -Werror=format-security. For details on marking up source to gain these warnings, see the glib macro documentation about gcc's "format" attribute.

Failure examples:

  • warning: format ‘%s’ expects type ‘char *’, but argument 3 has type ‘int’
    • For packages that aren't already building with -Wall, format character to argument types will be checked. Verify the correct variables for a given format string.

    warning: format not a string literal and no format arguments
    • This is caused by code that forgot to use "%s" for a *printf function. For example:

      • fprintf(stderr,buf);
      should be:
      • fprintf(stderr,"%s",buf);

Disabled with -Wno-format-security or -Wformat=0 in CPPFLAGS.

-Wl,-z,relro

First enabled in Ubuntu 8.10. Provides a read-only relocation table area in the final ELF. This option paves the way for using -z now which forces all relocations to be resolved at run-time (which would cause some additional initial load delay), providing an even higher level of protection to the relocation table -- it could then be entirely read-only which can be used to further harden long-running programs like daemons.

No known failure examples.

Disabled with -Wl,-z,norelro in LDFLAGS.

FTBFS Triage

FORTIFY return value checking

When encountering "ignoring return value of ..." build problems due to the FORTIFY flag, it is best to follow this general approach (and to refer to the above individual sections on a case-by-case basis):

  1. If there are examples of error handlers in near-by code, emulate them. An example can be seen with conntrack. This is, obviously, the preferred method of dealing with it.

  2. If there isn't an obvious way to handle the error, then stubbing out an empty handler is the next way to go -- fundamentally, this doesn't improve (or weaken) the quality of the code -- ignoring the return code is what's already happening, so this doesn't change anything. However, what it _does_ do, is allows the FORTIFY option to still be enabled, which means the program will gain the run-time FORTIFY protections.
  3. If there isn't a way to work around the error, and the problem is isolated to a small area of the source code, I've disabled FORTIFY for _only_ those source files, which can be a pain, depending on the package's build methods.

  4. If nothing works, and other people have looked at it, and no one has any ideas about how to work around a problem with FORTIFY, then it's time to disable it for the entire build using the -U_FORTIFY_SOURCE CFLAG.

Now, in all of these situations, upstream needs to be notified. Especially for the #2, where they need to make a larger design decision about how to deal with the unhandled error condition.

Also, in situations where you've had to disable FORTIFY (#3, #4), please document the issue in the "Problems" section below (preferably also with a bug).

Ignoring return code of write()

If the error is ignoring the return code of write() or a similar function then it indicates that the code is not implementing retry support for writes. write(2) says

       If  a  write()  is interrupted by a signal handler before any bytes are
       written, then the call fails with the error EINTR; if it is interrupted
       after  at  least  one  byte  has  been  written, the call succeeds, and
       returns the number of bytes written.

So something like the following should be used to retry when there is a short write (thanks to SteveLangasek)

  do {
      written = write(fd, buf, count);
      if (written == count)
          break;
      if (written > 0) {
          buf += written;
          count -= written;
      }
  } while (written >= 0 || errno == EINTR);
  if (written < 0) {
    /* Handle error */
  }

Extending FORTIFY

If you are writing libraries or other code where you want to provide mark-up in headers to yell loudly when a function is misused, you can add them yourself.

warn_if_result_ignored

  • Add to your function declaration: __attribute__ ((warn_unused_result))

Problems

If the upstream source cannot be reasonably fixed and a package must have compiler flags disabled or some other work-around, please open a launchpad bug, tag it with "hardening-ftbfs", and link to it here along with an explanation of what the problem is:

General

all source

libxfont1

sysklogd

synergy

openjdk-6

Valid Code, But Compiler Is Unhappy

bacula

zsnes

Test Suites Not Updated

binutils

  • Add -Wno-format-security to CFLAGS, CXXFLAGS, due to picky default hardening options.

binutils-avr

gdb

  • Testsuite reports false failures due to warnings, etc.

Notes

  • To see a list of all the #define'd variables, use: gcc -E -dM - </dev/null

ToolChain/CompilerFlags (last edited 2021-09-02 05:51:07 by paelzer)