Skip to main content
Redhat Developers  Logo
  • Products

    Featured

    • Red Hat Enterprise Linux
      Red Hat Enterprise Linux Icon
    • Red Hat OpenShift AI
      Red Hat OpenShift AI
    • Red Hat Enterprise Linux AI
      Linux icon inside of a brain
    • Image mode for Red Hat Enterprise Linux
      RHEL image mode
    • Red Hat OpenShift
      Openshift icon
    • Red Hat Ansible Automation Platform
      Ansible icon
    • Red Hat Developer Hub
      Developer Hub
    • View All Red Hat Products
    • Linux

      • Red Hat Enterprise Linux
      • Image mode for Red Hat Enterprise Linux
      • Red Hat Universal Base Images (UBI)
    • Java runtimes & frameworks

      • JBoss Enterprise Application Platform
      • Red Hat build of OpenJDK
    • Kubernetes

      • Red Hat OpenShift
      • Microsoft Azure Red Hat OpenShift
      • Red Hat OpenShift Virtualization
      • Red Hat OpenShift Lightspeed
    • Integration & App Connectivity

      • Red Hat Build of Apache Camel
      • Red Hat Service Interconnect
      • Red Hat Connectivity Link
    • AI/ML

      • Red Hat OpenShift AI
      • Red Hat Enterprise Linux AI
    • Automation

      • Red Hat Ansible Automation Platform
      • Red Hat Ansible Lightspeed
    • Developer tools

      • Red Hat Trusted Software Supply Chain
      • Podman Desktop
      • Red Hat OpenShift Dev Spaces
    • Developer Sandbox

      Developer Sandbox
      Try Red Hat products and technologies without setup or configuration fees for 30 days with this shared Openshift and Kubernetes cluster.
    • Try at no cost
  • Technologies

    Featured

    • AI/ML
      AI/ML Icon
    • Linux
      Linux Icon
    • Kubernetes
      Cloud icon
    • Automation
      Automation Icon showing arrows moving in a circle around a gear
    • View All Technologies
    • Programming Languages & Frameworks

      • Java
      • Python
      • JavaScript
    • System Design & Architecture

      • Red Hat architecture and design patterns
      • Microservices
      • Event-Driven Architecture
      • Databases
    • Developer Productivity

      • Developer productivity
      • Developer Tools
      • GitOps
    • Secure Development & Architectures

      • Security
      • Secure coding
    • Platform Engineering

      • DevOps
      • DevSecOps
      • Ansible automation for applications and services
    • Automated Data Processing

      • AI/ML
      • Data Science
      • Apache Kafka on Kubernetes
      • View All Technologies
    • Start exploring in the Developer Sandbox for free

      sandbox graphic
      Try Red Hat's products and technologies without setup or configuration.
    • Try at no cost
  • Learn

    Featured

    • Kubernetes & Cloud Native
      Openshift icon
    • Linux
      Rhel icon
    • Automation
      Ansible cloud icon
    • Java
      Java icon
    • AI/ML
      AI/ML Icon
    • View All Learning Resources

    E-Books

    • GitOps Cookbook
    • Podman in Action
    • Kubernetes Operators
    • The Path to GitOps
    • View All E-books

    Cheat Sheets

    • Linux Commands
    • Bash Commands
    • Git
    • systemd Commands
    • View All Cheat Sheets

    Documentation

    • API Catalog
    • Product Documentation
    • Legacy Documentation
    • Red Hat Learning

      Learning image
      Boost your technical skills to expert-level with the help of interactive lessons offered by various Red Hat Learning programs.
    • Explore Red Hat Learning
  • Developer Sandbox

    Developer Sandbox

    • Access Red Hat’s products and technologies without setup or configuration, and start developing quicker than ever before with our new, no-cost sandbox environments.
    • Explore Developer Sandbox

    Featured Developer Sandbox activities

    • Get started with your Developer Sandbox
    • OpenShift virtualization and application modernization using the Developer Sandbox
    • Explore all Developer Sandbox activities

    Ready to start developing apps?

    • Try at no cost
  • Blog
  • Events
  • Videos

GCC's new fortification level: The gains and costs

September 17, 2022
Siddhesh Poyarekar
Related topics:
C, C#, C++CompilersSecurity
Related products:
Developer Tools

Share:

    This article describes a new level of fortification supported in GCC. This new level detects more buffer overflows and bugs which mitigates security issues in applications at run time.

    C programs routinely suffer from memory management problems. For several years, a _FORTIFY_SOURCE preprocessor macro inserted error detection to address these problems at compile time and run time. To add an extra level of security, _FORTIFY_SOURCE=3 has been in the GNU C Library (glibc) since version 2.34. I described its mechanisms in my previous blog post, Broadening compiler checks for buffer overflows in _FORTIFY_SOURCE. There has been compiler support for this builtin in Clang for some time. Compiler support has also been available for GCC since the release of version 12 in May 2022. The new mitigation should be available in GNU/Linux distributions with packaged GCC 12.

    The following sections discuss two principal gains from this enhanced level of security mitigation and the resulting impact on applications.

    2 principal gains:

    1. Enhanced buffer size detection

    2. Better fortification coverage

    1. A new builtin provides enhanced buffer size detection

    There is a new builtin underneath the new _FORTIFY_SOURCE=3 macro n GCC 12 named __builtin_dynamic_object_size. This builtin is more powerful than the previous __builtin_object_size builtin used in _FORTIFY_SOURCE=2. When passed a pointer, __builtin_object_sizereturns as a compile-time constant that is either the maximum or minimum object size estimate of the object that pointer may be pointing to at that point in the program. On the other hand, __builtin_dynamic_object_size is capable of returning a size expression that is evaluated at execution time. Consequently, the _FORTIFY_SOURCE=3 builtin detects buffer overflows in many more places than _FORTIFY_SOURCE=2.

    The implementation of __builtin_dynamic_object_size in GCC is compatible with __builtin_object_size and thereby interchangeable, especially in the case of fortification. Whenever possible, the builtin computes a precise object size expression. When the builtin does not determine the size exactly, it returns either a maximum or minimum size estimate, depending on the size type argument.

    This code snippet demonstrates the key advantage of returning precise values:

    #include <string.h>
    #include <stdbool.h>
    #include <stdlib.h>
    
    char *b;
    char buf1[21];
    
    char *__attribute__ ((noinline)) do_set (bool cond)
    {
        char *buf = buf1;
        if (cond)
            buf = malloc (42);
    
        memset (buf, 0, 22);
        return buf;
    }
    
    int main (int argc, char **argv)
    {
        b = do_set (false);
        return 0;
    }

    The program runs to completion when built with -D_FORTIFY_SOURCE=2:

    gcc -O -D_FORTIFY_SOURCE=2 -o sample sample.c

    But the program aborts when built with -D_FORTIFY_SOURCE=3 and outputs the following message:

    *** buffer overflow detected ***: terminated
    Aborted (core dumped)

    The key enhancement stems from the difference in behavior between __builtin_object_size and __builtin_dynamic_object_size. _FORTIFY_SOURCE=2 uses __builtin_object_size and returns the maximum estimate for object size at pointer buf, which is 42. Hence, GCC assumes that the memset operation is safe at compile time and does not add a call to check the buffer size at run time.

    However, GCC with _FORTIFY_SOURCE=3 invokes __builtin_dynamic_object_size to emit an expression that returns the precise size of the buffer that buf points to at that part in the program. As a result, GCC realizes that the call to memset might not be safe. Thus, the compiler inserts a call to __memset_chk into the running code with that size expression as the bound for buf.

    2. Better fortification coverage

    Building distribution packages with _FORTIFY_SOURCE=3 revealed several issues that _FORTIFY_SOURCE=2 missed. Surprisingly, not all of these issues were straightforward buffer overflows. The improved fortification also encountered issues in the GNU C library (glibc) and raised interesting questions about object lifetimes.

    Thus, the benefit of improved fortification coverage has implications beyond buffer overflow mitigation. I will explain the outcomes of _FORTIFY_SOURCE=3 increased coverage in the following sections.

    More trapped buffer overflows

    Building applications with _FORTIFY_SOURCE=3 detected many simple buffer overflows, such as the off-by-one access in clisp issue. We expected these revelations, which strengthened our justification for building applications with _FORTIFY_SOURCE=3.

    To further support the use of _FORTIFY_SOURCE=3 to improve fortification, we used the Fortify metrics GCC plugin to estimate the number of times _FORTIFY_SOURCE=3 resulted in a call to a checking function (__memcpy_chk, __memset_chk, etc.). We used Fedora test distribution and some of the Server package group as the sample, which consisted of 96 packages. The key metric is fortification coverage, defined by counting the number of calls to __builtin_object_size that resulted in a successful size determination and the ratio of this number taken to the total number of __builtin_object_size calls. The plugin also shows the number of successful calls if using __builtin_dynamic_object_size instead of __builtin_object_size, allowing us to infer the fortification coverage if all __builtin_object_size calls were replaced with __builtin_dynamic_object_size.

    In this short study, we found that _FORTIFY_SOURCE=3 improved fortification by nearly 4 times. For example, the Bash shell went from roughly 3.4% coverage with _FORTIFY_SOURCE=2 to nearly 47% with _FORTIFY_SOURCE=3. This is an improvement of nearly 14 times. Also, fortification of programs in sudo went from a measly 1.3% to 49.57% — a jump of almost 38 times!

    The discovery of bugs in glibc

    The increased coverage of _FORTIFY_SOURCE=3 revealed programming patterns in application programs that tripped over the fortification without necessarily a buffer overflow. While there were some bugs in glibc, we had to either explain why we did not support it or discover ways to discourage those programming patterns.

    One example is wcrtomb, where glibc makes stronger assumptions about the object size passed than POSIX allowed. Specifically, glibc assumes that the buffer passed to wcrtomb is always at least MB_CUR_MAX bytes long. In contrast, the POSIX description makes no such assumption. Due to this discrepancy, any application that passed a smaller buffer would potentially make wcrtomb overflow the buffer during conversion. Then the fortified version __wcrtomb_chk aborts with a buffer overflow, expecting a buffer that is MB_CUR_MAX bytes long. We fixed this bug in glibc-2.36 by making glibc conform to POSIX .

    _FORTIFY_SOURCE=3 revealed another pattern. Applications such as systemd used malloc_usable_size to determine available space in objects and then used the residual space. The glibc manual discourages this type of usage, dictating that malloc_usable_size is for diagnostic purposes only. But applications use the function as a hack to avoid reallocating buffers when there is space in the underlying malloc chunk. The implementation of malloc_usable_size needs to be fixed to return the allocated object size instead of the chunk size in non-diagnostic use. Alternatively, another solution is to deprecate the function. But that is a topic for discussion by the glibc community.

    Strict C standards compliance

    One interesting use case exposed by _FORTIFY_SOURCE=3 raised the question of object lifetimes and what developers can do with freed pointers. The bug in question was in AutoGen, using a pointer value after reallocation to determine whether the same chunk extended to get the new block of memory. This practice allowed the developer to skip copying over some pointers to optimize for performance. At the same time, the program continued using the same pointer, not the realloc call result, since the old pointer did not change.

    Seeing that the old pointer continued without an update, the compiler assumed that the object size remained the same. How could it know otherwise? The compiler then failed to account for the reallocation, resulting in an abort due to the perceived buffer overflow.

    Strictly speaking, the C standards prohibit using a pointer to an object after its lifetime ends. It should neither be read nor dereferenced. In this context, it is a bug in the application.

    However, this idiom is commonly used by developers to prevent making redundant copies. Future updates to GCC may account for this idiom wherever possible, but applications should also explicitly indicate object lifetimes to remain compliant. In the AutoGen example, a simple fix is to unconditionally refresh the pointer after reallocation, ensuring the compiler can detect the new object size.

    The gains of improved security coverage outweigh the cost

    Building with _FORTIFY_SOURCE=3 may impact the size and performance of the code. Since _FORTIFY_SOURCE=2 generated only constant sizes, its overhead was negligible. However, _FORTIFY_SOURCE=3 may generate additional code to compute object sizes. These additions may also cause secondary effects, such as register pressure during code generation. Code size tends to increase the size of resultant binaries for the same reason.

    We need a proper study of performance and code size to understand the magnitude of the impact created by _FORTIFY_SOURCE=3 additional runtime code generation. However the performance and code size overhead may well be worth it due to the magnitude of improvement in security coverage.

    The future of buffer overflow detection

    _FORTIFY_SOURCE=3 has led to significant gains in security mitigation. GCC 12 support brings those gains to distribution builds. But the new level of fortification also revealed interesting issues that require additional work to support correctly. For more background information, check out my previous article, Enhance application security with FORTIFY_SOURCE.

    Object size determination and fortification remain relevant areas for improvements in compiler toolchains. The toolchain team at Red Hat continues to be involved in the GNU and LLVM communities to make these improvements.

    Last updated: November 8, 2023

    Related Posts

    • Broadening compiler checks for buffer overflows in _FORTIFY_SOURCE

    • Toward _FORTIFY_SOURCE parity between Clang and GCC

    • Enhance application security with FORTIFY_SOURCE &#124; Red Hat Security

    Recent Posts

    • GuideLLM: Evaluate LLM deployments for real-world inference

    • Unleashing multimodal magic with RamaLama

    • Integrate Red Hat AI Inference Server & LangChain in agentic workflows

    • Streamline multi-cloud operations with Ansible and ServiceNow

    • Automate dynamic application security testing with RapiDAST

    What’s up next?

    cheat sheet cover image

    Ready to level up your Linux knowledge? Our Intermediate Linux Cheat Sheet presents a collection of Linux commands and executables for developers and system administrators who want to move beyond the basics.

    Get the cheat sheet
    Red Hat Developers logo LinkedIn YouTube Twitter Facebook

    Products

    • Red Hat Enterprise Linux
    • Red Hat OpenShift
    • Red Hat Ansible Automation Platform

    Build

    • Developer Sandbox
    • Developer Tools
    • Interactive Tutorials
    • API Catalog

    Quicklinks

    • Learning Resources
    • E-books
    • Cheat Sheets
    • Blog
    • Events
    • Newsletter

    Communicate

    • About us
    • Contact sales
    • Find a partner
    • Report a website issue
    • Site Status Dashboard
    • Report a security problem

    RED HAT DEVELOPER

    Build here. Go anywhere.

    We serve the builders. The problem solvers who create careers with code.

    Join us if you’re a developer, software engineer, web designer, front-end designer, UX designer, computer scientist, architect, tester, product manager, project manager or team lead.

    Sign me up

    Red Hat legal and privacy links

    • About Red Hat
    • Jobs
    • Events
    • Locations
    • Contact Red Hat
    • Red Hat Blog
    • Inclusion at Red Hat
    • Cool Stuff Store
    • Red Hat Summit

    Red Hat legal and privacy links

    • Privacy statement
    • Terms of use
    • All policies and guidelines
    • Digital accessibility

    Report a website issue