Skip to main content
Redhat Developers  Logo
  • Products

    Platforms

    • Red Hat Enterprise Linux
      Red Hat Enterprise Linux Icon
    • Red Hat AI
      Red Hat AI
    • Red Hat OpenShift
      Openshift icon
    • Red Hat Ansible Automation Platform
      Ansible icon
    • View All Red Hat Products

    Featured

    • Red Hat build of OpenJDK
    • Red Hat Developer Hub
    • Red Hat JBoss Enterprise Application Platform
    • Red Hat OpenShift Dev Spaces
    • Red Hat OpenShift Local
    • Red Hat Developer Sandbox

      Try Red Hat products and technologies without setup or configuration fees for 30 days with this shared Openshift and Kubernetes cluster.
    • Try at no cost
  • Technologies

    Featured

    • AI/ML
      AI/ML Icon
    • Linux
      Linux Icon
    • Kubernetes
      Cloud icon
    • Automation
      Automation Icon showing arrows moving in a circle around a gear
    • View All Technologies
    • Programming Languages & Frameworks

      • Java
      • Python
      • JavaScript
    • System Design & Architecture

      • Red Hat architecture and design patterns
      • Microservices
      • Event-Driven Architecture
      • Databases
    • Developer Productivity

      • Developer productivity
      • Developer Tools
      • GitOps
    • Automated Data Processing

      • AI/ML
      • Data Science
      • Apache Kafka on Kubernetes
    • Platform Engineering

      • DevOps
      • DevSecOps
      • Ansible automation for applications and services
    • Secure Development & Architectures

      • Security
      • Secure coding
  • Learn

    Featured

    • Kubernetes & Cloud Native
      Openshift icon
    • Linux
      Rhel icon
    • Automation
      Ansible cloud icon
    • AI/ML
      AI/ML Icon
    • View All Learning Resources

    E-Books

    • GitOps Cookbook
    • Podman in Action
    • Kubernetes Operators
    • The Path to GitOps
    • View All E-books

    Cheat Sheets

    • Linux Commands
    • Bash Commands
    • Git
    • systemd Commands
    • View All Cheat Sheets

    Documentation

    • Product Documentation
    • API Catalog
    • Legacy Documentation
  • Developer Sandbox

    Developer Sandbox

    • Access Red Hat’s products and technologies without setup or configuration, and start developing quicker than ever before with our new, no-cost sandbox environments.
    • Explore Developer Sandbox

    Featured Developer Sandbox activities

    • Get started with your Developer Sandbox
    • OpenShift virtualization and application modernization using the Developer Sandbox
    • Explore all Developer Sandbox activities

    Ready to start developing apps?

    • Try at no cost
  • Blog
  • Events
  • Videos

Memory Error Detection Using GCC

February 22, 2017
Martin Sebor
Related products:
Red Hat OpenShift

Share:

    Introduction

    GCC has a rich set of features designed to help detect many kinds of programming errors. Of particular interest are those that corrupt the memory of a running program and, in some cases, makes it vulnerable to security threats. Since 2006, GCC has provided a solution to detect and prevent a subset of buffer overflows in C and C++ programs. Although it is based on compiler technology, it's best known under the name Fortify Source derived from the synonymous GNU C Library macro that controls the feature: _FORTIFY_SOURCE. GCC has changed and improved considerably since its 4.1 release in 2006, and with its ability to detect these sorts of errors. GCC 7, in particular, contains a number of enhancements that help detect several new kinds of programming errors in this area. This article provides a brief overview of these new features. For a comprehensive list of all major improvements in GCC 7, please see GCC 7 Changes document.

    Important Note

    The options discussed below are available both with and without optimization. When used with optimization they can expose problems caused even by non-constant function argument values being in excess of some maximum. Although these optimizations can discover some such defects, they cannot find all of them. It's important to keep in mind that a successful compilation with these options enabled and with no warnings is no guarantee of the absence of bugs in a program and no substitute for comprehensive testing.

    The -Walloc-size-larger-than Option

    The -Walloc-size-larger-than=size option detects calls to memory allocation functions whose argument exceeds the specified size. The option also detects arithmetic overflow in the computation of the size in two-argument allocation functions like calloc where the total size is the product of the two arguments. Since calls with an excessive size are unlikely to succeed (and no object can be larger than PTRDIFF_MAX bytes) they are typically the result of programming errors.

    The option works not only for standard memory allocation functions like malloc but also for user-defined functions decorated with attribute alloc_size. The attribute tells GCC that a function returns memory whose size is given by its argument, or by a product of its arguments.

    The -Walloc-size-larger-than=PTRDIFF_MAX option is included in -Wall.

    For example, the following call to malloc incorrectly tries to avoid passing a negative argument to the function and instead ends up unconditionally invoking it with an argument less than or equal to zero. Since, after conversion to the type of the argument of the function (size_t), a negative argument results in a value in excess of the maximum PTRDIFF_MAX, the call is diagnosed.

    void* f (int n)
    {
      return malloc (n > 0 ? 0 : n);
    }
    
    warning: argument 1 range [2147483648, 4294967295] exceeds maximum object size 2147483647 [-Walloc-size-larger-than=]

    The -Walloc-zero Option

    The -Walloc-zero option detects calls to standard as well as user-defined memory allocation functions decorated with attribute alloc_size with a zero argument. The -Walloc-zero option is not included in either -Wall or -Wextra and must be explicitly enabled.

    The -Walloca Option

    The alloca(size) function allocates size bytes on the stack and returns a pointer to the allocated space. The function doesn't check to make sure the requested amount of space is available and so can easily exhaust the stack when not used carefully. On x86_64, for example, the default stack size of a Linux process is 8 MB. On 32-bit Linux, it's just 2 MB. The function is considered dangerous and its use is generally discouraged. To help projects detect and remove users of the function the -Walloca option detects all calls to alloca in a program. The -Walloca option is not included in either -Wall or -Wextra and must be explicitly enabled.

    For example, the following function calls alloca to allocate space in a loop. Such use is dangerous because even though it's likely only needed for the duration of each iteration of the loop the allocated space is not released until the function exits. With a large number of iterations that could exhaust the stack space.

    void f (int n)
    {
      for (int i = 0; i < n; ++i)
        {
          void *p = alloca (i);
          …
        }
    }
    
    warning: unbounded use of 'alloca' [-Walloca]

    The -Walloca-larger-than Option

    The -Walloca-larger-than=size option is considerably more permissive than -Walloca and detects only calls to the alloca function whose argument either may exceed the specified size, or that is not known to be sufficiently constrained to avoid exceeding it. Other calls are considered safe and not diagnosed. The -Walloca-larger-than is not included in either -Wall or -Wextra and must be explicitly enabled.

    For example, compiling the following snippet with -Walloca-larger-than=1024 results in a warning because even though the code appears to call alloca only with sizes of 1KB and less since n is signed, a negative value would result in a call to the function well in excess of the limit.

    void f (int n)
    {
      char *d;
      if (n < 1025)
        d = alloca (n);
      else
        d = malloc (n);
      …
    }
    
    warning: argument to 'alloca may be too large due to conversion from 'int' to 'long unsigned int' [-Walloca-larger-than=]

    In contrast, a call to alloca that isn't bounded at all such as in the following function will elicit the warning below regardless of the size argument to the option.

    void f (size_t n)
    {
      char *d = alloca (n);
      …
    }
    
    warning: unbounded use of 'alloca' [-Walloca-larger-than=]

    The -Wformat-overflow Option

    The -Wformat-overflow=level option detects certain and likely buffer overflow in calls to the sprintf family of formatted output functions. The option starts by determining the size of the destination buffer, which can be allocated either statically or dynamically. It then iterates over directives in the format string, calculating the number of bytes each result in output. For integer directives like %i and %x it tries to determine either the exact value of the argument or its range of values and uses the result to calculate the exact or minimum and maximum number of bytes the directive can produce. Similarly for floating point directives such as %a and %f, and string directives such as %s. When it determines that the likely number of bytes a directive results in will not fit in the space remaining in the destination buffer it issues a warning.

    Although the option is enabled even without optimization, it works best with -O2 and higher where it can most accurately determine objects sizes and argument values. To avoid false positives, when it cannot determine the exact value or range for an argument, level 1 of the option makes some fairly conservative assumptions about the data, such as that the value of an unknown integer is one (and in base 10 results in just one byte of output). If you want to increase the likelihood of finding bugs, use level 2. At this level, the option assumes that unknown integers have the value that results in the most bytes on output (such as INT_MIN for an int argument).

    For example, in the following snippet, the call to sprintf is diagnosed because even though its output has been constrained using the modulo operation it could result in as many as three bytes if mday were negative. The solution is to either allocate a larger buffer or make sure the argument is not negative, for example by changing mday's type to unsigned or by making the type of the second operand of the modulo expression unsigned: 100U.

    void* f (int mday)
    {
      char *buf = malloc (3);
      sprintf (buf, "%02i", mday % 100);
      return buf;
    }
    
    warning: 'sprintf may write a terminating nul past the end of the destination [-Wformat-overflow=]
    note: 'sprintf' output between 3 and 4 bytes into a destination of size 3

    The -Wformat-truncation Option

    Similar to -Wformat-overflow, the -Wformat-truncation=level option detects certain and likely output truncation in calls to the snprintf family of formatted output functions. Output truncation isn't considered as dangerous as an overflow but it can nevertheless lead to bugs with potentially serious security consequences that are often difficult to debug. -Wformat-truncation=1 is included in -Wall and enabled without optimization but works best with -O2 and higher.

    For example, the following function attempts to format an integer between 0 and 255 in hexadecimal, including the 0x prefix, into a buffer of four characters. But since the function must always terminate output by the NUL character ('\0') such a buffer is only big enough to fit just one digit plus the prefix. Therefore, the snprintf call is diagnosed. To avoid the warning either uses a bigger buffer or inspects the function's return value and handles the truncation reported by it.

    void f (unsigned x)
    {
      char d[4];
      snprintf (d, sizeof d, "%#02x", x & 0xff);
      …
    }
    
    warning: 'snprintf' output may be truncated before the last format character [-Wformat-truncation=]
    note: 'snprintf' output between 3 and 5 bytes into a destination of size 4

    The -Wnonnull Option

    The -Wnonnull option has existed in GCC since version 3.3 but it was limited to detecting null pointer constants and couldn't detect incorrect uses of other null pointers. In GCC 7, -Wnonnull has been enhanced to detect a much broader set of cases of passing null pointers to functions that expect a non-null argument (those decorated with attribute nonnull). By taking advantage of optimization, the option can detect more cases of the problem than in prior GCC versions.

    The -Wstringop-overflow Option

    The -Wstringop-overflow=type option detects buffer overflow in calls to string handling functions like memcpy and strcpy. The option relies on Object Size Checking and has an effect similar to defining the _FORTIFY_SOURCE macro. The type argument to the option refers to the Object Size Checking type plus 1. -Wstringop-overflow=1 is enabled by default.

    For example, because the call to memcpy below copies more bytes into the local array b than its size it is diagnosed.

    #define N 8
    const int a[N] = { /* ... */ };
    
    void f (void)
    {
      int b[N];
      unsigned n = N * sizeof *a;
      memcpy (b, a, n * sizeof *a);
      …
    }
    
    warning: 'memcpy' writing 128 bytes into a region of size 32 overflows the destination [-Wstringop-overflow=]

    As another example, in the following snippet, because the call to strncat specifies a maximum that allows the function to write past the end of the destination, it is diagnosed. To correct the problem and avoid the overflow the function should be called with a size of at most sizeof d - strlen(d) - 1.

    void f (const char *fname)
    {
      char d[8];
      strncpy (d, "/tmp/", sizeof d);
      strncat (d, fname, sizeof d);
      …
    }
    
    warning: specified bound 8 equals the size of the destination [-Wstringop-overflow=]

    The -Wvla Option

    The -Wvla option isn't new in GCC 7 but it's mentioned here for completeness. Similar to the -Walloca option, -Wvla points out all uses of Variable Length Arrays (see the -Wvla-larger-than below for more).

    The -Wvla-larger-than Option

    Similar to -Walloca-larger-than, the -Wvla-larger-than=size option detects definitions of Variable Length Arrays that can either exceed the specified size or whose bound is not known to be sufficiently constrained to avoid exceeding it. Variable Length Arrays are local array variables whose number of elements, which is not a constant expression, may need to be computed at runtime. VLAs are considered dangerous because when not used carefully they too can cause stack exhaustion. The -Wvla-larger-than option is not included in either -Wall or -Wextra and must be explicitly enabled.

    For example, compiling the following snippet with -Wvla-larger-than=4096 results in a warning because even though the code constrains the number of elements of the array, since the overall size is the result of n * sizeof (int), a value of n greater than 1024 would result in allocating more than 4KB of stack space.

    int f (unsigned n)
    {
      if (n > 4096)
        return -1;
      int a [n];
      …
    }
    
    warning: argument to variable-length array may be too large [-Wvla-larger-than=]
    note: limit is 4096 bytes, but argument may be as large as 16384

    Summary

    We hope you will find these new options useful either in finding existing bugs in your code or in preventing new ones from making their way into it. If you find problems with any of these features or have ideas for improvements please open bugs in GCC Bugzilla or drop us a line us at gcc-help@gcc.gnu.org.


    Join Red Hat Developers, a developer program for you to learn, share, and code faster – and get access to Red Hat software for your development.  The developer program and software are both free!

    Last updated: March 20, 2023

    Recent Posts

    • Run Qwen3-Next on vLLM with Red Hat AI: A step-by-step guide

    • How to implement observability with Python and Llama Stack

    • Deploy a lightweight AI model with AI Inference Server containerization

    • vLLM Semantic Router: Improving efficiency in AI reasoning

    • Declaratively assigning DNS records to virtual machines

    Red Hat Developers logo LinkedIn YouTube Twitter Facebook

    Products

    • Red Hat Enterprise Linux
    • Red Hat OpenShift
    • Red Hat Ansible Automation Platform

    Build

    • Developer Sandbox
    • Developer Tools
    • Interactive Tutorials
    • API Catalog

    Quicklinks

    • Learning Resources
    • E-books
    • Cheat Sheets
    • Blog
    • Events
    • Newsletter

    Communicate

    • About us
    • Contact sales
    • Find a partner
    • Report a website issue
    • Site Status Dashboard
    • Report a security problem

    RED HAT DEVELOPER

    Build here. Go anywhere.

    We serve the builders. The problem solvers who create careers with code.

    Join us if you’re a developer, software engineer, web designer, front-end designer, UX designer, computer scientist, architect, tester, product manager, project manager or team lead.

    Sign me up

    Red Hat legal and privacy links

    • About Red Hat
    • Jobs
    • Events
    • Locations
    • Contact Red Hat
    • Red Hat Blog
    • Inclusion at Red Hat
    • Cool Stuff Store
    • Red Hat Summit
    © 2025 Red Hat

    Red Hat legal and privacy links

    • Privacy statement
    • Terms of use
    • All policies and guidelines
    • Digital accessibility

    Report a website issue