Diagnosing Function Pointer Security Flaws with a GCC plugin

A few months ago, I had to write some internal GCC passes to perform static analysis on the GNU C Library (glibc). I figured I might as well write them as plugins since they were unlikely to see the light of day outside of my little sandbox. Being a long time GCC contributor, but having no experience writing plugins I thought it'd be a good way to eat our own dog food, and perhaps write about my experience.

Unfortunately, I quickly realized that non-trivial plugins are basically GCC hacking with some boilerplate code, and the details of such were unlikely to benefit anyone but die-hard GCC contributors who were very much capable of contributing their own extensions to GCC.

However, there was one particular pass which was squarely sitting in my sandbox, that could serve not only as an example of a non-trivial plugin for the curious but as a tool to pinpoint security flaws involving pointers to functions. Consequently, I'll make the code available and talk about the plugin use itself, not its implementation.

It has been known for some time that dereferencing function pointers in code is a focus of exploits, yet they are frequently used in code everywhere. Any time a function pointer is stored in read-write memory, its contents can potentially be altered in such a way that the flow of the program is maliciously transferred elsewhere.

A typical use of function pointers is a dispatch table:

#include <stdio.h>

typedef void (*callback)(void);

void hello (void)
{
  printf ("hello world\n");
}

callback dispatch[] = { hello };

int main(int argc, char **argv)
{
  callback cb = dispatch[0];
  // do some things
  cb ();
  return 0;
}

However, the dispatch table itself could be corrupted by any number of memory alterations. Even if the variable `cb' itself resides in memory, it could be altered between the reading from dispatch[0] and the execution of cb().

One proposed solution to this problem has been to encrypt any function pointers residing in memory (the dispatch table in this case). For instance, we could XOR each item in the dispatch table by a random amount upon start-up, and at each indirect function execution point in the program, we could twiddle the bits back. This would ensure that any malicious change to the dispatch table behind our back would yield jumps to garbage.

A simple decryptor for a function pointer use would be:

#define PTR_DEMANGLE(var) \
  (var) = (__typeof(var)) ((uintptr_t) (var) ^ secret_random_number)

int main(int argc, char **argv)
{
  callback cb = dispatch[0];
  PTR_DEMANGLE (cb);
  cb ();
  return 0;
}

This is the gist of the problem at its simplest. Obviously, you probably don't want to store that number anywhere, but perhaps a thread local register. You would also likely want a fancier function than PTR_DEMANGLE above. For that matter, you may even want some fancy target specific assembly sequence that does this for you:

__attribute__((decryptor)) // NOTE: `decryptor' attribute explained below.
static callback asm_demangler (callback f)
{
  asm ("#decrypt_operation \$2, \$0" : "=r" (f) : "0" (f));
  return f;
}

Other problematic uses of function pointers include:

• Copying a decrypted function pointer to memory.  Note: Copying to a register is probably OK, inasmuch as that the register is not spilled to memory.
• Passing a non-decrypted function pointer as an argument to a function, unless you are sure your function will decrypt things appropriately.
• Comparing a non-decrypted function pointer to a constant (say NULL).
• Instead, you should probably do:

f = PTR_DEMANGLE (dispatch[n]);
  if (f != NULL)
    ok();

• Copying a function pointer to memory:
  dispatch[i] = &printf;
• Passing objects containing function pointers to functions:
  

  struct obj { callback f };
  void foo (void)
  {
    struct obj arg;
    bar(&arg);
  }

• In C++, virtual functions can be the target of sophisticated exploits.

Auditing every use of function pointers in something as large as the GNU C Library is cumbersome and error prone to say the least. That's why I thought getting a compiler plugin to do the work would be a better use of my time, not to mention fun.

The plugin git sources, as well as instructions on how to build and use it, can be found here:

https://pagure.io/funcp-encrypt

To try this plugin on an existing code base, you can probably just set CC and CXX to use the plugin:

$ export CC="your-gcc -fplugin=/path/to/plugin.so"
$ export CXX="your-gcc -fplugin=/path/to/plugin.so"
$ make YOUR_STUFF


This should get you errors such as these while building with the plugin:

foo.c: In function ‘init’:
foo.c:49:21: warning: copy of a function pointer to memory
__resume = resume;
~~~~~~~~~^~~~~~~~
...

bar.c:88:8: warning: possible use of non decrypted function pointer
(*__init [i]) (argc, argv, envp);
~^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~

So far, the plugin can diagnose the following cases:

• Comparisons between encrypted function pointers and a constant.
• Comparison between encrypted and unencrypted function pointers.
• Passing non-decrypted function pointers as arguments.
• Dereferencing non-decrypted function pointers.
• Possible copies of function pointers to memory.

For additional examples of what the plugin can do, you can look at the test case provided in test-funcp.c.

Note: As an enhancement to existing code bases, I added a function attribute to be able to mark functions that were permitted to operate on function pointers without the usual restrictions (for instance, a function decryptor). With this plugin, a function with __attribute__((decryptor)) is considered an acceptable way to decrypt a function pointer, similar to PTR_DEMANGLE above. For example:

__attribute__((decryptor))
static callback asm_demangler (callback f)
{
  asm ("#decrypt_operation \$2, \$0" : "=r" (f) : "0" (f));
  return f;
}

With this attribute, the plugin will allow the asm_demangler() function to accept function pointers and operate on them without issuing a warning.

Paranoia aside, all these jumps and hoops are unlikely to yield any benefit in your financial report's callbacks, but keeping track of function pointers can be incredibly useful for library writers and other security-sensitive code.

Feel free to comment on the plugin and/or function pointer usage!

Build your first app with native GCC on RHEL 6 or 7.