Skip to main content
Redhat Developers  Logo
  • Products

    Featured

    • Red Hat Enterprise Linux
      Red Hat Enterprise Linux Icon
    • Red Hat OpenShift AI
      Red Hat OpenShift AI
    • Red Hat Enterprise Linux AI
      Linux icon inside of a brain
    • Image mode for Red Hat Enterprise Linux
      RHEL image mode
    • Red Hat OpenShift
      Openshift icon
    • Red Hat Ansible Automation Platform
      Ansible icon
    • Red Hat Developer Hub
      Developer Hub
    • View All Red Hat Products
    • Linux

      • Red Hat Enterprise Linux
      • Image mode for Red Hat Enterprise Linux
      • Red Hat Universal Base Images (UBI)
    • Java runtimes & frameworks

      • JBoss Enterprise Application Platform
      • Red Hat build of OpenJDK
    • Kubernetes

      • Red Hat OpenShift
      • Microsoft Azure Red Hat OpenShift
      • Red Hat OpenShift Virtualization
      • Red Hat OpenShift Lightspeed
    • Integration & App Connectivity

      • Red Hat Build of Apache Camel
      • Red Hat Service Interconnect
      • Red Hat Connectivity Link
    • AI/ML

      • Red Hat OpenShift AI
      • Red Hat Enterprise Linux AI
    • Automation

      • Red Hat Ansible Automation Platform
      • Red Hat Ansible Lightspeed
    • Developer tools

      • Red Hat Trusted Software Supply Chain
      • Podman Desktop
      • Red Hat OpenShift Dev Spaces
    • Developer Sandbox

      Developer Sandbox
      Try Red Hat products and technologies without setup or configuration fees for 30 days with this shared Openshift and Kubernetes cluster.
    • Try at no cost
  • Technologies

    Featured

    • AI/ML
      AI/ML Icon
    • Linux
      Linux Icon
    • Kubernetes
      Cloud icon
    • Automation
      Automation Icon showing arrows moving in a circle around a gear
    • View All Technologies
    • Programming Languages & Frameworks

      • Java
      • Python
      • JavaScript
    • System Design & Architecture

      • Red Hat architecture and design patterns
      • Microservices
      • Event-Driven Architecture
      • Databases
    • Developer Productivity

      • Developer productivity
      • Developer Tools
      • GitOps
    • Secure Development & Architectures

      • Security
      • Secure coding
    • Platform Engineering

      • DevOps
      • DevSecOps
      • Ansible automation for applications and services
    • Automated Data Processing

      • AI/ML
      • Data Science
      • Apache Kafka on Kubernetes
      • View All Technologies
    • Start exploring in the Developer Sandbox for free

      sandbox graphic
      Try Red Hat's products and technologies without setup or configuration.
    • Try at no cost
  • Learn

    Featured

    • Kubernetes & Cloud Native
      Openshift icon
    • Linux
      Rhel icon
    • Automation
      Ansible cloud icon
    • Java
      Java icon
    • AI/ML
      AI/ML Icon
    • View All Learning Resources

    E-Books

    • GitOps Cookbook
    • Podman in Action
    • Kubernetes Operators
    • The Path to GitOps
    • View All E-books

    Cheat Sheets

    • Linux Commands
    • Bash Commands
    • Git
    • systemd Commands
    • View All Cheat Sheets

    Documentation

    • API Catalog
    • Product Documentation
    • Legacy Documentation
    • Red Hat Learning

      Learning image
      Boost your technical skills to expert-level with the help of interactive lessons offered by various Red Hat Learning programs.
    • Explore Red Hat Learning
  • Developer Sandbox

    Developer Sandbox

    • Access Red Hat’s products and technologies without setup or configuration, and start developing quicker than ever before with our new, no-cost sandbox environments.
    • Explore Developer Sandbox

    Featured Developer Sandbox activities

    • Get started with your Developer Sandbox
    • OpenShift virtualization and application modernization using the Developer Sandbox
    • Explore all Developer Sandbox activities

    Ready to start developing apps?

    • Try at no cost
  • Blog
  • Events
  • Videos

Debugging function parameters with Dyninst

August 9, 2021
William Cohen
Related topics:
C, C#, C++Linux
Related products:
Red Hat Enterprise Linux

Share:

    As a part of my work at Red Hat, I verify the accuracy of the debugging information that maps between the executable binary generated by compilers and the original source written by the developer. Additionally, I look for complications in the debugging information that stem from compiler optimizations. It is possible to manually inspect the binary and review the debugging information for discrepancies. However, for significant applications, it would take too much time to manually review the megabytes of data. It's also too easy to overlook one drop of erroneous information hidden in the vast sea of correctly generated debugging information.

    To automate this type of analysis, you need a tool that analyzes both the debugging information and the binary executable. Dyninst, available in Fedora and Red Hat Enterprise Linux, provides a suite of dynamic and static analysis and instrumentation tools that you can use for this purpose. A previous Red Hat Developer article, Using the SystemTap Dyninst runtime environment, discussed Dyninst's dynamic instrumentation. This article demonstrates how to write a simplified static analyzer in Dyninst.

    Analyzing function parameters

    In the past, SystemTap "guru mode" has been used to create temporary security fixes known as security-band-aids. A number of these temporary fixes are implemented by changing the value of a function parameter when the function is first entered. For example, in the following cve-2012-0056.stp script, setting the number of bytes written (the $count parameter) to zero disables the mem_write:

    probe kernel.function("mem_write@fs/proc/base.c").call {
        $count = 0
    }
    

    For this type of fix to work, the parameter being modified needs to be used by later machine instructions in the function. SystemTap gets details about where $count and other parameters are from the debugging information. The location lists generated by the compiler describe the type for the parameter and where the parameter is located (for example, the variable value is in a processor register or memory). However, there is a crucial piece of information that the debugging information might not include: whether the parameter's value is actually read by any instructions. With compiler optimizations such as constant propagation, it is possible that a variable passed to a function may never be used, and a SystemTap guru mode Common Vulnerabilities and Exposures (CVE) fix isn't possible for those cases.

    For each processor, there is an application binary interface (ABI) that describes how parameters are passed to functions. The System V Application Binary Interface AMD64 Architecture Processor Supplement describes the ABI for x86_64. To simplify this example, we'll assume the binary code is just simple x86_64 integer code running on Linux where there are up to six parameters passed in registers. This assumption will fail when floating point values or structures are passed. If we know how many parameters are being passed to a function, we can infer which registers are being used. The following list shows which processor register contains each function parameter:

    parameter1 %rdi
    parameter2 %rsi
    parameter3 %rdx
    parameter4 %rcx
    parameter5 %r8
    parameter6 %r9
    

    Liveness analysis

    The other essential part of the analysis is computing which register values on entry to a function are used by the binary code. This is known as liveness analysis. Dyninst provides a liveness analyzer as part of the Dataflow API. The liveness analyzer takes a function, determines the possible paths taken through the code, and computes at each instruction in the function the set of registers that hold values that a later instruction could use. If there are no possible uses of the value currently held by the register by later instructions, the value is considered dead. The liveness analyzer produces results for the entire function, and then those results can be queried to determine whether a specific register at a particular instruction is live. For this example, we are just going to analyze the registers holding the parameters on entry to the function.

    Building the static analyzer

    Now it is time to look at the code snippets used to create the analyzer. First, we need to include the Dyninst header files for the classes being used and to state the namespaces being used to make the code a bit more compact:

    #include <dyninst/Symtab.h>
    #include <dyninst/Function.h>
    #include <dyninst/liveness.h>
    
    using namespace Dyninst;
    using namespace SymtabAPI;
    using namespace ParseAPI;
    using namespace std;

    The code will do a simple check to make sure that the program has exactly two arguments: argv[0] (the analyzer program) and arg[1] (the binary being analyzed). The file is opened, then debugging information and binary are read in:

    	if (argc != 2) exit(-1);
    
    	// Parse the object file
    	Symtab *obj = NULL;
    	bool err = Symtab::openFile(obj, argv[1]);
    
    	if( err == false) exit(-1);
    
    	// Create a new binary code object from the filename argument
    	SymtabCodeSource *sts = new SymtabCodeSource(argv[1]);
    	if(!sts) return -1;
    
    	CodeObject *co = new CodeObject(sts);
    	if(!co) return -1;
    

    At this point, we run through each of the functions in the binary with the following for-loop:

    	// Iterate through each of the functions.
    	for(auto f: co->funcs()) {
    		// ... body of analysis goes here
    	}
    

    Inside the for-loop, we need to set up and run the liveness analysis on the function:

    		// Perform the liveness analysis on function.
    		LivenessAnalyzer la(f->obj()->cs()->getAddressWidth());
    		la.analyze(f);
    

    The liveness analysis generates data structures that can be queried for information about a specific location in the code. In our analysis, there is just one place location that we are interested in: the entry to the function. This is the first instruction of the first basic block. A basic block is a group of instructions that are executed in sequence, so only the last instruction can change the program control flow with a jump, call, or return:

    		// Get the first instruction of the first basic block (function entry).
    		Block *bb = *f->blocks().begin();
    		Address curAddr = bb->start();
    		Instruction curInsn = bb->getInsn(curAddr);
    
    		// Construct a liveness query location for the function entry.
    		InsnLoc i(bb,  curAddr, curInsn);
    		Location loc(f, i);
    

    Now we need to look up the information about parameters associated with the function. The findFuncByEntryOffset method locates the symbolic debugging information for the function. It is possible that there is no symbol information for a function. If there is symbol information, the getParams method obtains the list of parameters to the function. In this case, the code is just counting the number of parameters:

    		// Get the formal parameters and count them.
    		SymtabAPI :: Function *func_sym;
    		bool found = obj->findFuncByEntryOffset(func_sym, curAddr);
    		if (!found) continue; // Missing symbols move on to next function
    		vector <localVar *> parms;
    		func_sym->getParams(parms);
    		int num_parms = parms.size();

    One last step is to create a table to convert the parameter number into the Dyninst register name storing the parameter. This is implemented with an array of MachRegister entries. The arg_register array at the beginning of the program holds the register Dyninst names used to pass arguments to functions. This has been written for x86_64, but it could be adapted to run on PowerPC or AARCH64 architectures that Dyninst also supports by changing the register names to match the arguments for those architectures:

    const MachRegister arg_register[] = {x86_64::rdi, x86_64::rsi, x86_64::rdx, x86_64::rcx, x86_64::r8, x86_64::r9};
    

    Analyzing the function

    We now have all the information we need to analyze the function. We will have the analyzer print out the function's address to make it easier to examine disassembled code followed by the function name. Now we have a loop that queries each of the parameters used by the function. The result is placed in the bool used. If used is false, the parameter (arg number) is printed:

          		// Output the results for the function
    		cout  << hex << curAddr << " " << f->name() << ": ";
    		for (int i=0; i<min(num_parms, num_reg_args); ++i) {
    			// Print up arg number if associated register is unused.
    			bool used;
    			la.query(loc, LivenessAnalyzer::Before, arg_register[i], used);
    			if (!used) 
    				cout << "arg" << i+1 << " ";
    		}
    		cout << endl;
    

    Here are the snippets of codes combined into the program:

    // Example DataFlowAPI program; notes which arguments on x86_64 functions are unused
    //
    // William E. Cohen (wcohen at redhat dot com)
    //
    
    #include <dyninst/Symtab.h>
    #include <dyninst/Function.h>
    #include <dyninst/liveness.h>
    
    using namespace Dyninst;
    using namespace SymtabAPI;
    using namespace ParseAPI;
    using namespace std;
    
    // This could be extended to other architectures by replacing arg_register[] entries with appropriate registers
    // Based on Figure 3.4: Register Usage of
    // https://web.archive.org/web/20160801075146/http://www.x86-64.org/documentation/abi.pdf
    const MachRegister arg_register[] = {x86_64::rdi, x86_64::rsi, x86_64::rdx, x86_64::rcx, x86_64::r8, x86_64::r9};
    
    int main(int argc, char **argv){
    	int num_reg_args = sizeof(arg_register)/sizeof(arg_register[0]);
    
    	if (argc != 2) exit(-1);
    
    	// Parse the object file
    	Symtab *obj = NULL;
    	bool err = Symtab::openFile(obj, argv[1]);
    
    	if( err == false) exit(-1);
    
    	// Create a new binary code object from the filename argument
    	SymtabCodeSource *sts = new SymtabCodeSource(argv[1]);
    	if(!sts) return -1;
    
    	CodeObject *co = new CodeObject(sts);
    	if(!co) return -1;
    
    	// Iterate through each of the functions.
    	for(auto f: co->funcs()) {
    		// Perform the liveness analysis on function.
    		LivenessAnalyzer la(f->obj()->cs()->getAddressWidth());
    		la.analyze(f);
    
    		// Get the first instruction of the first basic block (function entry).
    		Block *bb = *f->blocks().begin();
    		Address curAddr = bb->start();
    		Instruction curInsn = bb->getInsn(curAddr);
    
    		// Construct a liveness query location for the function entry.
    		InsnLoc i(bb,  curAddr, curInsn);
    		Location loc(f, i);
    
    		// Get the formal parameters and count them.
    		SymtabAPI :: Function *func_sym;
    		bool found = obj->findFuncByEntryOffset(func_sym, curAddr);
    		if (!found) continue; // Missing symbols move on to next function
    		vector <localVar *> parms;
    		func_sym->getParams(parms);
    		int num_parms = parms.size();
    
    		// Output the results for the function
    		cout  << hex << curAddr << " " << f->name() << ": ";
    		for (int i=0; i<min(num_parms, num_reg_args); ++i) {
    			// Print up arg number if associated register is unused.
    			bool used;
    			la.query(loc, LivenessAnalyzer::Before, arg_register[i], used);
    			if (!used) 
    				cout << "arg" << i+1 << " ";
    		}
    		cout << endl;
    	}
    	return (0);
    }

    To build the analysis, the code needs to be compiled and linked with the various Dyninst libraries with the following command line:

    g++ -O2 -g -std=c++17  -L /usr/lib64/dyninst \
    -l parseAPI -l symtabAPI -l instructionAPI \
    -l tbb -l common  unused_arg.C   -o unused_arg
    

    Analyzing a binary for unused function arguments

    We want to test and see that the analyzer works as expected. The following example is a test program that has a main function that calls a couple of other functions: foo and bar. The functions are marked with __attribute__ ((noinline)) to ensure that the compiler doesn't attempt to inline the simple functions as the unused_arg analyzer isn't going to work for inlined functions:

    #include <stdio.h>
    
    int __attribute__ ((noinline))
    foo(int a, int b)
    {
        return 0;
    }
    
    int __attribute__ ((noinline))
    bar(int a, int b)
    {
        return b;
    }
    
    int main(int argc, char *argv[])
    {
        printf("foo(1,2) = %d\n", foo(1,2));
        printf("bar(1,2) = %d\n", bar(1,2));
        return 0;
    }
    

    We build the test program with:

    gcc -O2 -g test_unused.c   -o test_unused

    Inspecting the preceding code, we would expect that neither parameter 1 or 2 for foo would be used. Function bar actually uses the second argument, so only parameter 1 is unused. Finally, function main doesn't use either of its parameters (arguments). The following is the unused_arg analysis of the test_unused program showing the expected results with additional internal functions listed:

    $ ./unused_arg ./test_unused
    401000 _init: 
    401030 printf: 
    401040 main: arg1 arg2 
    401090 _start: 
    4010c0 _dl_relocate_static_pie: 
    4010d0 deregister_tm_clones: 
    401100 register_tm_clones: 
    401140 __do_global_dtors_aux: 
    401170 frame_dummy: 
    401180 foo: arg1 arg2 
    401190 bar: arg1 
    4011a0 __libc_csu_init: 
    401210 __libc_csu_fini: 
    401218 _fini:

    Summary

    This article showed how you can use Dyninst to better understand the characteristics of programs. Dyninst provides very powerful tools that analyze both debugging information and binaries. Take a look at Dyninst examples for additional static analysis control-flow graphs (CFGs) and dynamic instrumentation for code coverage examples. For additional information about the tool, check out the upstream Dyninst website.

    Last updated: October 6, 2022

    Related Posts

    • Using the SystemTap Dyninst runtime environment

    • Automating the testing process for SystemTap, Part 1: Test automation with libvirt and Buildbot

    • Analyzing and reducing SystemTap's startup cost for scripts

    Recent Posts

    • AI meets containers: My first step into Podman AI Lab

    • Live migrating VMs with OpenShift Virtualization

    • Storage considerations for OpenShift Virtualization

    • Upgrade from OpenShift Service Mesh 2.6 to 3.0 with Kiali

    • EE Builder with Ansible Automation Platform on OpenShift

    Red Hat Developers logo LinkedIn YouTube Twitter Facebook

    Products

    • Red Hat Enterprise Linux
    • Red Hat OpenShift
    • Red Hat Ansible Automation Platform

    Build

    • Developer Sandbox
    • Developer Tools
    • Interactive Tutorials
    • API Catalog

    Quicklinks

    • Learning Resources
    • E-books
    • Cheat Sheets
    • Blog
    • Events
    • Newsletter

    Communicate

    • About us
    • Contact sales
    • Find a partner
    • Report a website issue
    • Site Status Dashboard
    • Report a security problem

    RED HAT DEVELOPER

    Build here. Go anywhere.

    We serve the builders. The problem solvers who create careers with code.

    Join us if you’re a developer, software engineer, web designer, front-end designer, UX designer, computer scientist, architect, tester, product manager, project manager or team lead.

    Sign me up

    Red Hat legal and privacy links

    • About Red Hat
    • Jobs
    • Events
    • Locations
    • Contact Red Hat
    • Red Hat Blog
    • Inclusion at Red Hat
    • Cool Stuff Store
    • Red Hat Summit
    © 2025 Red Hat

    Red Hat legal and privacy links

    • Privacy statement
    • Terms of use
    • All policies and guidelines
    • Digital accessibility

    Report a website issue