Skip to main content
Redhat Developers  Logo
  • AI

    Get started with AI

    • Red Hat AI
      Accelerate the development and deployment of enterprise AI solutions.
    • AI learning hub
      Explore learning materials and tools, organized by task.
    • AI interactive demos
      Click through scenarios with Red Hat AI, including training LLMs and more.
    • AI/ML learning paths
      Expand your OpenShift AI knowledge using these learning resources.
    • AI quickstarts
      Focused AI use cases designed for fast deployment on Red Hat AI platforms.
    • No-cost AI training
      Foundational Red Hat AI training.

    Featured resources

    • OpenShift AI learning
    • Open source AI for developers
    • AI product application development
    • Open source-powered AI/ML for hybrid cloud
    • AI and Node.js cheat sheet

    Red Hat AI Factory with NVIDIA

    • Red Hat AI Factory with NVIDIA is a co-engineered, enterprise-grade AI solution for building, deploying, and managing AI at scale across hybrid cloud environments.
    • Explore the solution
  • Learn

    Self-guided

    • Documentation
      Find answers, get step-by-step guidance, and learn how to use Red Hat products.
    • Learning paths
      Explore curated walkthroughs for common development tasks.
    • See all learning

    Hands-on

    • Developer Sandbox
      Spin up Red Hat's products and technologies without setup or configuration.
    • Interactive labs
      Learn by doing in these hands-on, browser-based experiences.
    • Interactive demos
      Click through product features in these guided tours.

    Browse by topic

    • AI/ML
    • Automation
    • Java
    • Kubernetes
    • Linux
    • See all topics

    Training & certifications

    • Courses and exams
    • Certifications
    • Skills assessments
    • Red Hat Academy
    • Learning subscription
    • Explore training
  • Build

    Get started

    • Red Hat build of Podman Desktop
      A downloadable, local development hub to experiment with our products and builds.
    • Developer Sandbox
      Spin up Red Hat's products and technologies without setup or configuration.

    Download products

    • Access product downloads to start building and testing right away.
    • Red Hat Enterprise Linux
    • Red Hat AI
    • Red Hat OpenShift
    • Red Hat Ansible Automation Platform
    • See all products

    Featured

    • Red Hat build of OpenJDK
    • Red Hat JBoss Enterprise Application Platform
    • Red Hat OpenShift Dev Spaces
    • Red Hat Developer Toolset

    References

    • E-books
    • Documentation
    • Cheat sheets
    • Architecture center
  • Community

    Get involved

    • Events
    • Live AI events
    • Red Hat Summit
    • Red Hat Accelerators
    • Community discussions

    Follow along

    • Articles & blogs
    • Developer newsletter
    • Videos
    • Github

    Get help

    • Customer service
    • Customer support
    • Regional contacts
    • Find a partner

    Join the Red Hat Developer program

    • Download Red Hat products and project builds, access support documentation, learning content, and more.
    • Explore the benefits

Debugging function parameters with Dyninst

August 9, 2021
William Cohen
Related topics:
C, C#, C++Linux
Related products:
Red Hat Enterprise Linux

    As a part of my work at Red Hat, I verify the accuracy of the debugging information that maps between the executable binary generated by compilers and the original source written by the developer. Additionally, I look for complications in the debugging information that stem from compiler optimizations. It is possible to manually inspect the binary and review the debugging information for discrepancies. However, for significant applications, it would take too much time to manually review the megabytes of data. It's also too easy to overlook one drop of erroneous information hidden in the vast sea of correctly generated debugging information.

    To automate this type of analysis, you need a tool that analyzes both the debugging information and the binary executable. Dyninst, available in Fedora and Red Hat Enterprise Linux, provides a suite of dynamic and static analysis and instrumentation tools that you can use for this purpose. A previous Red Hat Developer article, Using the SystemTap Dyninst runtime environment, discussed Dyninst's dynamic instrumentation. This article demonstrates how to write a simplified static analyzer in Dyninst.

    Analyzing function parameters

    In the past, SystemTap "guru mode" has been used to create temporary security fixes known as security-band-aids. A number of these temporary fixes are implemented by changing the value of a function parameter when the function is first entered. For example, in the following cve-2012-0056.stp script, setting the number of bytes written (the $count parameter) to zero disables the mem_write:

    probe kernel.function("mem_write@fs/proc/base.c").call {
        $count = 0
    }
    

    For this type of fix to work, the parameter being modified needs to be used by later machine instructions in the function. SystemTap gets details about where $count and other parameters are from the debugging information. The location lists generated by the compiler describe the type for the parameter and where the parameter is located (for example, the variable value is in a processor register or memory). However, there is a crucial piece of information that the debugging information might not include: whether the parameter's value is actually read by any instructions. With compiler optimizations such as constant propagation, it is possible that a variable passed to a function may never be used, and a SystemTap guru mode Common Vulnerabilities and Exposures (CVE) fix isn't possible for those cases.

    For each processor, there is an application binary interface (ABI) that describes how parameters are passed to functions. The System V Application Binary Interface AMD64 Architecture Processor Supplement describes the ABI for x86_64. To simplify this example, we'll assume the binary code is just simple x86_64 integer code running on Linux where there are up to six parameters passed in registers. This assumption will fail when floating point values or structures are passed. If we know how many parameters are being passed to a function, we can infer which registers are being used. The following list shows which processor register contains each function parameter:

    parameter1 %rdi
    parameter2 %rsi
    parameter3 %rdx
    parameter4 %rcx
    parameter5 %r8
    parameter6 %r9
    

    Liveness analysis

    The other essential part of the analysis is computing which register values on entry to a function are used by the binary code. This is known as liveness analysis. Dyninst provides a liveness analyzer as part of the Dataflow API. The liveness analyzer takes a function, determines the possible paths taken through the code, and computes at each instruction in the function the set of registers that hold values that a later instruction could use. If there are no possible uses of the value currently held by the register by later instructions, the value is considered dead. The liveness analyzer produces results for the entire function, and then those results can be queried to determine whether a specific register at a particular instruction is live. For this example, we are just going to analyze the registers holding the parameters on entry to the function.

    Building the static analyzer

    Now it is time to look at the code snippets used to create the analyzer. First, we need to include the Dyninst header files for the classes being used and to state the namespaces being used to make the code a bit more compact:

    #include <dyninst/Symtab.h>
    #include <dyninst/Function.h>
    #include <dyninst/liveness.h>
    
    using namespace Dyninst;
    using namespace SymtabAPI;
    using namespace ParseAPI;
    using namespace std;

    The code will do a simple check to make sure that the program has exactly two arguments: argv[0] (the analyzer program) and arg[1] (the binary being analyzed). The file is opened, then debugging information and binary are read in:

    	if (argc != 2) exit(-1);
    
    	// Parse the object file
    	Symtab *obj = NULL;
    	bool err = Symtab::openFile(obj, argv[1]);
    
    	if( err == false) exit(-1);
    
    	// Create a new binary code object from the filename argument
    	SymtabCodeSource *sts = new SymtabCodeSource(argv[1]);
    	if(!sts) return -1;
    
    	CodeObject *co = new CodeObject(sts);
    	if(!co) return -1;
    

    At this point, we run through each of the functions in the binary with the following for-loop:

    	// Iterate through each of the functions.
    	for(auto f: co->funcs()) {
    		// ... body of analysis goes here
    	}
    

    Inside the for-loop, we need to set up and run the liveness analysis on the function:

    		// Perform the liveness analysis on function.
    		LivenessAnalyzer la(f->obj()->cs()->getAddressWidth());
    		la.analyze(f);
    

    The liveness analysis generates data structures that can be queried for information about a specific location in the code. In our analysis, there is just one place location that we are interested in: the entry to the function. This is the first instruction of the first basic block. A basic block is a group of instructions that are executed in sequence, so only the last instruction can change the program control flow with a jump, call, or return:

    		// Get the first instruction of the first basic block (function entry).
    		Block *bb = *f->blocks().begin();
    		Address curAddr = bb->start();
    		Instruction curInsn = bb->getInsn(curAddr);
    
    		// Construct a liveness query location for the function entry.
    		InsnLoc i(bb,  curAddr, curInsn);
    		Location loc(f, i);
    

    Now we need to look up the information about parameters associated with the function. The findFuncByEntryOffset method locates the symbolic debugging information for the function. It is possible that there is no symbol information for a function. If there is symbol information, the getParams method obtains the list of parameters to the function. In this case, the code is just counting the number of parameters:

    		// Get the formal parameters and count them.
    		SymtabAPI :: Function *func_sym;
    		bool found = obj->findFuncByEntryOffset(func_sym, curAddr);
    		if (!found) continue; // Missing symbols move on to next function
    		vector <localVar *> parms;
    		func_sym->getParams(parms);
    		int num_parms = parms.size();

    One last step is to create a table to convert the parameter number into the Dyninst register name storing the parameter. This is implemented with an array of MachRegister entries. The arg_register array at the beginning of the program holds the register Dyninst names used to pass arguments to functions. This has been written for x86_64, but it could be adapted to run on PowerPC or AARCH64 architectures that Dyninst also supports by changing the register names to match the arguments for those architectures:

    const MachRegister arg_register[] = {x86_64::rdi, x86_64::rsi, x86_64::rdx, x86_64::rcx, x86_64::r8, x86_64::r9};
    

    Analyzing the function

    We now have all the information we need to analyze the function. We will have the analyzer print out the function's address to make it easier to examine disassembled code followed by the function name. Now we have a loop that queries each of the parameters used by the function. The result is placed in the bool used. If used is false, the parameter (arg number) is printed:

          		// Output the results for the function
    		cout  << hex << curAddr << " " << f->name() << ": ";
    		for (int i=0; i<min(num_parms, num_reg_args); ++i) {
    			// Print up arg number if associated register is unused.
    			bool used;
    			la.query(loc, LivenessAnalyzer::Before, arg_register[i], used);
    			if (!used) 
    				cout << "arg" << i+1 << " ";
    		}
    		cout << endl;
    

    Here are the snippets of codes combined into the program:

    // Example DataFlowAPI program; notes which arguments on x86_64 functions are unused
    //
    // William E. Cohen (wcohen at redhat dot com)
    //
    
    #include <dyninst/Symtab.h>
    #include <dyninst/Function.h>
    #include <dyninst/liveness.h>
    
    using namespace Dyninst;
    using namespace SymtabAPI;
    using namespace ParseAPI;
    using namespace std;
    
    // This could be extended to other architectures by replacing arg_register[] entries with appropriate registers
    // Based on Figure 3.4: Register Usage of
    // https://web.archive.org/web/20160801075146/http://www.x86-64.org/documentation/abi.pdf
    const MachRegister arg_register[] = {x86_64::rdi, x86_64::rsi, x86_64::rdx, x86_64::rcx, x86_64::r8, x86_64::r9};
    
    int main(int argc, char **argv){
    	int num_reg_args = sizeof(arg_register)/sizeof(arg_register[0]);
    
    	if (argc != 2) exit(-1);
    
    	// Parse the object file
    	Symtab *obj = NULL;
    	bool err = Symtab::openFile(obj, argv[1]);
    
    	if( err == false) exit(-1);
    
    	// Create a new binary code object from the filename argument
    	SymtabCodeSource *sts = new SymtabCodeSource(argv[1]);
    	if(!sts) return -1;
    
    	CodeObject *co = new CodeObject(sts);
    	if(!co) return -1;
    
    	// Iterate through each of the functions.
    	for(auto f: co->funcs()) {
    		// Perform the liveness analysis on function.
    		LivenessAnalyzer la(f->obj()->cs()->getAddressWidth());
    		la.analyze(f);
    
    		// Get the first instruction of the first basic block (function entry).
    		Block *bb = *f->blocks().begin();
    		Address curAddr = bb->start();
    		Instruction curInsn = bb->getInsn(curAddr);
    
    		// Construct a liveness query location for the function entry.
    		InsnLoc i(bb,  curAddr, curInsn);
    		Location loc(f, i);
    
    		// Get the formal parameters and count them.
    		SymtabAPI :: Function *func_sym;
    		bool found = obj->findFuncByEntryOffset(func_sym, curAddr);
    		if (!found) continue; // Missing symbols move on to next function
    		vector <localVar *> parms;
    		func_sym->getParams(parms);
    		int num_parms = parms.size();
    
    		// Output the results for the function
    		cout  << hex << curAddr << " " << f->name() << ": ";
    		for (int i=0; i<min(num_parms, num_reg_args); ++i) {
    			// Print up arg number if associated register is unused.
    			bool used;
    			la.query(loc, LivenessAnalyzer::Before, arg_register[i], used);
    			if (!used) 
    				cout << "arg" << i+1 << " ";
    		}
    		cout << endl;
    	}
    	return (0);
    }

    To build the analysis, the code needs to be compiled and linked with the various Dyninst libraries with the following command line:

    g++ -O2 -g -std=c++17  -L /usr/lib64/dyninst \
    -l parseAPI -l symtabAPI -l instructionAPI \
    -l tbb -l common  unused_arg.C   -o unused_arg
    

    Analyzing a binary for unused function arguments

    We want to test and see that the analyzer works as expected. The following example is a test program that has a main function that calls a couple of other functions: foo and bar. The functions are marked with __attribute__ ((noinline)) to ensure that the compiler doesn't attempt to inline the simple functions as the unused_arg analyzer isn't going to work for inlined functions:

    #include <stdio.h>
    
    int __attribute__ ((noinline))
    foo(int a, int b)
    {
        return 0;
    }
    
    int __attribute__ ((noinline))
    bar(int a, int b)
    {
        return b;
    }
    
    int main(int argc, char *argv[])
    {
        printf("foo(1,2) = %d\n", foo(1,2));
        printf("bar(1,2) = %d\n", bar(1,2));
        return 0;
    }
    

    We build the test program with:

    gcc -O2 -g test_unused.c   -o test_unused

    Inspecting the preceding code, we would expect that neither parameter 1 or 2 for foo would be used. Function bar actually uses the second argument, so only parameter 1 is unused. Finally, function main doesn't use either of its parameters (arguments). The following is the unused_arg analysis of the test_unused program showing the expected results with additional internal functions listed:

    $ ./unused_arg ./test_unused
    401000 _init: 
    401030 printf: 
    401040 main: arg1 arg2 
    401090 _start: 
    4010c0 _dl_relocate_static_pie: 
    4010d0 deregister_tm_clones: 
    401100 register_tm_clones: 
    401140 __do_global_dtors_aux: 
    401170 frame_dummy: 
    401180 foo: arg1 arg2 
    401190 bar: arg1 
    4011a0 __libc_csu_init: 
    401210 __libc_csu_fini: 
    401218 _fini:

    Summary

    This article showed how you can use Dyninst to better understand the characteristics of programs. Dyninst provides very powerful tools that analyze both debugging information and binaries. Take a look at Dyninst examples for additional static analysis control-flow graphs (CFGs) and dynamic instrumentation for code coverage examples. For additional information about the tool, check out the upstream Dyninst website.

    Last updated: October 6, 2022

    Related Posts

    • Using the SystemTap Dyninst runtime environment

    • Automating the testing process for SystemTap, Part 1: Test automation with libvirt and Buildbot

    • Analyzing and reducing SystemTap's startup cost for scripts

    Recent Posts

    • Confidential virtual machine storage attack scenarios

    • Introducing virtualization platform autopilot

    • Integrate zero trust workload identity manager with Red Hat OpenShift GitOps

    • Best Practice Configuration and Tuning for Linux and Windows VMs

    • Red Hat UBI 8 builders have been promoted to the Paketo Buildpacks organization

    Red Hat Developers logo LinkedIn YouTube Twitter Facebook

    Platforms

    • Red Hat AI
    • Red Hat Enterprise Linux
    • Red Hat OpenShift
    • Red Hat Ansible Automation Platform
    • See all products

    Build

    • Developer Sandbox
    • Developer tools
    • Interactive tutorials
    • API catalog

    Quicklinks

    • Learning resources
    • E-books
    • Cheat sheets
    • Blog
    • Events
    • Newsletter

    Communicate

    • About us
    • Contact sales
    • Find a partner
    • Report a website issue
    • Site status dashboard
    • Report a security problem

    RED HAT DEVELOPER

    Build here. Go anywhere.

    We serve the builders. The problem solvers who create careers with code.

    Join us if you’re a developer, software engineer, web designer, front-end designer, UX designer, computer scientist, architect, tester, product manager, project manager or team lead.

    Sign me up

    Red Hat legal and privacy links

    • About Red Hat
    • Jobs
    • Events
    • Locations
    • Contact Red Hat
    • Red Hat Blog
    • Inclusion at Red Hat
    • Cool Stuff Store
    • Red Hat Summit
    © 2026 Red Hat

    Red Hat legal and privacy links

    • Privacy statement
    • Terms of use
    • All policies and guidelines
    • Digital accessibility

    Chat Support

    Please log in with your Red Hat account to access chat support.