This article explores how you can leverage application binary interface (ABI) compatibility testing with libabigail to determine the functionality of binaries running on older distributions or versions.
Software evolution and compatibility
Linux distributions are a living thing. During their lifetime, they are born at release, receive updates, and eventually "die" when support ends. They spawn descendant distributions. They sometimes even fight! The whole point of the distribution is to make an interdependent, cohesive whole, so there are many shared components. When those shared components are changed within a distribution, the question of compatibility is imminent.
Application binary interface (ABI) compatibility is a specific type of dependency: between separately compiled binaries and shared libraries. If the provider or the user of any interface—function call? variable type?—is changed, will the complete system still work at run time? The answer is an easy YES if you recompile the world, but that is not practical (unless you're into Gentoo.) You end up with a new library and an old binary (or vice versa.) A well-written library will exhibit discipline: using symbol versioning for functions, limiting or eliminating visible data type changes. But, even with the necessary expertise, mistakes can happen. You will wonder: will this still run? You can test the program and hope for the best, but a successful run doesn't guarantee that every code path will mesh its new counterpart. A mismatch can cause crashes or data corruption later.
libabigail for static ABI compatibility testing
Instead of haphazard testing, you need tools for systematic comparison of the two sides of the linkage interface between binaries. From this need, libabigail was built in 2013. It includes tools to analyze the debugging data of each binary, to infer fine-grained ABI data on both sides of an interface: what a caller uses, and what the callee provides. Another tool compares two alternative versions of the same binary or library, to answer whether they are substitutable—whether they provide the same interface. Note that libabigail compatibility checks currently focus on function presence, and type compatibility related to the layout of complex variables in memory. These are the most common sources of incompatibility that occur with evolving software. Other aspects of ABI compatibility are also being investigated.
But operating that tool requires direct access to the two or more candidate binaries. This may be easy if you just compiled one of them, and it's sitting in a new build tree. Or it may be harder, if you're trying to compare two versions of a distribution package. That would mean they have the same path and they can't be installed concurrently. Actually, I lied a little. libabigail added a mode where it can make and take an XML document form instead of a binary plus debuginfo. But for that, one needs to have the XML data for the binary or library being compared. If only someone figured out how to make it easy to collect and share all that XML!
Introducing abidb
One way is with a new tool, abidb, which is shipped with libabigail version 2.5. It is a wrapper on the libabigail tools to make and use ABI XML documents. The "db" part of the name stands for "database", and relates to its method of archiving these ABI XML documents: in a git repository. The repository is structured so that an entire Linux distribution's shared libraries worth of XML files may be imported into a single branch. Each version of each shared library gets a unique file name (derived from the library's file path, its ELF SONAME, and its unique build-id hash). That means that different versions of the same library will be siblings in the git filesystem. Many Linux distributions can coexist in the same git repo, on different branches. Because of the genius and efficiency of git, they can all be stored together, compressed fairly efficiently.
This first version of abidb has two modes. abidb --submit /path/to/libfoo.so generates an ABI XML document for the given file using libabigail's abidw, and deposits it into an appropriate place in the current git repository. abidb can also process entire RPM/DEB archive files, and run the "submit" operation on each shared library found inside. If debuginfo is not immediately available, the tool consults any configured debuginfo servers to fetch them automatically. You may rerun to add data for new versions installed since last time. This results in a Git branch that you may push to a central repository, or use in situ. All the usual git tools and techniques work to manage distribution, control, and garbage collection. Figure 1 depicts this mode.
The other mode is abidb --check BINARY, as shown in Figure 2. This mode takes a given binary, identifies all the shared libraries it uses (by the outgoing DT_NEEDED SONAME references). Then, using the libabigail abidiff tool, it then compares the binary to the ABI XML of each shared library with the same SONAME from the git repo. If this reveals any ABI mismatch, then details are printed, and you will find out that some version of a shared library is not in fact compatible with the binary. Note that it's not necessary to install the shared library files at all: just get a copy of the XML git repo from somewhere. In fact, the comparison target need not be the same distribution you're running, as long as someone collected its ABI XML bits and merged them into your git repo.
We anticipate other modes coming on line soon, such as comparing shared library versions directly to each other, based on their ABI XML. This could assist distribution maintainers who need to maintain ABI compatibility guarantees, by comparing new builds to all relevant previous ones. It could be a part of build pipeline automation. This may assist certain multiplatform binary packaging efforts like AppImage.
abidb public repo
To accompany the abidb tool, we are prototyping a public git repo of ABI XML data for a variety of Linux distributions. We have started populating this with some of the distributions closest to our hearts, and would love to extend it with others. Simple shell scripts can take installation media or a bunch of archives, and import an entire distribution into an abidb Git repo over a few hours. If you wish to publish that content to the world, rather than using it privately, we may be able to help by merging it into our public abidb.git repo on sourceware.org. Get in touch at libabigail@sourceware.org and help crowdsource.
