How to create Python binding for a Rust library

This article continues a series about how to take advantage of the recent Rust support added to Linux. The previous articles in the series are:

• 3 essentials for writing a Linux system library in Rust (part 1)
• How to create C binding for a Rust library (part 2)
• Build trust in continuous integration for your Rust library (part 4)

This third installment demonstrates how to create Python bindings so that Python projects can use your Rust library.

You can download the demo code from its GitHub repository. The package contains:

• An echo server listening on the Unix socket /tmp/librabc
• A Rust crate that connects to the socket and sends a ping packet every 2 seconds
• A C/Python binding
• A command-line interface (CLI) for the client

Elements of a Python binding

The PyO3 project can generate a Python-compiled extension. But I personally like to wrap the C library shown in the previous section into a Python module using the ctypes module instead of depending on the big, feature-rich PyO3 project.

You can check the full code of my binding in the GitHub repository. The basic workflow in the code is:

1. Use ctypes.cdll.LoadLibrary("librabc.so.0") to load the C library.
2. Implement __init__() for class RabcClient to start the connection.
3. Pass output pointers to C functions using ctyps.byref().
4. Free the memory used by the logs, error messages, etc.
5. Implement __del__() for the RabcClientclass to drop the connection.

Unlike pure Python projects, when using the C library, your need to take care of memory management. We will look at memory management for various types.

Output pointer to a C string

The following Python code creates a pointer to a string:

foo = ctypes.c_char_p()

The code is equivalent to the following C code:

char * foo = NULL

The Python function byref(foo) is equivalent to (char **) & foo in C. In order to store the output pointer to the C string, use bytes.decode() to copy and convert the content to Python string, then free the C memory:

c_reply = c_char_p()
rc = lib.rabc_client_process(
    # Many lines omitted
    ctypes.byref(c_reply),
)

if reply:
    reply = c_reply.value.decode("utf-8")
    lib.rabc_cstring_free(c_reply)
    return reply

Output pointer to a C opaque struct

An opaque struct in C does not have a struct definition in the public header, so there is no field or size information available for the struct. The following excerpt from the demo code shows how to use such a struct in Python:

# Opaque struct
class _ClibRabcClient(ctypes.Structure):
    pass

class RabcClient:
    def __init__(self):
        self._c_pointer = ctypes.POINTER(_ClibRabcClient)()
        # Many lines omitted
        rc = lib.rabc_client_new(
            ctypes.byref(self._c_pointer),
            # Many arguments omitted
        )

    def __del__(self):
        if self._c_pointer:
            lib.rabc_client_free(self._c_pointer)

The clause ctypes.POINTER(_ClibRabcClient)() is equivalent to struct rabc_client *client = NULL in C.

Wrap the variable in ctypes.byref() to use it as an output pointer.

Output pointer to an integer array

Unlike an opaque struct, Python knows the memory size of an integer. So once you have the memory address of the first element and the length of an integer array, you can iterate over the array's contents using (c_uint64 * event_count.value).from_address().

This technique is illustrated in the following example code:

c_events = ctypes.POINTER(c_uint64)()
event_count = c_uint64(0)
rc = lib.rabc_client_poll(
    c_uint32(wait_time),
    ctypes.byref(c_events),
    # Many arguments omitted
)

# Many lines omitted

events = list(
    (c_uint64 * event_count.value).from_address(
        ctypes.addressof(c_events.contents)
    )
)
lib.rabc_events_free(c_events, event_count)
return events

Bindings allow multiple languages to use a Rust library

The general procedure in this series is to create a Rust library along with a thread-safe and memory-safe C binding. We then used Python code for a Python binding, invoking the C library. With this workflow, you need to deal only with the memory and structure differences between Rust and C. Fortunately, Rust handles the differences very smoothly. We hope you found this series informative.