Item 34: Control what crosses FFI boundaries

Even though Rust comes with a comprehensive standard library and a burgeoning crate ecosystem, there is still a lot more non-Rust code in the world than there is Rust code.

As with other recent languages, Rust helps with this problem by offering a foreign function interface (FFI) mechanism, which allows interoperation with code and data structures written in different languages—despite the name, FFI is not restricted to just functions. This opens up the use of existing libraries in different languages, not just those that have succumbed to the Rust community's efforts to "rewrite it in Rust" (RiiR).

The default target for Rust's interoperability is the C programming language, which is the same interop target that other languages aim at. This is partly driven by the ubiquity of C libraries but is also driven by simplicity: C acts as a "least common denominator" of interoperability, because it doesn't need toolchain support of any of the more advanced features that would be necessary for compatibility with other languages (e.g., garbage collection for Java or Go, exceptions and templates for C++, function overrides for Java and C++, etc.).

However, that's not to say that interoperability with plain C is simple. By including code written in a different language, all of the guarantees and protections that Rust offers are up for grabs, particularly those involving memory safety.

As a result, FFI code in Rust is automatically unsafe, and the advice in Item 16 has to be bypassed. This Item explores some replacement advice, and Item 35 will explore some tooling that helps to avoid some (but not all) of the footguns involved in working with FFI. (The FFI chapter of the Rustonomicon also contains helpful advice and information.)

Invoking C Functions from Rust

The simplest FFI interaction is for Rust code to invoke a C function, taking "immediate" arguments that don't involve pointers, references, or memory addresses:

/* File lib.c */
#include "lib.h"

/* C function definition. */
int add(int x, int y) {
  return x + y;
}

This C code provides a definition of the function and is typically accompanied by a header file that provides a declaration of the function, which allows other C code to use it:

/* File lib.h */
#ifndef LIB_H
#define LIB_H

/* C function declaration. */
int add(int x, int y);

#endif  /* LIB_H */

The declaration roughly says: somewhere out there is a function called add, which takes two integers as input and returns another integer as output. This allows C code to use the add function, subject to a promise that the actual code for add will be provided at a later date—specifically, at link time.

Rust code that wants to use add needs to have a similar declaration, with a similar purpose: to describe the signature of the function and to indicate that the corresponding code will be available later:

#![allow(unused)]
fn main() {
use std::os::raw::c_int;
extern "C" {
    pub fn add(x: c_int, y: c_int) -> c_int;
}
}

The declaration is marked as extern "C" to indicate that an external C library will provide the code for the function.¹ The extern "C" marker also automatically marks the function as #[no_mangle], which we explore in a following section.

Linking logistics

The details of how the C toolchain generates an external C library—and its format—are environment-specific and beyond the scope of a Rust book like this. However, one simple variant that's common on Unix-like systems is a static library file, which will normally have the form lib<something>.a (e.g., libcffi.a) and which can be generated using the ar tool.

The Rust build system then needs an indication of which library holds the relevant C code. This can be specified either via the link attribute in the code:

#[link(name = "cffi")] // An external library like `libcffi.a` is needed
extern "C" {
    // ...
}

or via a build script that emits a cargo:rustc-link-lib instruction to cargo:²

// File build.rs
fn main() {
    // An external library like `libcffi.a` is needed
    println!("cargo:rustc-link-lib=cffi");
}

The latter option is more flexible, because the build script can examine its environment and behave differently depending on what it finds.

In either case, the Rust build system is also likely to need information about how to find the C library, if it's not in a standard system location. This can be specified by having a build script that emits a cargo:rustc-link-search instruction to cargo, containing the library location:

// File build.rs
fn main() {
    // ...

    // Retrieve the location of `Cargo.toml`.
    let dir = std::env::var("CARGO_MANIFEST_DIR").unwrap();
    // Look for native libraries one directory higher up.
    println!(
        "cargo:rustc-link-search=native={}",
        std::path::Path::new(&dir).join("..").display()
    );
}

Code concerns

Returning to the source code, even this simplest of examples comes with some gotchas. First, use of FFI functions is automatically unsafe:

let x = add(1, 1);

error[E0133]: call to unsafe function is unsafe and requires unsafe function
              or block
   --> src/main.rs:176:13
    |
176 |     let x = add(1, 1);
    |             ^^^^^^^^^ call to unsafe function
    |
    = note: consult the function's documentation for information on how to
            avoid undefined behavior

and so needs to be wrapped in unsafe { }.

The next thing to watch out for is the use of C's int type, represented as std::os::raw::c_int. How big is an int? It's probably true that the following two things are the same:

• The size of an int for the toolchain that compiled the C library
• The size of a std::os::raw::c_int for the Rust toolchain

But why take the chance? Prefer sized types at FFI boundaries, where possible—which for C means making use of the types (e.g., uint32_t) defined in <stdint.h>. However, if you're dealing with an existing codebase that already uses int/long/size_t, this may be a luxury you don't have.

The final practical concern is that the C code and the equivalent Rust declaration need to exactly match. Worse still, if there's a mismatch, the build tools will not emit a warning—they will just silently emit incorrect code.

Item 35 discusses the use of the bindgen tool to prevent this problem, but it's worth understanding the basics of what's going on under the covers to understand why the build tools can't detect the problem on their own. In particular, it's worth understanding the basics of name mangling.

Name mangling

Compiled languages generally support separate compilation, where different parts of the program are converted into machine code as separate chunks (object files), which can then be combined into a complete program by the linker. This means that if only one small part of the program's source code changes, only the corresponding object file needs to be regenerated; the link step then rebuilds the program, combining both the changed object and all the other unmodified objects.

The link step is (roughly speaking) a "join-the-dots" operation: some object files provide definitions of functions and variables, and other object files have placeholder markers indicating that they expect to use a definition from some other object, but it wasn't available at compile time. The linker combines the two: it ensures that any placeholder in the compiled code is replaced with a reference to the corresponding concrete definition.

The linker performs this correlation between the placeholders and the definitions by simply checking for a matching name, meaning that there is a single global namespace for all of these correlations.

Historically, this was fine for linking C language programs, where a single name could not be reused in any way—the name of a function is exactly what appears in the object file. (As a result, a common convention for C libraries is to manually add a prefix to all symbols so that lib1_process doesn't clash with lib2_process.)

However, the introduction of C++ caused a problem because C++ allows overridden definitions with the same name:

// C++ code
namespace ns1 {
int32_t add(int32_t a, int32_t b) { return a+b; }
int64_t add(int64_t a, int64_t b) { return a+b; }
}
namespace ns2 {
int32_t add(int32_t a, int32_t b) { return a+b; }
}

The solution for this is name mangling: the compiler encodes the signature and type information for the overridden functions into the name that's emitted in the object file, and the linker continues to perform its simple-minded 1:1 correlation between placeholders and definitions.

On Unix-like systems, the nm tool can help show what the linker works with:

% nm ffi-lib.o | grep add  # what the linker sees for C
0000000000000000 T _add

% nm ffi-cpp-lib.o | grep add  # what the linker sees for C++
0000000000000000 T __ZN3ns13addEii
0000000000000020 T __ZN3ns13addExx
0000000000000040 T __ZN3ns23addEii

In this case, it shows three mangled symbols, all of which refer to code (the T indicates the text section of the binary, which is the traditional name for where code lives).

The c++filt tool helps translate this back into what would be visible in C++ code:

% nm ffi-cpp-lib.o | grep add | c++filt  # what the programmer sees
0000000000000000 T ns1::add(int, int)
0000000000000020 T ns1::add(long long, long long)
0000000000000040 T ns2::add(int, int)

Because the mangled name includes type information, the linker can and will complain about any mismatch in the type information between placeholder and definition. This gives some measure of type safety: if the definition changes but the place using it is not updated, the toolchain will complain.

Returning to Rust, extern "C" foreign functions are implicitly marked as #[no_mangle], and the symbol in the object file is the bare name, exactly as it would be for a C program. This means that the type safety of function signatures is lost: because the linker sees only the bare names for functions, if there are any differences in type expectations between definition and use, the linker will carry on regardless and problems will arise only at runtime.

Accessing C Data from Rust

The C add example in the previous section passed the simplest possible type of data back and forth between Rust and C: an integer that fits in a machine register. Even so, there were still things to be careful about, so it's no surprise then that dealing with more complex data structures also has wrinkles to watch out for.

Both C and Rust use the struct to combine related data into a single data structure. However, when a struct is realized in memory, the two languages may well choose to put different fields in different places or even in different orders (the layout). To prevent mismatches, use #[repr(C)] for Rust types used in FFI; this representation is designed for the purpose of allowing C interoperability:

/* C data structure definition. */
/* Changes here must be reflected in lib.rs. */
typedef struct {
    uint8_t byte;
    uint32_t integer;
} FfiStruct;

#![allow(unused)]
fn main() {
// Equivalent Rust data structure.
// Changes here must be reflected in lib.h / lib.c.
#[repr(C)]
pub struct FfiStruct {
    pub byte: u8,
    pub integer: u32,
}
}

The structure definitions have a comment to remind the humans involved that the two places need to be kept in sync. Relying on the constant vigilance of humans is likely to go wrong in the long term; as for function signatures, it's better to automate this synchronization between the two languages via a tool like bindgen (Item 35).

One particular type of data that's worth thinking about carefully for FFI interactions is strings. The default definitions of what makes up a string are somewhat different between C and Rust:

• A Rust String holds UTF-8 encoded data, possibly including zero bytes, with an explicitly known length.
• A C string (char *) holds byte values (which may or may not be signed), with its length implicitly determined by the first zero byte (\0) found in the data.

Fortunately, dealing with C-style strings in Rust is comparatively straightforward, because the Rust library designers have already done the heavy lifting by providing a pair of types to encode them. Use the CString type to hold (owned) strings that need to be interoperable with C, and use the corresponding CStr type when dealing with borrowed string values. The latter type includes the as_ptr() method, which can be used to pass the string's contents to any FFI function that's expecting a const char* C string. Note that the const is important: this can't be used for an FFI function that needs to modify the contents (char *) of the string that's passed to it.

Lifetimes

Most data structures are too big to fit in a register and so have to be held in memory instead. That in turn means that access to the data is performed via the location of that memory. In C terms, this means a pointer: a number that encodes a memory address—with no other semantics attached (Item 8).

In Rust, a location in memory is generally represented as a reference, and its numeric value can be extracted as a raw pointer, ready to feed into an FFI boundary:

extern "C" {
    // C function that does some operation on the contents
    // of an `FfiStruct`.
    pub fn use_struct(v: *const FfiStruct) -> u32;
}

let v = FfiStruct {
    byte: 1,
    integer: 42,
};
let x = unsafe { use_struct(&v as *const FfiStruct) };

However, a Rust reference comes with additional constraints around the lifetime of the associated chunk of memory, as described in Item 14; these constraints get lost in the conversion to a raw pointer.

As a result, the use of raw pointers is inherently unsafe, as a marker that Here Be Dragons: the C code on the other side of the FFI boundary could do any number of things that will destroy Rust's memory safety:

• The C code could hang onto the value of the pointer and use it at a later point when the associated memory has either been freed from the heap or reused on the stack (use-after-free).
• The C code could decide to cast away the const-ness of a pointer that's passed to it and modify data that Rust expects to be immutable.
• The C code is not subject to Rust's Mutex protections, so the specter of data races (Item 17) rears its ugly head.
• The C code could mistakenly return associated heap memory to the allocator (by calling C's free() library function), meaning that the Rust code might now be performing use-after-free operations.

All of these dangers form part of the cost-benefit analysis of using an existing library via FFI. On the plus side, you get to reuse existing code that's (presumably) in good working order, with only the need to write (or auto-generate) corresponding declarations. On the minus side, you lose the memory protections that are a big reason to use Rust in the first place.

As a first step to reduce the chances of memory-related problems, allocate and free memory on the same side of the FFI boundary. For example, this might appear as a symmetric pair of functions:

/* C functions. */

/* Allocate an `FfiStruct` */
FfiStruct* new_struct(uint32_t v);
/* Free a previously allocated `FfiStruct` */
void free_struct(FfiStruct* s);

with corresponding Rust FFI declarations:

extern "C" {
    // C code to allocate an `FfiStruct`.
    pub fn new_struct(v: u32) -> *mut FfiStruct;
    // C code to free a previously allocated `FfiStruct`.
    pub fn free_struct(s: *mut FfiStruct);
}

To make sure that allocation and freeing are kept in sync, it can be a good idea to implement an RAII wrapper that automatically prevents C-allocated memory from being leaked (Item 11). The wrapper structure owns the C-allocated memory:

/// Wrapper structure that owns memory allocated by the C library.
struct FfiWrapper {
    // Invariant: inner is non-NULL.
    inner: *mut FfiStruct,
}

and the Drop implementation returns that memory to the C library to avoid the potential for leaks:

/// Manual implementation of [`Drop`], which ensures that memory allocated
/// by the C library is freed by it.
impl Drop for FfiWrapper {
    fn drop(&mut self) {
        // Safety: `inner` is non-NULL, and besides `free_struct()` copes
        // with NULL pointers.
        unsafe { free_struct(self.inner) }
    }
}

The same principle applies to more than just heap memory: implement Drop to apply RAII to FFI-derived resources—open files, database connections, etc. (see Item 11).

Encapsulating the interactions with the C library into a wrapper struct also makes it possible to catch some other potential footguns, for example, by transforming an otherwise invisible failure into a Result:

type Error = String;

impl FfiWrapper {
    pub fn new(val: u32) -> Result<Self, Error> {
        let p: *mut FfiStruct = unsafe { new_struct(val) };
        // Raw pointers are not guaranteed to be non-NULL.
        if p.is_null() {
            Err("Failed to get inner struct!".into())
        } else {
            Ok(Self { inner: p })
        }
    }
}

The wrapper structure can then offer safe methods that allow use of the C library's functionality:

impl FfiWrapper {
    pub fn set_byte(&mut self, b: u8) {
        // Safety: relies on invariant that `inner` is non-NULL.
        let r: &mut FfiStruct = unsafe { &mut *self.inner };
        r.byte = b;
    }
}

Alternatively, if the underlying C data structure has an equivalent Rust mapping, and if it's safe to directly manipulate that data structure, then implementations of the AsRef and AsMut traits (described in Item 8) allow more direct use:

impl AsMut<FfiStruct> for FfiWrapper {
    fn as_mut(&mut self) -> &mut FfiStruct {
        // Safety: `inner` is non-NULL.
        unsafe { &mut *self.inner }
    }
}

let mut wrapper = FfiWrapper::new(42).expect("real code would check");
// Directly modify the contents of the C-allocated data structure.
wrapper.as_mut().byte = 12;

This example illustrates a useful principle for dealing with FFI: encapsulate access to an unsafe FFI library inside safe Rust code. This allows the rest of the application to follow the advice in Item 16 and avoid writing unsafe code. It also concentrates all of the dangerous code in one place, which you can then study (and test) carefully to uncover problems—and treat as the most likely suspect when something does go wrong.

Invoking Rust from C

What counts as "foreign" depends on where you're standing: if you're writing an application in C, then it may be a Rust library that's accessed via a foreign function interface.

The basics of exposing a Rust library to C code are similar to the opposite direction:

• Rust functions that are exposed to C need an extern "C" marker to ensure they're C-compatible.
• Rust symbols are name mangled by default (like C++),³ so function definitions also need a #[no_mangle] attribute to ensure that they're accessible via a simple name. This in turn means that the function name is part of a single global namespace that can clash with any other symbol defined in the program. As such, consider using a prefix for exposed names to avoid ambiguities (mylib_...).
• Data structure definitions need the #[repr(C)] attribute to ensure that the layout of the contents is compatible with an equivalent C data structure.

Also like the opposite direction, more subtle problems arise when dealing with pointers, references, and lifetimes. A C pointer is different from a Rust reference, and you forget that at your peril:

#[no_mangle]
pub extern "C" fn add_contents(p: *const FfiStruct) -> u32 {
    // Convert the raw pointer provided by the caller into
    // a Rust reference.
    let s: &FfiStruct = unsafe { &*p }; // Ruh-roh
    s.integer + s.byte as u32
}

/* C code invoking Rust. */
uint32_t result = add_contents(NULL); // Boom!

When you're dealing with raw pointers, it's your responsibility to ensure that any use of them complies with Rust's assumptions and guarantees around references:

#[no_mangle]
pub extern "C" fn add_contents_safer(p: *const FfiStruct) -> u32 {
    let s = match unsafe { p.as_ref() } {
        Some(r) => r,
        None => return 0, // Pesky C code gave us a NULL.
    };
    s.integer + s.byte as u32
}

In these examples, the C code provides a raw pointer to the Rust code, and the Rust code converts it to a reference in order to operate on the structure. But where did that pointer come from? What does the Rust reference refer to?

The very first example in Item 8 showed how Rust's memory safety prevents references to expired stack objects from being returned; those problems reappear if you hand out a raw pointer:

impl FfiStruct {
    pub fn new(v: u32) -> Self {
        Self {
            byte: 0,
            integer: v,
        }
    }
}

// No compilation errors here.
#[no_mangle]
pub extern "C" fn new_struct(v: u32) -> *mut FfiStruct {
    let mut s = FfiStruct::new(v);
    &mut s // return raw pointer to a stack object that's about to expire!
}

Any pointers passed back from Rust to C should generally refer to heap memory, not stack memory. But naively trying to put the object on the heap via a Box doesn't help:

// No compilation errors here either.
#[no_mangle]
pub extern "C" fn new_struct_heap(v: u32) -> *mut FfiStruct {
    let s = FfiStruct::new(v); // create `FfiStruct` on stack
    let mut b = Box::new(s); // move `FfiStruct` to heap
    &mut *b // return raw pointer to a heap object that's about to expire!
}

The owning Box is on the stack, so when it goes out of scope, it will free the heap object and the returned raw pointer will again be invalid.

The tool for the job here is Box::into_raw, which abnegates responsibility for the heap object, effectively "forgetting" about it:

#[no_mangle]
pub extern "C" fn new_struct_raw(v: u32) -> *mut FfiStruct {
    let s = FfiStruct::new(v); // create `FfiStruct` on stack
    let b = Box::new(s); // move `FfiStruct` to heap

    // Consume the `Box` and take responsibility for the heap memory.
    Box::into_raw(b)
}

This raises the question of how the heap object now gets freed. The previous advice was to perform allocation and freeing of memory on the same side of the FFI boundary, which means that we need to persuade the Rust side of things to do the freeing. The corresponding tool for the job is Box::from_raw, which builds a Box from a raw pointer:

#[no_mangle]
pub extern "C" fn free_struct_raw(p: *mut FfiStruct) {
    if p.is_null() {
        return; // Pesky C code gave us a NULL
    }
    let _b = unsafe {
        // Safety: p is known to be non-NULL
        Box::from_raw(p)
    };
} // `_b` drops at end of scope, freeing the `FfiStruct`

This still leaves the Rust code at the mercy of the C code; if the C code gets confused and asks Rust to free the same pointer twice, Rust's allocator is likely to become terminally confused.

That illustrates the general theme of this Item: using FFI exposes you to risks that aren't present in standard Rust. That may well be worthwhile, as long as you're aware of the dangers and costs involved. Controlling the details of what passes across the FFI boundary helps to reduce that risk but by no means eliminates it.

Controlling the FFI boundary for C code invoking Rust also involves one final concern: if your Rust code ignores the advice in Item 18, you should prevent panic!s from crossing the FFI boundary, as this always results in undefined behavior—undefined but bad!⁴

Things to Remember

• Interfacing with code in other languages uses C as a least common denominator, which means that symbols all live in a single global namespace.
• Minimize the chances of problems at the FFI boundary by doing the following:
  • Encapsulating unsafe FFI code in safe wrappers
  • Allocating and freeing memory consistently on one side of the boundary or the other
  • Making data structures use C-compatible layouts
  • Using sized integer types
  • Using FFI-related helpers from the standard library
  • Preventing panic!s from escaping from Rust