Module marker_adapter::context

source ·
Expand description

The marker_api crate is designed to be driver-independent and lightweight. Communication from the lint-crate to the driver is done via structs inside the marker_api::context module. For example, marker_api::MarkerContext.

The communication needs to be ABI safe, since it goes over an FFI function boundary. For this reason, it’s not (directly) possible to use dyn Traitobjects. Marker instead uses *Callback structs, that contain function pointers, which can be filled by the driver. This is like a dyn Trait object, without all the nice syntax and build in compiler support.

These *Callbacks structs need to be filled and some ABI-specific transformations have to be performed. This module hides all the gore behind some structs and traits, which can be implemented by the specific driver. The following is an explanation of this structure and some naming conventions. The explanation will use an imaginary Magic struct, to illustrate the structure:

  • The marker_api crate defines the Magic struct with an interface for external and internal consumers.

    It also defines the MagicCallbacks struct, which stores function pointers. All *Callbacks structs have a special data: &'ast MarkerContextData field that will be passed to all stored function pointers, as the first argument. All function pointers are marked as extern "C" and need to use FFI safe types. Lifetimes are allowed by Rust, but not enforced over FFI bounds.

  • The marker_adapter crate defines a MagicDriver trait, that has all the functions, needed to provide a backend for the Magic struct.

    It also defines the MagicWrapper struct, that has a driver: &'ast dyn MagicDriver field. This struct will be used to fill the MagicCallbacks.data field. Wrapping it in a separate *Wrapper struct makes the &'ast *Wrapper pointer a thin pointer and cleans up the interface.

    The marker_adapter module containing the MagicWrapper struct defines a bunch of extern "C" functions, which build the counterpart to the extern "C" functions inside MagicCallbacks. These functions cast the data argument into the &'ast MagicWrapper instance it originated from and calls the corresponding trait function from the driver: &'ast dyn MagicDriver field stored inside the Wrapper struct. The extern "C" functions are also responsible for converting all types into FFI safe types and back.

  • The driver simply implements the MagicDriver trait and instantiates MagicWrapper. This wrapper instance is then used to fill MagicCallbacks and instantiate the Magic struct from marker_api.

Short Q&A with @xFrednet

  • Isn’t there a simpler way?

    Most likely… Or better say hopefully there is.

    However, I couldn’t find one. Most libraries I found either didn’t support these types of callbacks, were experimental, or are unsound.

    I also have high hopes for Rust adding some cross crate boundary communication support some time in the future. But there is no point in waiting on this. There are lints which need to be written, and we can always replace this infrastructure later.

  • Isn’t there a simpler way to make the types FFI safe?

    Not really. There are some tools to generate ABI safe types, but they are geared towards C/C++ consumers. As a result, they usually lose lifetime information, which we want to keep since both sides are using Rust.

    There are also crates that rely on serialization. A nice and simple solution, but not feasible for an entire AST over multiple FFI boundaries for every lint crate.

  • Do you use code generation for all this infrastructure?

    I would love to! However, I haven’t found the time to implement this. Normal macro rules are basically out of the question due to all the required FFI type transformation.

    If anyone is interested in implementing this, I would be grateful!!! See rust-marker/marker#122

  • Is this implementation even safe and sound?

    Theoretically speaking? From my understanding? Yes, it is, assuming that both sides reconstruct the lifetimes correctly.

    Practically speaking? It wouldn’t surprise me if there were several bugs. So far, it has been working suspiciously well, but I won’t complain.

Structs

Traits