pyo3_bindgen

Automatic generation of Rust FFI bindings to Python modules via PyO3. Python modules are analyzed recursively to generate Rust bindings with an identical structure for all public classes, functions, properties, and constants. Any available docstrings and type annotations are also preserved in their Rust equivalents.

An example of a generated Rust function signature and its intended usage is shown below. Of course, manually wrapping parts of the generated bindings in a more idiomatic Rust API might be beneficial in most cases.

 
def answer_to(question: str) -> int:
  """Returns answer to question."""

  return 42

 

 
def main():
  assert answer_to("life") == 42


if __name__ == "__main__":
  main()
 

/// Returns answer to question.
pub fn answer_to<'py>(
  py: ::pyo3::marker::Python<'py>,
  question: &str,
) -> ::pyo3::PyResult<i64> {
  ... // Calls function via `pyo3`
}

pub fn main() -> pyo3::PyResult<()> {
  pyo3::Python::with_gil(|py| {
    assert_eq!(
      answer_to(py, "universe")?, 42
    );
    Ok(())
  })
}

This project is intended to simplify the integration or transition of existing Python codebases into Rust. You, as a developer, gain immediate access to the Rust type system and countless other benefits of modern compiled languages with the generated bindings. Furthermore, the entire stock of high-quality crates from crates.io becomes at your disposal.

On its own, the generated Rust code does not provide any performance benefits over using the Python code (it might actually be slower — yet to be benchmarked). However, it can be used as a starting point for further optimization if you decide to rewrite performance-critical parts of your codebase in pure Rust.

Overview

The workspace contains these packages:

• pyo3_bindgen: Public API for generation of bindings (in build.rs or via procedural macros)
• pyo3_bindgen_cli: CLI tool for generation of bindings via pyo3_bindgen executable
• pyo3_bindgen_engine: The underlying engine for generation of bindings
• pyo3_bindgen_macros: [Experimental] Procedural macros for in-place generation

Instructions

Add pyo3 as a dependency and pyo3_bindgen as a build dependency to your Cargo.toml manifest (auto-initialize feature of pyo3 is optional and shown here for your convenience).

[dependencies]
pyo3 = { version = "0.20", features = ["auto-initialize"] }

[build-dependencies]
pyo3_bindgen = { version = "0.1" }

Option 1: Build script

Create a build.rs script in the root of your crate that generates bindings to the target_module Python module.

// build.rs

fn main() {
    // Generate Rust bindings to the Python module
    pyo3_bindgen::build_bindings(
        "target_module",
        std::path::Path::new(&std::env::var("OUT_DIR").unwrap()).join("bindings.rs"),
    )
    .unwrap();
}

Afterwards, include the generated bindings anywhere in your crate.

include!(concat!(env!("OUT_DIR"), "/bindings.rs"));
pub use target_module::*;

Option 2: CLI tool

Install the pyo3_bindgen executable with cargo.

cargo install --locked pyo3_bindgen_cli

Afterwards, run the pyo3_bindgen executable while passing the name of the target Python module.

# Pass `--help` to show the usage and available options
pyo3_bindgen -m target_module -o bindings.rs

Option 3 [Experimental]: Procedural macros

Note: This feature is experimental and will probably fail in many cases. It is recommended to use build scripts instead.

Enable the macros feature of pyo3_bindgen.

[build-dependencies]
pyo3_bindgen = { version = "0.1", features = ["macros"] }

Then, you can call the import_python! macro anywhere in your crate.

pyo3_bindgen::import_python!("target_module");
pub use target_module::*;

Status

This project is in early development, and as such, the API of the generated bindings is not yet stable.

• Not all Python types are mapped to their Rust equivalents yet. For this reason, some additional typecasting might be currently required when using the generated bindings (e.g. let typed_value: target_module::Class = any_value.extract()?;).
• The binding generation is primarily designed to be used inside build scripts or via procedural macros. Therefore, the performance of the codegen process is benchmarked to understand the potential impact on build times. Although there is currently plenty of room for optimization in the current naive implementation, even the largest modules are processed in less than a second on a modern laptop.
• The generation of bindings should never panic as long as the target Python module can be successfully imported. If it does, it is a bug resulting from an unexpected edge-case Python module structure or an unforeseen combination of enabled PyO3 features.
• However, the generated bindings might not directly compile in some specific cases. Currently, there are two known issue; bindings will contain duplicate function definitions if present in the original code, and function parameters might use the same name as a class defined in the same scope (allowed in Python but not in Rust). If you encounter any other issues, consider manually rewriting the problematic parts of the bindings.
• Although implemented, the procedural macros might not work in all cases - especially when some PyO3 features are enabled. In most cases, PyO3 fails to import the target Python module when used from within a proc_macro crate. Therefore, it is recommended to use build scripts instead for now.
• The code will be refactored and cleaned up in the upcoming releases. The current implementation is a result of a very quick prototype that was built to test the feasibility of the idea. For example, configurability of the generated bindings is planned (e.g. allowlist/ignorelist of attributes). Furthermore, automatic generation of dependent Python modules will be considered in order to provide a more complete typing experience.

Please report any issues that you might encounter. Contributions are more than welcome! If you are looking for a place to start, consider searching for TODO comments in the codebase.

License

This project is dual-licensed to be compatible with the Rust project, under either the MIT or Apache 2.0 licenses.

Contributing

Unless you explicitly state otherwise, any contribution intentionally submitted for inclusion in the work by you, as defined in the Apache-2.0 license, shall be dual licensed as above, without any additional terms or conditions.