tflite-c-rs

A safe Rust wrapper around the TensorFlow Lite C API that resolves the shared library at runtime with libloading instead of linking it at build time. No build.rs, no -sys crate, no TFLite headers required — point the loader at a libtensorflowlite_c.so / .dylib / .dll you already have on the target and go.

Note: the bare tflite-c name on crates.io is published as tflite-c-rs (matching the precedent set by sht4x-rs). The library itself is imported as use tflite_c::....

Quick start

use tflite_c::{TfLiteLibrary, Model, InterpreterOptions, Interpreter};

# fn run() -> Result<(), tflite_c::Error> {
let lib = TfLiteLibrary::load_default()?;
let model = Model::from_file("model.tflite", lib.clone())?;

let mut options = InterpreterOptions::new(lib.clone());
options.num_threads(2);

let mut interp = Interpreter::new(model, options)?;

// Fill input 0 with whatever bytes your model expects.
interp.input_mut(0)?.data_mut()?.fill(0);

interp.invoke()?;

let out = interp.output(0)?;
let probs = out.to_vec_f32()?; // dequantizes int8/uint8 automatically
println!("first prob: {}", probs[0]);
# Ok(()) }

To verify your install is reachable from Rust before wiring up a model:

cargo run --example load_library

Why dynamic loading?

TensorFlow Lite is famously awkward to link against. The upstream build system is Bazel-only, the prebuilt releases are platform-specific, and cross-compiling a static link tends to pull in the entire TensorFlow source tree along with it. For most real-world deployments — embedded Linux, edge accelerators, vendor-shipped NPU stacks — the simplest path is to ship the prebuilt libtensorflowlite_c.so that the vendor already gives you and load it at runtime.

That's all this crate does:

1. dlopen the TFLite C shared object (probing a few standard locations, or an explicit path you provide).
2. Resolve and cache the symbols needed for Model → Interpreter → invoke → read output.
3. Wrap the raw handles in RAII Rust types with Drop impls that call the matching destructors in the right order.

The result is a Rust API that compiles on any target where you can compile Rust + libloading, regardless of whether TFLite headers or .so files are present at build time. They only need to be present at run time, on the host that actually runs the inference.

External delegates (NNAPI, GPU, XNNPACK, VX/NPU…)

TFLite ships a generic external delegate C API that lets any plugin shared object register itself as an inference backend. This crate exposes that mechanism directly:

use tflite_c::{TfLiteLibrary, ExternalDelegate, InterpreterOptions};
use std::sync::Arc;

# fn run(lib: Arc<TfLiteLibrary>) -> Result<(), tflite_c::Error> {
let delegate = ExternalDelegate::builder(lib.clone(), "/usr/lib/libvx_delegate.so")?
    .with_option("allowed_cache_mode", "AUTO")?
    .build()?;

let mut options = InterpreterOptions::new(lib.clone());
options.add_delegate(delegate);
# Ok(()) }

The crate itself stays vendor-neutral — point it at any external-delegate plugin (NNAPI, GPU, XNNPACK, vendor NPU) and it will hand it to TFLite without needing to know what kind of accelerator sits behind it.

Threading

Interpreter: Send + Sync, but TensorFlow Lite's C interpreter is not internally thread-safe. The standard pattern is to wrap in Arc<Mutex<Interpreter>> (or Arc<parking_lot::Mutex<Interpreter>>) and serialize every call.

Running the integration test

Default cargo test exercises the loader-failure path, dequantization math, and error formatting without needing TFLite installed. To run the real end-to-end test against a host TFLite, set two env vars:

TFLITE_LIB=/path/to/libtensorflowlite_c.dylib \
TFLITE_MODEL=/path/to/model.tflite \
cargo test -- --ignored --nocapture

When TFLITE_MODEL is upstream TFLite's add.bin (a 544-byte y = 3x graph with float32 [1,8,8,3] input/output), the test asserts the numerical result. Any other model just exercises load → allocate → invoke → read.

Status

v0.0.1 — minimal, end-to-end validated against a prebuilt libtensorflowlite_c.2.17.1.dylib (darwin arm64). Covers Model::from_file, InterpreterOptions (threads + external delegates), Interpreter::invoke, typed and quantization-aware tensor accessors. Unit tests exercise dequantization and error formatting; integration tests cover loader failure modes and (when TFLite is present) full inference. v0.1 will add string-input/quantized-output convenience helpers and additional dtype coverage. PRs welcome at https://github.com/KarpagamKarthikeyan/tflite-c-rs.

External delegate symbols (TfLiteExternalDelegate*) live in TFLite's experimental header and are routinely missing from prebuilt shared objects. The crate detects this at load time and surfaces a clean Error::ExternalDelegateApiUnavailable at ExternalDelegate::builder call time — the core load → invoke path keeps working regardless.

License

Licensed under either of

• Apache License, Version 2.0 (LICENSE-APACHE or http://www.apache.org/licenses/LICENSE-2.0)
• MIT license (LICENSE-MIT or http://opensource.org/licenses/MIT)

at your option.

Contribution

Unless you explicitly state otherwise, any contribution intentionally submitted for inclusion in the work by you, as defined in the Apache-2.0 license, shall be dual licensed as above, without any additional terms or conditions.