Inferential Rust SDK

Rust client SDK for Inferential inference orchestration. Communicates with the server using ZMQ DEALER sockets and the shared protobuf wire protocol.

Includes both synchronous (Connection/Model) and async (AsyncConnection/AsyncModel) clients.

Install

Add to your Cargo.toml:

[dependencies]
inferential = "0.3"

Or via the CLI:

cargo add inferential

Requires libzmq installed on the system (apt install libzmq3-dev or brew install zmq).

Build from Source

Requires Rust 1.70+, libzmq installed on the system, and protoc (protobuf compiler).

# Build
cargo build

# Run tests (single-threaded — tests share ZMQ ports)
cargo test -- --test-threads=1

# Run examples
cargo run --example client_demo
cargo run --example async_client_demo

# Lint
cargo clippy

Or use the root Makefile:

make build-rust
make test-rust

Usage

Sync

use inferential::{Connection, proto};

let conn = Connection::new("tcp://localhost:5555", "agent-01", "franka");
let model = conn.model("policy-v2", 30.0, 1);

let joints: Vec<f32> = vec![0.1, 0.2, 0.3, 0.4, 0.5, 0.6, 0.7];
model.observe()
    .urgency(0.8)
    .tensor_f32("joint_positions", &joints, &[7])
    .send();

if let Some(result) = model.get_result(100) {
    let actions = result["actions"].as_f32();
}

conn.close();  // also called on Drop

Async

use inferential::AsyncConnection;

#[tokio::main]
async fn main() {
    let mut conn = AsyncConnection::new("tcp://localhost:5555", "agent-01", "franka").await;
    let mut model = conn.model("policy-v2", 30.0, 1);

    let joints: Vec<f32> = vec![0.1, 0.2, 0.3, 0.4, 0.5, 0.6, 0.7];
    model.observe()
        .urgency(0.8)
        .tensor_f32("joint_positions", &joints, &[7])
        .send()
        .await;

    if let Some(result) = model.get_result(100).await {
        let actions = result["actions"].as_f32();
    }

    // Must drop model before closing conn (borrow rules)
    drop(model);
    conn.close().await;  // consumes self
}

API Reference

Connection (sync)

Connection::new(server: &str, client_id: &str, client_type: &str) -> Self
Connection::with_options(server, client_id, client_type, reconnect_ivl_ms, reconnect_max_ms) -> Self

Creates a ZMQ DEALER connection. Auto-prefixes tcp:// if missing. Implements Drop for cleanup.

AsyncConnection

AsyncConnection::new(server: &str, client_id: &str, client_type: &str) -> Self  // async

Tokio-based async connection using the zeromq crate.

Model<'a> / AsyncModel<'a>

Obtained via conn.model(...). Borrows the connection.

ObservationBuilder / AsyncObservationBuilder

Fluent builder — chain methods and finalize with .send().

All builder methods consume and return self (move semantics).

TensorData

Holds tensor data received from get_result().

pub struct TensorData {
    pub key: String,
    pub data: Vec<u8>,       // raw bytes
    pub shape: Vec<i64>,
    pub dtype: i32,          // proto::DType as i32
}

Complete Example

Sync

use inferential::Connection;
use std::{thread, time::Duration};

fn main() {
    let conn = Connection::new("tcp://localhost:5555", "rust-demo", "sim");
    let model = conn.model("policy-v2", 30.0, 1);

    for step in 0..100 {
        let state: Vec<f32> = (0..7).map(|i| i as f32 * 0.1).collect();

        model.observe()
            .urgency(0.5)
            .steps_remaining(100 - step)
            .tensor_f32("state", &state, &[7])
            .metadata("task", "pick_and_place")
            .send();

        if let Some(result) = model.get_result(100) {
            let actions = result["actions"].as_f32();
            if step % 20 == 0 {
                println!("step {}: actions={:?}", step, &actions[..3.min(actions.len())]);
            }
        }

        thread::sleep(Duration::from_millis(50));
    }
}

Async

use inferential::AsyncConnection;
use tokio::time::{sleep, Duration};

#[tokio::main]
async fn main() {
    let mut conn = AsyncConnection::new("tcp://localhost:5555", "rust-async", "sim").await;

    {
        let mut model = conn.model("policy-v2", 30.0, 1);

        for step in 0u32..100 {
            let state: Vec<f32> = (0..7).map(|i| i as f32 * 0.1).collect();

            model.observe()
                .urgency(0.5)
                .steps_remaining(100 - step)
                .tensor_f32("state", &state, &[7])
                .metadata("task", "pick_and_place")
                .send()
                .await;

            if let Some(result) = model.get_result(100).await {
                let actions = result["actions"].as_f32();
                if step % 20 == 0 {
                    println!("step {}: actions={:?}", step, &actions[..3.min(actions.len())]);
                }
            }

            sleep(Duration::from_millis(50)).await;
        }
    } // model dropped here, releasing borrow on conn

    conn.close().await;
}

Notes

• Sync vs Async: The sync client uses the zmq crate (libzmq binding). The async client uses the zeromq crate (native Rust, tokio-based).
• Borrow rules: Model borrows &Connection, AsyncModel borrows &mut AsyncConnection. Drop the model before calling conn.close().
• Thread safety: Connection is not Send/Sync (ZMQ sockets are single-threaded). Create one connection per thread.
• Test threading: Tests must run single-threaded (--test-threads=1) because they bind to fixed ZMQ ports.
• Wire protocol: See Architecture for the shared protobuf wire protocol.

Documentation

• Architecture — Wire protocol, system design
• Examples — Multi-language examples
• Contributing — Development workflow