bentoml

An unofficial async Rust client for BentoML services.

BentoML services expose their @bentoml.api methods as HTTP POST endpoints whose route is derived from the method name. Because endpoints are defined dynamically per-service, this crate doesn't generate typed bindings: instead it offers a generic call over [serde] types, plus extension traits for the rest of the HTTP surface.

Features

Generic calls: invoke any endpoint with call(route, payload) over your own serde request and response types, with no codegen or per-service bindings.
Async task queues: submit @bentoml.task jobs and poll status, fetch results, retry, or cancel through a TaskHandle.
File and streaming I/O: multipart/form-data file inputs, raw-binary root inputs, binary responses, and chunked streaming endpoints (feature stream).
Resilient transport: per-request timeouts and exponential-backoff retries via reqwest-middleware, bearer-token auth, and a cheap-to-clone Arc-backed client.

Usage

Add the dependency:

[dependencies]
bentoml = "0.1"
tokio = { version = "1", features = ["macros", "rt-multi-thread"] }
serde = { version = "1", features = ["derive"] }

The minimum supported Rust version (MSRV) is 1.91.

use bentoml::prelude::*;
use serde::{Deserialize, Serialize};

#[derive(Serialize)]
struct SummarizeRequest { text: String }

#[derive(Deserialize)]
struct SummarizeResponse { summary: String }

#[tokio::main]
async fn main() -> Result<()> {
    let client = Client::builder()
        .with_base_url("http://localhost:3000")
        .build()?;

    let resp: SummarizeResponse = client
        .endpoint("summarize")
        .call(&SummarizeRequest { text: "...".into() })
        .await?;

    println!("{}", resp.summary);
    Ok(())
}

A Client::endpoint(route) handle names the route once; calls are made on it. See examples/ for runnable examples.

Capabilities

A Client::endpoint(route) handle implements a set of extension traits (all in the prelude) covering the BentoML HTTP surface:

Tasks: async task queues (@bentoml.task); submit returns a TaskHandle for status / get / retry / cancel.
Files: multipart/form-data file inputs, raw-binary root inputs, and binary responses.
Streaming: stream returns a Stream of response chunks (feature stream).

The Client itself provides Readiness: is_ready / is_live health checks and wait_until_ready.

These are gated by feature flags:

rustls-tls (default): HTTPS via Rustls.
native-tls: HTTPS via the platform-native TLS stack.
stream: streaming response endpoints (Streaming).
tracing: #[tracing::instrument] on request methods.

Changelog

See CHANGELOG.md for release notes and version history.

License