Crate bentoml 

bentoml

An unofficial async Rust client for BentoML services.

BentoML services expose their @bentoml.api methods as HTTP POST endpoints whose route is derived from the method name. Because endpoints are defined dynamically per-service, this crate doesn’t generate typed bindings: instead it offers a generic call over serde types, plus extension traits for the rest of the HTTP surface.

Features

• Generic calls: invoke any endpoint with call(route, payload) over your own serde request and response types, with no codegen or per-service bindings.
• Async task queues: submit @bentoml.task jobs and poll status, fetch results, retry, or cancel through a TaskHandle.
• File and streaming I/O: multipart/form-data file inputs, raw-binary root inputs, binary responses, and chunked streaming endpoints (feature stream).
• Resilient transport: per-request timeouts and exponential-backoff retries via reqwest-middleware, bearer-token auth, and a cheap-to-clone Arc-backed client.

Usage

Add the dependency:

[dependencies]
bentoml = "0.1"
tokio = { version = "1", features = ["macros", "rt-multi-thread"] }
serde = { version = "1", features = ["derive"] }

The minimum supported Rust version (MSRV) is 1.91.

use bentoml::prelude::*;
use serde::{Deserialize, Serialize};

#[derive(Serialize)]
struct SummarizeRequest { text: String }

#[derive(Deserialize)]
struct SummarizeResponse { summary: String }

#[tokio::main]
async fn main() -> Result<()> {
    let client = Client::builder()
        .with_base_url("http://localhost:3000")
        .build()?;

    let resp: SummarizeResponse = client
        .endpoint("summarize")
        .call(&SummarizeRequest { text: "...".into() })
        .await?;

    println!("{}", resp.summary);
    Ok(())
}

A Client::endpoint(route) handle names the route once; calls are made on it. See examples/ for runnable examples.

Capabilities

A Client::endpoint(route) handle implements a set of extension traits (all in the prelude) covering the BentoML HTTP surface:

• Tasks: async task queues (@bentoml.task); submit returns a TaskHandle for status / get / retry / cancel.
• Files: multipart/form-data file inputs, raw-binary root inputs, and binary responses.
• Streaming: stream returns a Stream of response chunks (feature stream).

The Client itself provides Readiness: is_ready / is_live health checks and wait_until_ready.

These are gated by feature flags:

• rustls-tls (default): HTTPS via Rustls.
• native-tls: HTTPS via the platform-native TLS stack.
• stream: streaming response endpoints (Streaming).
• tracing: #[tracing::instrument] on request methods.

Changelog

See CHANGELOG.md for release notes and version history.

License

Licensed under the MIT License.

Modules

model

Shared data models for BentoML services.

service

Capability traits for talking to a BentoML service.

Structs

Client

An async client for a single BentoML service.

ClientBuilder

A builder for Client.

Endpoint

A handle to a single service endpoint, pairing a route with its Client.

Enums

Error

The error type returned by client operations.

Constants

DEFAULT_BASE_URL

The default base URL used when none is configured.

DEFAULT_MAX_RETRIES

The default number of retries for transient request failures.

DEFAULT_TIMEOUT

The default request timeout applied when none is configured.

Type Aliases

Result

A convenient alias for results returned by this crate.