hotdata

Official Rust client for the Hotdata HTTP API: workspaces, connections, databases, SQL queries, results, uploads, indexes, jobs, embedding providers, and workspace context.

The crate pairs a fully generated, typed API surface (hotdata::apis, hotdata::models) with a hand-written ergonomic layer: a flat Client that wires up authentication (a static API token, or a pluggable per-request bearer) and workspace scoping, plus an optional Apache Arrow result decoder.

Requirements

Rust 1.74+ and a Tokio runtime (the client is async).

Install

Add the crate to your Cargo.toml:

[dependencies]
hotdata = "0.18"
tokio = { version = "1", features = ["macros", "rt-multi-thread"] }

For an unreleased revision:

[dependencies]
hotdata = { git = "https://github.com/hotdata-dev/sdk-rust.git" }

By default the crate builds against native-tls. To use rustls instead:

[dependencies]
hotdata = { version = "0.18", default-features = false, features = ["rustls"] }

Authentication

The API authenticates with an API token sent as Authorization: Bearer <token>, plus an X-Workspace-Id header on requests scoped to a workspace.

Your API token (prefixed hd_) is the only credential you need. It is sent verbatim as the bearer token on every request — there is nothing to exchange, refresh, or cache.

Deprecated and removed. Earlier versions exchanged the API token for a short-lived JWT against /v1/auth/jwt and refreshed it in the background. That exchange, TokenManager, ClientBuilder::client_id, and the HOTDATA_DISABLE_JWT_EXCHANGE escape hatch are all gone and are not coming back — see the CHANGELOG. If you passed an API token to ClientBuilder::api_token, nothing changes for you.

use hotdata::prelude::*;

let client = Client::builder()
    .api_token("hd_your_api_token")
    .workspace_id("your_workspace_id")
    .build()?;

base_url defaults to https://api.hotdata.dev. Override it if you target another environment.

Supplying a bearer per request

A static API token is fixed for the life of the Configuration, which is all most callers need. A host that owns its own credential lifecycle — an interactive login whose access token expires in minutes, say — can install a BearerTokenProvider on Configuration::token_provider instead. The SDK then asks it for a bearer once per request, so a long call (a multi-gigabyte upload_file, a slow query, a large parallel batch) can refresh mid-flight rather than 401 on a token that expired after it started:

use hotdata::auth::{async_trait, BearerTokenError, BearerTokenProvider};
use hotdata::prelude::*;

#[derive(Debug)]
struct MySession { /* refresh token, expiry, mutex, ... */ }

#[async_trait]
impl BearerTokenProvider for MySession {
    async fn bearer_value(&self) -> Result<String, BearerTokenError> {
        // Refresh if needed, then hand back a currently valid access token.
        Ok("eyJ...".to_owned())
    }
}

let mut client = Client::builder().api_token("unused").build()?;
client.configuration_mut().token_provider = Some(std::sync::Arc::new(MySession {}));

bearer_value is called on every request, including the create-session and finalize legs of upload_file, so a credential that rotates between them is picked up. It is a hook and nothing more: the SDK never exchanges one credential for another. With no provider installed, bearer_access_token behaves exactly as before.

If a provider returns an error, the SDK logs a warning on the log facade and sends the request unauthenticated, which surfaces as a 401 from the server. That warning is only visible if your binary installed a log implementation with warn enabled for the hotdata target — without one, a provider failure looks like any other 401, so wire up a logger before debugging one.

Quickstart

use hotdata::prelude::*;

#[tokio::main]
async fn main() -> Result<(), Box<dyn std::error::Error>> {
    let client = Client::builder()
        .api_token("hd_your_api_token")
        .workspace_id("your_workspace_id")
        // .base_url("https://api.hotdata.dev")  // optional
        .build()?;

    // Queries, results, and query runs are scoped to a database (the required
    // `X-Database-Id` header), so pick one first.
    let database_id = "your_database_id";

    // Submit a query. `query_in` transparently retries HTTP 429 (server overload)
    // and auto-follows a truncated large result to its full row set. Rows come
    // back inline, plus a result_id persisted for later retrieval.
    let response = client
        .query_in(QueryRequest::new("SELECT 1 AS n".to_string()), database_id)
        .await?;

    if let Some(result_id) = response.result_id.flatten() {
        // Poll the persisted result to `ready` without hand-rolling a loop.
        let result = client
            .await_result(&result_id, database_id, PollConfig::default())
            .await?;
        println!("result {} is {}", result.result_id, result.status);
    }

    Ok(())
}

Resource handles

The OpenAPI generator emits free functions; the Client groups them into ergonomic, workspace-scoped handles so you never pass a Configuration around:

// Grouped handles: client.<resource>().<operation>(..)
let connections = client.connections().list().await?;
let connection  = client.connections().get(&connections.connections[0].id).await?;
// Query runs are database-scoped: pass the database id first.
let runs     = client.query_runs().list(database_id, Some(50), None, None, None).await?;

Handles exist for every resource — connections, databases, database_context, embedding_providers, indexes, information_schema, jobs, queries, query_runs, results, saved_queries, uploads, workspaces. The hottest operations also have flat shortcuts directly on Client (query — with query_in to scope to a database, query_preview to skip auto-follow, and query_with for a per-call QueryConfig — plus get_result, list_results, list_query_runs, list_workspaces). The result and query-run operations take a database_id argument — they are scoped to a database via the required X-Database-Id header.

For anything not yet wrapped, the full generated surface is one call away via client.configuration():

use hotdata::apis::workspaces_api;
let workspaces = workspaces_api::list_workspaces(client.configuration(), None).await?;

Typed status

Result and query-run status fields are plain strings on the wire. Interpret them with the typed [ResultStatus] / [QueryRunStatus] enums via the result_status() / run_status() accessors:

use hotdata::prelude::*;

let result = client.await_result(&result_id, database_id, PollConfig::default()).await?;
if result.result_status().is_ready() {
    // ...
}

let run = client.query_runs().get(&query_run_id, database_id).await?;
if run.run_status().is_terminal() { /* ... */ }

Both enums carry an Other(String) variant, so a status the server adds later round-trips instead of breaking deserialization.

Updating nullable fields

Several update requests model a field that is both optional (omit to leave unchanged) and nullable (send null to clear) as Option<Option<T>>. The field helpers name the three intents so call sites read clearly:

use hotdata::field;

let mut req = UpdateSavedQueryRequest::new();
req.name = field::set("renamed");       // set
req.description = field::clear();        // send null (clear)
// req.sql left as None -> omitted -> unchanged
client.saved_queries().update(&saved_query_id, req).await?;

Every resource lives under hotdata::apis::<resource>_api, and request/response types under hotdata::models. The flat prelude re-exports Client, ClientBuilder, PollConfig, Configuration, the resource handles, and all models for convenience.

Errors from generated operations are returned as hotdata::Error<T>; builder and configuration failures are hotdata::ClientError. The enhanced query family returns hotdata::QueryError — Overloaded (429 retries exhausted), Submit (the underlying request failed), AsyncRequested (use submit_query for async queries), Async (the server fell back to asynchronous execution with a 202; the acknowledgement is passed through), Poll (an API error while polling during auto-follow), and Result(ResultError) for truncation auto-follow failures (TooLarge / Timeout / Incomplete / …). Result-polling and one-call helpers return hotdata::AwaitResultError / hotdata::QueryToArrowError. The SDK's own error enums are #[non_exhaustive], so match them with a wildcard arm.

Arrow results

Query results can be fetched as an Apache Arrow IPC stream instead of JSON, which is faster and far more memory-efficient for large result sets. The decoder is behind an optional arrow feature (off by default):

[dependencies]
hotdata = { version = "0.18", features = ["arrow"] }

ArrowResult hands back arrow types, so a crate that names them must depend on arrow 59 — the same major this SDK uses. Two arrow majors in one build are distinct types and will not compile together.

use hotdata::prelude::*;

#[tokio::main]
async fn main() -> Result<(), Box<dyn std::error::Error>> {
    let client = Client::builder()
        .api_token("hd_your_api_token")
        .workspace_id("your_workspace_id")
        .build()?;

    // Results are database-scoped (the required `X-Database-Id` header).
    let database_id = "your_database_id";

    // Buffered: decodes every batch into a Vec<RecordBatch>.
    let result = client.get_result_arrow(&result_id, database_id, None, None).await?;
    println!("schema: {:?}", result.schema);
    println!("total rows: {:?}", result.total_row_count);
    for batch in &result.batches {
        // work with each arrow_array::RecordBatch
    }

    // Streaming: yields batches lazily without holding them all at once.
    let mut stream = client.stream_result_arrow(&result_id, database_id, None, None).await?;
    for batch in stream.by_ref() {
        let batch = batch?;
        // ...
    }

    Ok(())
}

A third, open_result_arrow, decodes straight off the socket: it pulls body chunks as batches are asked for, so peak memory is one record batch rather than the whole result. Use it for a result larger than memory — the other two collect the entire body before returning. Its schema is available before the first batch, and the pooled connection stays checked out until the stream is drained or dropped.

let mut stream = client
    .open_result_arrow(&result_id, &database_id, None, None)
    .await?;
println!("columns: {:?}", stream.schema().fields());
while let Some(batch) = stream.next_batch().await? {
    // ... one batch at a time; the rest is still on the wire
}

All three accept offset and limit for pagination, and all carry the same authentication and workspace headers as the generated operations. They return ArrowError::NotReady if the result is still pending or processing — poll client.get_result(result_id, database_id) until its status is ready first. ArrowResult also surfaces the X-Total-Row-Count header (total_row_count) and the rel="next" pagination Link (next_link).

To run a query and get its result as Arrow in a single call — submit, await ready, and decode — use query_to_arrow:

let arrow = client
    .query_to_arrow(
        QueryRequest::new("SELECT * FROM big_table".to_string()),
        database_id,
        PollConfig::default(),
        None, // offset
        None, // limit
    )
    .await?;

Debug logging

Every HTTP call the SDK makes — generated operations and the hand-written submit_query, upload_file, and Arrow fetch — emits log::debug! records on the hotdata::http target: the request (>>> METHOD url, headers, body) and the response (<<< status, body). Authorization bearer tokens and sensitive body fields (api_token, secret, password, …) are masked before logging.

The SDK installs no logger and prints nothing on its own. To see the records, wire any log backend and enable the hotdata::http target at debug level. For example with env_logger:

// RUST_LOG=hotdata::http=debug cargo run
env_logger::init();

[dependencies]
env_logger = "0.11"

>>> POST https://api.hotdata.dev/v1/query
  authorization: Bearer hd_a...cdef
  content-type: application/json
{"sql":"SELECT 1"}
<<< 200 OK
{"result_id":"…","columns":[…]}

API reference

Generated documentation builds on docs.rs (with all-features enabled, so the arrow surface is included).

Generated Markdown for every operation and model also lives in docs/:

Resource APIs: docs/*Api.md (for example QueryApi.md)
Request and response models: docs/<ModelName>.md

Support

Questions and issues: github.com/hotdata-dev/sdk-rust.

hotdata 0.18.0