Crate allama_http

Expand description

The HTTP transport seam of allama’s HTTP provider crates.

Provider crates (allama-anthropic, allama-gemini, allama-openai-compat, allama-openai, allama-systemone) build an HttpRequest, send it through an HttpTransport, and decode the HttpResponse. Their request mapping, error classification and decoding never touch an HTTP client directly, so the transport can be swapped:

ReqwestTransport (feature reqwest, default) is what providers use unless told otherwise.
ReplayTransport serves canned responses in memory and records the requests: transport-level fixtures that exercise the whole adapter without a server, on any target.
Custom transports cover other HTTP stacks, proxies, or recording.

This is a seam for provider crates, not middleware for applications: it carries bytes, and the portable allama types stay above it. Beside the seam, it holds the helpers every provider shares, first-party or not, starting with the HTTP error helpers:

status_error is what an error status means on its own, the fallback every provider refines from its error codes and body, so one status means the same on every provider.
error_context reads an error response’s status and headers (request id, retry hint, server retry directive) into an ErrorContext.
error_envelope reads the common JSON error envelopes and keeps the provider’s text out of the redacted ErrorLog (see ErrorEnvelope::apply); error_envelope_value reads one already parsed.
malformed_response is the error for a success response whose body cannot be decoded.
Headers::retry_after reads retry hints.
TransportError::into_error maps a request that got no response, and TransportError::into_stream_error a streamed response whose body could not be read to the end.

And the streaming helpers of every streaming adapter. A framer turns a response body into frames, in linear time and under a size cap (the provider builder’s max_stream_event_size): SseStream into SSE events, in sse, and NdjsonStream into NDJSON lines, in ndjson. Whatever the framing, FramedChatStream drives an adapter’s ChatFrameDecoder over the frames (SseChatStream is its SSE form), and VendorFrames surfaces the frames an adapter does not model, in framing.

It also defines the caller’s per-request transport options, HttpRequestOptions (headers and Credentials, under the shared OPTIONS_NAMESPACE), which every adapter applies with HttpRequest::apply_options and its own CredentialSlot: a credential is addressed to a credential scope, and only the adapter with that scope sends it.

§Trace context propagation

allama-http sends no trace context of its own, and depends on no OpenTelemetry crate. To continue the application’s trace in a gateway that joins distributed traces (LiteLLM proxy, vLLM, an instrumented internal gateway), wrap the transport in an HttpTransport that injects the current span’s W3C traceparent into HttpRequest::headers before it sends, with the application’s own OpenTelemetry version. Adapters call HttpRequest::apply_options before send, so the wrapper sees the final headers and can keep a traceparent the caller set. The observability guide has the recipe, for provider builders and for the provider registry.

§Timeouts

Every layer bounds a different phase, and each one can be turned off:

Bound	Where	Default	Disable
Connect (TCP + TLS)	`ReqwestTransport::new`	`DEFAULT_CONNECT_TIMEOUT` (not on `wasm32`)	pass your own client to `ReqwestTransport::from_client`
Whole non-streaming exchange	`HttpRequest::timeout`, set by the provider	`DEFAULT_REQUEST_TIMEOUT` in the bundled crates	the provider builder’s `timeout`
Stream headers, gap between stream chunks	`StreamTimeouts`, applied by the provider	`StreamTimeouts::default`	the provider builder’s `stream_timeouts`

allama::wrappers::TimeoutProvider sits above all of them and bounds decoded calls and stream events, with the same Error::Timeout shape. Whichever bound is shorter fires first, so use TimeoutProvider for tight, workload-specific limits and leave these defaults as the dead-connection safety net.

§Secrets

The Debug output of Headers (and so of HttpRequest and HttpResponse) redacts credential headers (see is_sensitive_header) and every credential HttpRequest::apply_options placed. Transport errors drop the URL’s user info, query and fragment, and the bundled providers’ Debug output shows base URLs through config::redact_url.

§Network defaults

What a broken or hostile endpoint (a tenant-configured gateway, a compromised proxy) can do to the caller is bounded:

No redirects. ReqwestTransport::new never follows one, and the bundled providers report a 3xx as an UnexpectedResponse naming the target host (redirect_error): following it would send the key and the prompt to a host the application never configured. A client you pass to ReqwestTransport::from_client keeps its own redirect policy; on wasm32, the host fetch’s applies.
Bounded bodies. Success bodies are read with HttpResponse::bytes_limited under the builder’s max_response_size (DEFAULT_MAX_RESPONSE_SIZE), error bodies with HttpResponse::map_error (at most MAX_ERROR_BODY_SIZE, within ERROR_BODY_TIMEOUT), and stream events under max_stream_event_size. A whole stream has no byte cap: its length is the answer’s, which the request bounds with max_output_tokens, and its duration is bounded by the stream timeouts (StreamTimeouts, and TimeoutProvider::stream_total in allama). A byte cap would cut off a long answer that is legitimate, mid-stream. To bound what one call may cost, count its usage, or its bytes, in a wrapper around the stream.
Checked configuration. config refuses malformed base URLs, keys no header can carry, and credential headers among a builder’s static headers, at build().

§Versioning

allama-http names these third-party crates in its public API, so moving any of them to a new major (minor, before 1.0) version is a breaking release of allama-http:

bytes 1: Bytes, re-exported, carries request and response bodies;
futures-core 0.3: the Body stream and the framers’ stream bounds;
serde and serde_json 1: HttpRequestOptions’ JSON form, ReplayResponse::json_value and the JSON the error and framing helpers read;
reqwest 0.13 (feature reqwest): ReqwestTransport::from_client;
reqwest-middleware 0.5 (feature reqwest-middleware): ReqwestTransport::from_middleware.

The provider crates take a client only as an HttpTransport (http_transport(ReqwestTransport::from_client(..))), so reqwest appears in no other public signature.

The allama crates are released together, at one version, for now. So a breaking release of allama-http is today a breaking release of every allama crate. That is how 0.1 is released, not a promise about every later release. The reason is HttpRequestOptions: options are found by their type, so with allama-http at two versions over one allama, an adapter built on one version reads a different type than the one attached with the other. The call fails with Error::InvalidRequest, naming the allama-http namespace and both types, instead of sending the configured key without the tenant’s credential and headers; with one version for every crate, the same mix is a compile error.

use allama_http::{HttpRequest, HttpTransport, ReplayResponse, ReplayTransport};

let transport = ReplayTransport::new([ReplayResponse::json(200, r#"{"ok":true}"#)]);
let response = transport
    .send(HttpRequest::post("https://api.example.com/v1/chat", "{}"))
    .await?;
assert_eq!(response.status, 200);
assert_eq!(transport.requests()[0].url, "https://api.example.com/v1/chat");

Modules§

config: Checking a provider builder’s configuration: base URLs, secrets, and the headers a provider sends with every request, and how a URL shows in Debug output.
framing: Streaming for any wire framing: the chat stream driver and vendor frames.
ndjson: Newline-delimited JSON (NDJSON) for streaming adapters.
sse: Server-sent events (SSE) for streaming adapters.

Structs§

Bytes: A cheaply cloneable and sliceable chunk of contiguous memory.
Credential: A credential for one request: a typed, redacted secret.
CredentialSlot: Where an adapter takes its per-request credential from, and where it puts it: the adapter’s side of a Credential.
ErrorEnvelope: The fields of a recognised JSON error envelope (see error_envelope).
Headers: HTTP headers, matched case-insensitively and kept in order.
HttpRequest: An HTTP request built by a provider.
HttpRequestOptions: Transport options for one request: HTTP headers and credentials to send with it.
HttpResponse: An HTTP response: status and headers, then a streamed body.
ReplayResponse: A canned response served by ReplayTransport.
ReplayTransport: An HttpTransport that serves canned responses in order and records every request.
ReqwestTransportreqwest: An HttpTransport backed by reqwest.
StreamTimeouts: Time bounds for a streaming request, enforced above the transport.
TransportError: Why a request got no (complete) response.

Enums§

CredentialKind: The kind of a Credential.
CredentialPlacement: Where an adapter puts its scope’s Credential::api_key.
Method: HTTP method.
TransportErrorKind: Kinds of TransportError.

Constants§

DEFAULT_CONNECT_TIMEOUTreqwest: The connect timeout (TCP and TLS handshake) of ReqwestTransport::new: 10 s.
DEFAULT_MAX_RESPONSE_SIZE: The default limit on a success response’s body that the bundled providers read whole (every non-streaming call): 256 MiB. Change it with the provider builder’s max_response_size.
DEFAULT_REQUEST_TIMEOUT: The bound on a whole non-streaming call that the bundled providers set on every unary request (HttpRequest::timeout) unless the builder’s timeout is set: 120 s, whatever the transport. Only timeout(None) removes it.
DEFAULT_USER_AGENT: The User-Agent the bundled providers send unless the builder’s header("user-agent", ..) replaces it: allama/<version>.
ERROR_BODY_TIMEOUT: How long HttpResponse::map_error reads an error response’s body: 30 s. A streaming request has no total deadline, only bounds on the wait for each chunk, so a body that keeps trickling would otherwise hold the call forever.
MAX_ERROR_BODY_SIZE: The most of an error response’s body HttpResponse::map_error reads: 64 KiB. An error keeps only ErrorContext::MAX_BODY_BYTES of it.
OPTIONS_NAMESPACE: The options namespace of HttpRequestOptions: "allama-http".

Traits§

HttpTransport: Sends HTTP requests for a provider.

Functions§

error_context: The diagnostic context of an HTTP error response, read from its status and headers:
error_envelope: Recognise a JSON error envelope in an error response body or an in-band error event.
error_envelope_value: error_envelope of a body already parsed as JSON, for a caller that parsed it to tell an error from its other frames (an NDJSON line, an SSE event’s data). It recognises the same shapes, listed on error_envelope.
is_credential_header: Returns true when name is a header that carries a credential by definition, so a plain header with that name is refused (see HttpRequest::apply_options).
is_sensitive_header: Returns true when a header named name likely carries a credential, so its value must not be logged.
malformed_response: The error for a success response whose body could not be decoded: an HTML page from a proxy, a truncated body, or JSON of another shape.
parse_retry_after: Parse a retry-after hint from header values.
redirect_error: The error for a redirect (3xx) response, which the bundled providers never follow: an allama::Error::UnexpectedResponse with the status, the response’s error_context, and an authored message naming the host the Location header points to (never its path or query, which may carry tokens). It is not retryable: the server answers the same way again.
status_error: The error an HTTP error status indicates on its own, before the body is read.

Type Aliases§

Body: A response body: a stream of byte chunks.
SendFuture: A boxed future returned by HttpTransport::send.