allama_http/lib.rs
1#![forbid(unsafe_code)]
2#![cfg_attr(docsrs, feature(doc_cfg))]
3//! The HTTP transport seam of allama's HTTP provider crates.
4//!
5//! Provider crates (`allama-anthropic`, `allama-gemini`,
6//! `allama-openai-compat`, `allama-openai`, `allama-systemone`) build an
7//! [`HttpRequest`], send it through an [`HttpTransport`], and decode the [`HttpResponse`]. Their
8//! request mapping, error classification and decoding never touch an HTTP
9//! client directly, so the transport can be swapped:
10//!
11//! - [`ReqwestTransport`] (feature `reqwest`, default) is what providers use
12//! unless told otherwise.
13//! - [`ReplayTransport`] serves canned responses in memory and records the
14//! requests: transport-level fixtures that exercise the whole adapter
15//! without a server, on any target.
16//! - Custom transports cover other HTTP stacks, proxies, or recording.
17//!
18//! This is a seam for provider crates, not middleware for applications: it
19//! carries bytes, and the portable `allama` types stay above it. Beside the
20//! seam, it holds the helpers every provider shares, first-party or not,
21//! starting with the HTTP error helpers:
22//!
23//! - [`status_error`] is what an error status means on its own, the
24//! fallback every provider refines from its error codes and body, so one
25//! status means the same on every provider.
26//! - [`error_context`] reads an error response's status and headers
27//! (request id, retry hint, server retry directive) into an
28//! [`ErrorContext`](allama::ErrorContext).
29//! - [`error_envelope`] reads the common JSON error envelopes and keeps the
30//! provider's text out of the redacted
31//! [`ErrorLog`](allama::ErrorLog) (see [`ErrorEnvelope::apply`]);
32//! [`error_envelope_value`] reads one already parsed.
33//! - [`malformed_response`] is the error for a success response whose body
34//! cannot be decoded.
35//! - [`Headers::retry_after`] reads retry hints.
36//! - [`TransportError::into_error`] maps a request that got no response,
37//! and [`TransportError::into_stream_error`] a streamed response whose
38//! body could not be read to the end.
39//!
40//! And the streaming helpers of every streaming adapter. A framer turns a
41//! response body into frames, in linear time and under a size cap (the
42//! provider builder's `max_stream_event_size`):
43//! [`SseStream`](sse::SseStream) into SSE events, in [`sse`], and
44//! [`NdjsonStream`](ndjson::NdjsonStream) into NDJSON lines, in [`ndjson`].
45//! Whatever the framing, [`FramedChatStream`](framing::FramedChatStream)
46//! drives an adapter's [`ChatFrameDecoder`](framing::ChatFrameDecoder) over
47//! the frames ([`SseChatStream`](sse::SseChatStream) is its SSE form), and
48//! [`VendorFrames`](framing::VendorFrames) surfaces the frames an adapter
49//! does not model, in [`framing`].
50//!
51//! It also defines the caller's per-request transport options,
52//! [`HttpRequestOptions`] (headers and [`Credential`]s, under the shared
53//! [`OPTIONS_NAMESPACE`]), which every adapter applies with
54//! [`HttpRequest::apply_options`] and its own [`CredentialSlot`]: a
55//! credential is addressed to a credential scope, and only the adapter with
56//! that scope sends it.
57//!
58//! # Trace context propagation
59//!
60//! `allama-http` sends no trace context of its own, and depends on no
61//! OpenTelemetry crate. To continue the application's trace in a gateway
62//! that joins distributed traces (LiteLLM proxy, vLLM, an instrumented
63//! internal gateway), wrap the transport in an [`HttpTransport`] that
64//! injects the current span's W3C `traceparent` into
65//! [`HttpRequest::headers`] before it sends, with the application's own
66//! OpenTelemetry version. Adapters call [`HttpRequest::apply_options`]
67//! before [`send`](HttpTransport::send), so the wrapper sees the final
68//! headers and can keep a `traceparent` the caller set. The
69//! [observability guide]
70//! has the recipe, for provider builders and for the provider registry.
71//!
72//! # Timeouts
73//!
74//! Every layer bounds a different phase, and each one can be turned off:
75//!
76//! | Bound | Where | Default | Disable |
77//! | --- | --- | --- | --- |
78//! | Connect (TCP + TLS) | [`ReqwestTransport::new`] | [`DEFAULT_CONNECT_TIMEOUT`] (not on `wasm32`) | pass your own client to [`ReqwestTransport::from_client`] |
79//! | Whole non-streaming exchange | [`HttpRequest::timeout`], set by the provider | [`DEFAULT_REQUEST_TIMEOUT`] in the bundled crates | the provider builder's `timeout` |
80//! | Stream headers, gap between stream chunks | [`StreamTimeouts`], applied by the provider | [`StreamTimeouts::default`] | the provider builder's `stream_timeouts` |
81//!
82//! `allama::wrappers::TimeoutProvider` sits above all of them and bounds
83//! decoded calls and stream events, with the same `Error::Timeout` shape.
84//! Whichever bound is shorter fires first, so use `TimeoutProvider` for
85//! tight, workload-specific limits and leave these defaults as the
86//! dead-connection safety net.
87//!
88//! # Secrets
89//!
90//! The `Debug` output of [`Headers`] (and so of [`HttpRequest`] and
91//! [`HttpResponse`]) redacts credential headers (see
92//! [`is_sensitive_header`]) and every credential
93//! [`HttpRequest::apply_options`] placed. Transport errors drop the URL's
94//! user info, query and fragment, and the bundled providers' `Debug`
95//! output shows base URLs through [`config::redact_url`].
96//!
97//! # Network defaults
98//!
99//! What a broken or hostile endpoint (a tenant-configured gateway, a
100//! compromised proxy) can do to the caller is bounded:
101//!
102//! - **No redirects.** [`ReqwestTransport::new`] never follows one, and the
103//! bundled providers report a `3xx` as an
104//! [`UnexpectedResponse`](allama::Error::UnexpectedResponse) naming the
105//! target host ([`redirect_error`]): following it would send the key and
106//! the prompt to a host the application never configured. A client you
107//! pass to [`ReqwestTransport::from_client`] keeps its own redirect
108//! policy; on `wasm32`, the host `fetch`'s applies.
109//! - **Bounded bodies.** Success bodies are read with
110//! [`HttpResponse::bytes_limited`] under the builder's
111//! `max_response_size` ([`DEFAULT_MAX_RESPONSE_SIZE`]), error bodies with
112//! [`HttpResponse::map_error`] (at most [`MAX_ERROR_BODY_SIZE`], within
113//! [`ERROR_BODY_TIMEOUT`]), and stream events under
114//! `max_stream_event_size`. A whole stream has no byte cap: its length is
115//! the answer's, which the request bounds with `max_output_tokens`, and its
116//! duration is bounded by the stream timeouts ([`StreamTimeouts`], and
117//! `TimeoutProvider::stream_total` in allama). A byte cap would cut off a
118//! long answer that is legitimate, mid-stream. To bound what one call may
119//! cost, count its usage, or its bytes, in a wrapper around the stream.
120//! - **Checked configuration.** [`config`] refuses malformed base URLs,
121//! keys no header can carry, and credential headers among a builder's
122//! static headers, at `build()`.
123//!
124//! # Versioning
125//!
126//! `allama-http` names these third-party crates in its public API, so
127//! moving any of them to a new major (minor, before 1.0) version is a
128//! breaking release of `allama-http`:
129//!
130//! - `bytes` 1: [`Bytes`], re-exported, carries request and response
131//! bodies;
132//! - `futures-core` 0.3: the [`Body`] stream and the framers' stream
133//! bounds;
134//! - `serde` and `serde_json` 1: [`HttpRequestOptions`]' JSON form,
135//! [`ReplayResponse::json_value`] and the JSON the error and framing
136//! helpers read;
137//! - `reqwest` 0.13 (feature `reqwest`): [`ReqwestTransport::from_client`];
138//! - `reqwest-middleware` 0.5 (feature `reqwest-middleware`):
139//! `ReqwestTransport::from_middleware`.
140//!
141//! The provider crates take a client only as an [`HttpTransport`]
142//! (`http_transport(ReqwestTransport::from_client(..))`), so `reqwest`
143//! appears in no other public signature.
144//!
145//! The allama crates are released together, at one version, for now. So a
146//! breaking release of `allama-http` is today a breaking release of every
147//! allama crate. That is how 0.1 is released, not a promise about every
148//! later release. The reason is [`HttpRequestOptions`]: options are found
149//! by their type, so with `allama-http` at two versions over one `allama`,
150//! an adapter built on one version reads a different type than the one
151//! attached with the other. The call fails with
152//! [`Error::InvalidRequest`](allama::Error::InvalidRequest), naming the
153//! `allama-http` namespace and both types, instead of sending the
154//! configured key without the tenant's credential and headers; with one
155//! version for every crate, the same mix is a compile error.
156//!
157//! ```
158//! use allama_http::{HttpRequest, HttpTransport, ReplayResponse, ReplayTransport};
159//!
160//! # async fn example() -> Result<(), allama_http::TransportError> {
161//! let transport = ReplayTransport::new([ReplayResponse::json(200, r#"{"ok":true}"#)]);
162//! let response = transport
163//! .send(HttpRequest::post("https://api.example.com/v1/chat", "{}"))
164//! .await?;
165//! assert_eq!(response.status, 200);
166//! assert_eq!(transport.requests()[0].url, "https://api.example.com/v1/chat");
167//! # Ok(()) }
168//! ```
169//!
170#![doc = concat!(
171 "[observability guide]: https://github.com/sagikazarmark/allama/blob/v",
172 env!("CARGO_PKG_VERSION"),
173 "/docs/guides/observability/README.md#trace-context-propagation",
174)]
175#![cfg_attr(
176 not(feature = "reqwest"),
177 doc = "[`ReqwestTransport`]: crate\n[`ReqwestTransport::new`]: crate\n[`DEFAULT_CONNECT_TIMEOUT`]: crate\n[`ReqwestTransport::from_client`]: crate"
178)]
179#![warn(missing_docs)]
180#![warn(missing_debug_implementations)]
181
182use std::fmt;
183use std::future::Future;
184use std::pin::Pin;
185use std::sync::Arc;
186use std::time::{Duration, SystemTime, UNIX_EPOCH};
187
188pub use bytes::Bytes;
189use futures_core::Stream;
190use futures_util::StreamExt;
191
192pub mod config;
193mod credential;
194mod error;
195pub mod framing;
196pub mod ndjson;
197mod options;
198mod replay;
199#[cfg(feature = "reqwest")]
200mod reqwest;
201mod retry_after;
202pub mod sse;
203mod timeout;
204
205pub use credential::{
206 Credential, CredentialKind, CredentialPlacement, CredentialSlot, is_credential_header,
207};
208pub use error::{
209 ErrorEnvelope, error_context, error_envelope, error_envelope_value, malformed_response,
210 status_error,
211};
212pub use options::{HttpRequestOptions, OPTIONS_NAMESPACE};
213pub use replay::{ReplayResponse, ReplayTransport};
214#[cfg(feature = "reqwest")]
215pub use reqwest::{DEFAULT_CONNECT_TIMEOUT, ReqwestTransport};
216pub use retry_after::parse_retry_after;
217pub use timeout::StreamTimeouts;
218
219/// A boxed future returned by [`HttpTransport::send`].
220pub type SendFuture<'a> =
221 Pin<Box<dyn Future<Output = Result<HttpResponse, TransportError>> + Send + 'a>>;
222
223/// A response body: a stream of byte chunks.
224pub type Body = Pin<Box<dyn Stream<Item = Result<Bytes, TransportError>> + Send>>;
225
226/// Sends HTTP requests for a provider.
227///
228/// Implementations return the response as soon as its status and headers are
229/// known; the body streams afterwards (SSE responses are read incrementally).
230/// A response with an error status is still `Ok`: classifying HTTP errors is
231/// the provider's job. `Err` means no response arrived.
232pub trait HttpTransport: Send + Sync + 'static {
233 /// Send `request`.
234 fn send(&self, request: HttpRequest) -> SendFuture<'_>;
235}
236
237impl<T: HttpTransport + ?Sized> HttpTransport for Arc<T> {
238 fn send(&self, request: HttpRequest) -> SendFuture<'_> {
239 T::send(self, request)
240 }
241}
242
243impl<T: HttpTransport + ?Sized> HttpTransport for Box<T> {
244 fn send(&self, request: HttpRequest) -> SendFuture<'_> {
245 T::send(self, request)
246 }
247}
248
249/// HTTP method.
250#[derive(Debug, Clone, Copy, PartialEq, Eq, Hash)]
251#[non_exhaustive]
252pub enum Method {
253 /// `GET`.
254 Get,
255 /// `POST`.
256 Post,
257}
258
259impl Method {
260 /// The method name.
261 #[must_use]
262 pub fn as_str(self) -> &'static str {
263 match self {
264 Self::Get => "GET",
265 Self::Post => "POST",
266 }
267 }
268}
269
270/// An HTTP request built by a provider.
271///
272/// `Debug` output redacts credential headers (see [`Headers`]).
273#[derive(Debug, Clone, PartialEq, Eq)]
274#[non_exhaustive]
275pub struct HttpRequest {
276 /// Method.
277 pub method: Method,
278 /// Absolute URL.
279 pub url: String,
280 /// Headers, in order.
281 pub headers: Headers,
282 /// Body, if any.
283 pub body: Option<Bytes>,
284 /// Bound on the whole exchange, when the provider sets one. Streaming
285 /// requests usually leave it unset.
286 pub timeout: Option<Duration>,
287}
288
289impl HttpRequest {
290 /// A `GET` request.
291 #[must_use]
292 pub fn get(url: impl Into<String>) -> Self {
293 Self {
294 method: Method::Get,
295 url: url.into(),
296 headers: Headers::new(),
297 body: None,
298 timeout: None,
299 }
300 }
301
302 /// A `POST` request with `body`.
303 #[must_use]
304 pub fn post(url: impl Into<String>, body: impl Into<Bytes>) -> Self {
305 Self {
306 method: Method::Post,
307 url: url.into(),
308 headers: Headers::new(),
309 body: Some(body.into()),
310 timeout: None,
311 }
312 }
313
314 /// Add a header.
315 #[must_use]
316 pub fn header(mut self, name: impl Into<String>, value: impl Into<String>) -> Self {
317 self.headers.insert(name, value);
318 self
319 }
320
321 /// Set (or clear) the timeout.
322 #[must_use]
323 pub fn timeout(mut self, timeout: Option<Duration>) -> Self {
324 self.timeout = timeout;
325 self
326 }
327
328 /// The body as UTF-8 text, if it is valid UTF-8.
329 #[must_use]
330 pub fn body_text(&self) -> Option<&str> {
331 std::str::from_utf8(self.body.as_deref()?).ok()
332 }
333
334 /// Add a header whose value is a secret: `Debug` output redacts it
335 /// whatever its name. For an adapter that places a credential itself
336 /// ([`CredentialPlacement::Custom`]).
337 #[must_use]
338 pub fn sensitive_header(mut self, name: impl Into<String>, value: impl Into<String>) -> Self {
339 self.headers.insert_sensitive(name, value);
340 self
341 }
342
343 /// Set each of `headers`, replacing the request's headers of the same
344 /// name (ignoring case) and adding the others, in order. For the
345 /// headers a provider sends with every request: the bundled providers
346 /// set their [`DEFAULT_USER_AGENT`] and the builder's own headers
347 /// (`header(name, value)`, validated by [`config::static_headers`])
348 /// with it, after their protocol headers and before
349 /// [`apply_options`](Self::apply_options), so a request's own headers
350 /// still win.
351 ///
352 /// ```
353 /// use allama_http::{Headers, HttpRequest};
354 ///
355 /// let defaults: Headers = [("user-agent", "my-app/1.0"), ("x-route", "eu")].into_iter().collect();
356 /// let request = HttpRequest::post("https://api.example.com/v1/chat", "{}")
357 /// .header("User-Agent", "allama/0.1.0")
358 /// .with_headers(&defaults);
359 /// assert_eq!(request.headers.get("user-agent"), Some("my-app/1.0"));
360 /// assert_eq!(request.headers.iter().count(), 2);
361 /// ```
362 #[must_use]
363 pub fn with_headers(mut self, headers: &Headers) -> Self {
364 for (name, _) in headers.iter() {
365 self.headers.remove(name);
366 }
367 for header in &headers.0 {
368 self.headers.0.push(header.clone());
369 }
370 self
371 }
372
373 /// Apply the caller's transport options for this request: the
374 /// [`HttpRequestOptions`] in `options`, typed or in their JSON form,
375 /// with the credentials addressed to `slot`'s scope.
376 ///
377 /// Call it once the request carries every header the adapter sets,
378 /// including its configured credential, and before sending it. Every
379 /// bundled HTTP adapter calls it with the options of the request it
380 /// serves and its own [`CredentialSlot`], so callers get the same
381 /// behaviour from first- and third-party adapters. In order, it:
382 ///
383 /// 1. refuses, before anything is sent, a plain header that would carry
384 /// a credential (see Errors);
385 /// 2. replaces the request's headers with the plain headers of the same
386 /// name, and adds the others;
387 /// 3. places the [`Credential::api_key`] addressed to the slot's scope
388 /// as the slot's [placement](CredentialSlot::placement) says,
389 /// removing the configured credential and every header the slot
390 /// [replaces](CredentialSlot::replaces). A
391 /// [`Custom`](CredentialPlacement::Custom) slot is left alone: the
392 /// adapter signs or places its credential after this call;
393 /// 4. adds the [`Credential::header`] values addressed to the scope,
394 /// each replacing the headers of the same name.
395 ///
396 /// Credentials addressed to other scopes are never sent. Placed
397 /// credentials are redacted from `Debug` whatever the header name.
398 ///
399 /// ```
400 /// use allama::ChatRequest;
401 /// use allama_http::{Credential, CredentialPlacement, CredentialSlot, HttpRequest, HttpRequestOptions};
402 ///
403 /// let slot = CredentialSlot::new("anthropic", CredentialPlacement::Header("x-api-key".into()));
404 /// let request = ChatRequest::new("m").user("Hi").with_option(
405 /// HttpRequestOptions::new()
406 /// .credential("anthropic", Credential::api_key("tenant-key"))
407 /// .credential("openai", Credential::api_key("other-account"))
408 /// .header("x-tenant-id", "acme"),
409 /// );
410 /// let http = HttpRequest::post("https://api.example.com/v1/messages", "{}")
411 /// .header("x-api-key", "provider-key")
412 /// .apply_options(&request.options, &slot)?;
413 /// assert_eq!(http.headers.get("x-api-key"), Some("tenant-key"));
414 /// assert_eq!(http.headers.get("x-tenant-id"), Some("acme"));
415 /// assert!(!format!("{http:?}").contains("other-account"));
416 /// # Ok::<(), allama::Error>(())
417 /// ```
418 ///
419 /// # Errors
420 ///
421 /// - [`allama::Error::InvalidRequest`] when the JSON form of the options
422 /// is malformed, names a header on the exact credential list
423 /// ([`is_credential_header`]), or sets credentials.
424 /// - An authored [`allama::Error::Configuration`], before anything is
425 /// sent, when a plain header is named on the exact credential list
426 /// ([`is_credential_header`]) or is one the slot writes or replaces,
427 /// or when the slot is [required](CredentialSlot::required) and no
428 /// credential is addressed to its scope. These are the application's
429 /// bugs, not the end user's, so they are not retried, never fall over
430 /// to another provider, and map to a server error. Error text names
431 /// headers and scopes, never secrets.
432 pub fn apply_options(
433 mut self,
434 options: &allama::RequestOptions,
435 slot: &CredentialSlot,
436 ) -> allama::Result<Self> {
437 let transport = options.resolve::<HttpRequestOptions>()?;
438 let transport = transport.as_deref();
439 if let Some(transport) = transport {
440 refuse_credential_headers(transport, slot)?;
441 }
442 let credential = transport.and_then(|transport| transport.credential_for(&slot.scope));
443 if slot.required && credential.is_none() {
444 return Err(allama::Error::configuration(format!(
445 "no credential is addressed to credential scope `{}`, and the provider has no \
446 configured credential: attach one with `HttpRequestOptions::credential`",
447 slot.scope
448 ))
449 .authored());
450 }
451 let Some(transport) = transport else {
452 return Ok(self);
453 };
454 for (name, _) in transport.headers.iter() {
455 self.headers.remove(name);
456 }
457 for (name, value) in transport.headers.iter() {
458 self.headers.insert(name, value);
459 }
460 if let Some(credential) = credential {
461 self.place(credential, slot)?;
462 }
463 for (name, secret) in transport.header_credentials_for(&slot.scope) {
464 self.headers.remove(name);
465 self.headers.insert_sensitive(name, secret);
466 }
467 Ok(self)
468 }
469
470 /// Place the scope's own `credential` as `slot` says.
471 fn place(&mut self, credential: &Credential, slot: &CredentialSlot) -> allama::Result<()> {
472 let (name, value) = match (&slot.placement, credential.kind()) {
473 (CredentialPlacement::Custom, _) => return Ok(()),
474 (CredentialPlacement::Bearer, CredentialKind::ApiKey) => (
475 "authorization",
476 format!("Bearer {}", credential.expose_secret()),
477 ),
478 (CredentialPlacement::Header(name), CredentialKind::ApiKey) => {
479 (name.as_ref(), credential.expose_secret().to_owned())
480 }
481 _ => return Err(credential.unsupported(&slot.scope)),
482 };
483 self.headers.remove(name);
484 for replaced in &slot.replaces {
485 self.headers.remove(replaced);
486 }
487 self.headers.insert_sensitive(name, value);
488 Ok(())
489 }
490}
491
492/// Refuse the plain headers in `transport` that would carry a credential:
493/// those on the exact credential list, and those `slot` writes or replaces.
494fn refuse_credential_headers(
495 transport: &HttpRequestOptions,
496 slot: &CredentialSlot,
497) -> allama::Result<()> {
498 for (name, _) in transport.headers.iter() {
499 if is_credential_header(name) {
500 return Err(allama::Error::configuration(format!(
501 "the plain header `{name}` carries a credential: send it with \
502 `HttpRequestOptions::credential`, addressed to a credential scope"
503 ))
504 .authored());
505 }
506 if slot.owns_header(name) {
507 return Err(allama::Error::configuration(format!(
508 "the plain header `{name}` is where credential scope `{}` puts its credential: \
509 send it with `HttpRequestOptions::credential`",
510 slot.scope
511 ))
512 .authored());
513 }
514 }
515 Ok(())
516}
517
518/// HTTP headers, matched case-insensitively and kept in order.
519///
520/// `Debug` output redacts the values of credential headers (see
521/// [`is_sensitive_header`]), so requests and responses can be logged.
522#[derive(Clone, Default, PartialEq, Eq)]
523pub struct Headers(Vec<Header>);
524
525impl Headers {
526 /// No headers.
527 #[must_use]
528 pub fn new() -> Self {
529 Self::default()
530 }
531
532 /// Append a header.
533 pub fn insert(&mut self, name: impl Into<String>, value: impl Into<String>) {
534 self.0.push(Header {
535 name: name.into(),
536 value: value.into(),
537 sensitive: false,
538 });
539 }
540
541 /// Append a header whose value is a secret, which `Debug` output
542 /// redacts whatever the header's name.
543 pub fn insert_sensitive(&mut self, name: impl Into<String>, value: impl Into<String>) {
544 self.0.push(Header {
545 name: name.into(),
546 value: value.into(),
547 sensitive: true,
548 });
549 }
550
551 /// Remove every value of `name`, ignoring case.
552 pub fn remove(&mut self, name: &str) {
553 self.0
554 .retain(|header| !header.name.eq_ignore_ascii_case(name));
555 }
556
557 /// The first value of `name`, ignoring case.
558 #[must_use]
559 pub fn get(&self, name: &str) -> Option<&str> {
560 self.0
561 .iter()
562 .find(|header| header.name.eq_ignore_ascii_case(name))
563 .map(|header| header.value.as_str())
564 }
565
566 /// Iterate over `(name, value)` pairs.
567 pub fn iter(&self) -> impl Iterator<Item = (&str, &str)> {
568 self.0
569 .iter()
570 .map(|header| (header.name.as_str(), header.value.as_str()))
571 }
572
573 /// How long the server asks the client to wait before retrying: the
574 /// `name` header (usually `retry-after`) or `retry-after-ms`, parsed
575 /// with [`parse_retry_after`] relative to the current time.
576 ///
577 /// Unlike passing `SystemTime::now()` yourself, this also works on
578 /// `wasm32-unknown-unknown`.
579 #[must_use]
580 pub fn retry_after(&self, name: &str) -> Option<Duration> {
581 parse_retry_after(self.get(name), self.get("retry-after-ms"), now())
582 }
583}
584
585impl fmt::Debug for Headers {
586 fn fmt(&self, f: &mut fmt::Formatter<'_>) -> fmt::Result {
587 f.debug_map()
588 .entries(self.0.iter().map(|header| (&header.name, header)))
589 .finish()
590 }
591}
592
593/// One header of [`Headers`].
594#[derive(Clone, PartialEq, Eq)]
595struct Header {
596 name: String,
597 value: String,
598 /// Set for a placed credential: redacted whatever the name.
599 sensitive: bool,
600}
601
602impl Header {
603 /// Whether `Debug` output redacts the value.
604 fn is_sensitive(&self) -> bool {
605 self.sensitive || is_sensitive_header(&self.name)
606 }
607}
608
609/// The value as `Debug` shows it: redacted when the header is sensitive.
610impl fmt::Debug for Header {
611 fn fmt(&self, f: &mut fmt::Formatter<'_>) -> fmt::Result {
612 if self.is_sensitive() {
613 f.write_str("<redacted>")
614 } else {
615 fmt::Debug::fmt(&self.value, f)
616 }
617 }
618}
619
620/// Returns `true` when a header named `name` likely carries a credential,
621/// so its value must not be logged.
622///
623/// Matches, ignoring case, the exact credential list
624/// ([`is_credential_header`]), `set-cookie`, and any name containing `key`,
625/// `token`, `secret` or `auth`. Erring towards redaction is deliberate: a
626/// rate-limit header hidden from a debug log costs less than a leaked key.
627/// This rule only redacts: a plain header it matches but the exact list
628/// does not, such as `Idempotency-Key`, is still sent.
629///
630/// ```
631/// use allama_http::is_sensitive_header;
632///
633/// assert!(is_sensitive_header("Authorization"));
634/// assert!(is_sensitive_header("x-my-gateway-token"));
635/// assert!(!is_sensitive_header("x-request-id"));
636/// ```
637#[must_use]
638pub fn is_sensitive_header(name: &str) -> bool {
639 const FRAGMENTS: [&str; 4] = ["key", "token", "secret", "auth"];
640 let name = name.to_ascii_lowercase();
641 is_credential_header(&name)
642 || name == "set-cookie"
643 || FRAGMENTS.iter().any(|part| name.contains(part))
644}
645
646/// The current time as a `std` `SystemTime`, read through `web-time`:
647/// `SystemTime::now` panics on `wasm32-unknown-unknown`.
648fn now() -> SystemTime {
649 let since_epoch = web_time::SystemTime::now()
650 .duration_since(web_time::UNIX_EPOCH)
651 .unwrap_or_default();
652 UNIX_EPOCH + since_epoch
653}
654
655impl<K: Into<String>, V: Into<String>> FromIterator<(K, V)> for Headers {
656 fn from_iter<I: IntoIterator<Item = (K, V)>>(iter: I) -> Self {
657 let mut headers = Self::new();
658 for (name, value) in iter {
659 headers.insert(name, value);
660 }
661 headers
662 }
663}
664
665/// An HTTP response: status and headers, then a streamed body.
666#[non_exhaustive]
667pub struct HttpResponse {
668 /// Status code.
669 pub status: u16,
670 /// Headers.
671 pub headers: Headers,
672 body: Body,
673}
674
675impl fmt::Debug for HttpResponse {
676 fn fmt(&self, f: &mut fmt::Formatter<'_>) -> fmt::Result {
677 f.debug_struct("HttpResponse")
678 .field("status", &self.status)
679 .field("headers", &self.headers)
680 .finish_non_exhaustive()
681 }
682}
683
684impl HttpResponse {
685 /// A response with a streamed body.
686 #[must_use]
687 pub fn new(status: u16, headers: Headers, body: Body) -> Self {
688 Self {
689 status,
690 headers,
691 body,
692 }
693 }
694
695 /// A response with the whole body at once.
696 #[must_use]
697 pub fn from_bytes(status: u16, headers: Headers, body: impl Into<Bytes>) -> Self {
698 let body = body.into();
699 Self::new(
700 status,
701 headers,
702 Box::pin(futures_util::stream::once(async move { Ok(body) })),
703 )
704 }
705
706 /// Returns `true` for 2xx statuses.
707 #[must_use]
708 pub fn is_success(&self) -> bool {
709 (200..300).contains(&self.status)
710 }
711
712 /// Read the whole body, however large.
713 ///
714 /// For a response from a server you do not control, read with
715 /// [`bytes_limited`](Self::bytes_limited) instead: a broken or hostile
716 /// server can send a body that does not fit in memory.
717 ///
718 /// # Errors
719 ///
720 /// The transport failed while the body was read.
721 pub async fn bytes(self) -> Result<Bytes, TransportError> {
722 self.bytes_limited(usize::MAX).await
723 }
724
725 /// Read the whole body as text (invalid UTF-8 is replaced), however
726 /// large. See [`bytes`](Self::bytes); prefer
727 /// [`text_limited`](Self::text_limited).
728 ///
729 /// # Errors
730 ///
731 /// The transport failed while the body was read.
732 pub async fn text(self) -> Result<String, TransportError> {
733 self.text_limited(usize::MAX).await
734 }
735
736 /// Read the whole body, failing as soon as it passes `limit` bytes. The
737 /// bundled providers read success bodies with it, under the builder's
738 /// `max_response_size` (default [`DEFAULT_MAX_RESPONSE_SIZE`]).
739 ///
740 /// The buffer never grows past `limit`: the chunk that would take it
741 /// past is refused before it is copied, and the rest of the body is
742 /// never read.
743 ///
744 /// ```
745 /// use allama_http::{Headers, HttpResponse, TransportErrorKind};
746 ///
747 /// # futures_util::FutureExt::now_or_never(async {
748 /// let response = HttpResponse::from_bytes(200, Headers::new(), "0123456789");
749 /// let err = response.bytes_limited(4).await.unwrap_err();
750 /// assert_eq!(err.kind(), TransportErrorKind::TooLarge);
751 /// assert!(matches!(err.into_error(), allama::Error::UnexpectedResponse { .. }));
752 /// # }).unwrap();
753 /// ```
754 ///
755 /// # Errors
756 ///
757 /// - A [`TransportErrorKind::TooLarge`] error once the body passes
758 /// `limit`, with an [authored](TransportError::authored) message. It
759 /// maps to [`allama::Error::UnexpectedResponse`], which is not
760 /// retryable: asking again returns the same body.
761 /// - The transport's error when reading the body failed.
762 pub async fn bytes_limited(self, limit: usize) -> Result<Bytes, TransportError> {
763 let mut body = self.body;
764 let mut buffer = Vec::new();
765 while let Some(chunk) = body.next().await {
766 let chunk = chunk?;
767 if chunk.len() > limit.saturating_sub(buffer.len()) {
768 return Err(TransportError::new(
769 TransportErrorKind::TooLarge,
770 format!("the response body is larger than the limit of {limit} bytes"),
771 )
772 .authored());
773 }
774 if buffer.is_empty() && body_is_done(&body) {
775 return Ok(chunk);
776 }
777 buffer.extend_from_slice(&chunk);
778 }
779 Ok(buffer.into())
780 }
781
782 /// Read the whole body as text (invalid UTF-8 is replaced), failing as
783 /// soon as it passes `limit` bytes, as
784 /// [`bytes_limited`](Self::bytes_limited) does.
785 ///
786 /// # Errors
787 ///
788 /// See [`bytes_limited`](Self::bytes_limited).
789 pub async fn text_limited(self, limit: usize) -> Result<String, TransportError> {
790 let bytes = self.bytes_limited(limit).await?;
791 Ok(String::from_utf8_lossy(&bytes).into_owned())
792 }
793
794 /// The error for an error response (one that is not a success): reads
795 /// the start of its body for diagnostics and maps the status, headers
796 /// and body text with `map`, the adapter's own classification.
797 ///
798 /// Every bundled HTTP provider maps its error responses here, so these
799 /// rules hold on all of them:
800 ///
801 /// - **The body read is bounded.** At most [`MAX_ERROR_BODY_SIZE`]
802 /// bytes are read, within [`ERROR_BODY_TIMEOUT`]; the rest of the
803 /// body is never read. An error keeps only the first
804 /// [`ErrorContext::MAX_BODY_BYTES`](allama::ErrorContext::MAX_BODY_BYTES)
805 /// of it anyway, and a streaming request has no total deadline, so an
806 /// unbounded read of a body that never ends would hang the call and
807 /// grow without limit.
808 /// - **A failed read is recorded, not hidden.** `map` gets the text read
809 /// before the failure, and the failure (a transport error, or the time
810 /// bound) is added to the error's
811 /// [upstream message](allama::Error::upstream_message).
812 /// - **A redirect is never followed.** A `3xx` is the
813 /// [`redirect_error`] for the response, and `map` is not called: an
814 /// LLM API never answers a request with a redirect, and following one
815 /// would send the request, credentials included, somewhere the
816 /// application never configured.
817 ///
818 /// ```
819 /// use allama::Error;
820 /// use allama_http::{HttpResponse, Headers, error_context, status_error};
821 ///
822 /// # futures_util::FutureExt::now_or_never(async {
823 /// let map = |status: u16, headers: &Headers, body: &str| {
824 /// status_error(status, format!("HTTP {status}"))
825 /// .with_context(error_context(status, headers))
826 /// .authored()
827 /// .with_body(body.to_owned())
828 /// };
829 ///
830 /// let response = HttpResponse::from_bytes(429, Headers::new(), "slow down");
831 /// let err = response.map_error(map).await;
832 /// assert!(matches!(err, Error::RateLimited { .. }));
833 /// assert_eq!(err.context().and_then(|c| c.capped_body()), Some("slow down"));
834 ///
835 /// let headers = [("location", "https://elsewhere.example/v1/chat")].into_iter().collect();
836 /// let redirect = HttpResponse::from_bytes(307, headers, "");
837 /// let err = redirect.map_error(map).await;
838 /// assert!(matches!(err, Error::UnexpectedResponse { .. }));
839 /// assert!(err.to_string().contains("elsewhere.example"));
840 /// # }).unwrap();
841 /// ```
842 pub async fn map_error(
843 self,
844 map: impl FnOnce(u16, &Headers, &str) -> allama::Error,
845 ) -> allama::Error {
846 if (300..400).contains(&self.status) {
847 return redirect_error(self.status, &self.headers);
848 }
849 let (body, failure) =
850 read_error_body(self.body, MAX_ERROR_BODY_SIZE, ERROR_BODY_TIMEOUT).await;
851 let error = map(self.status, &self.headers, &String::from_utf8_lossy(&body));
852 let Some(failure) = failure else {
853 return error;
854 };
855 let note = format!("reading the error body failed: {failure}");
856 let message = match error.upstream_message() {
857 Some(message) => format!("{message} ({note})"),
858 None => note,
859 };
860 error.with_upstream_message(message)
861 }
862
863 /// The body as a stream of chunks (for SSE).
864 #[must_use]
865 pub fn into_body(self) -> Body {
866 self.body
867 }
868}
869
870/// The bound on a whole non-streaming call that the bundled providers set
871/// on every unary request ([`HttpRequest::timeout`]) unless the builder's
872/// `timeout` is set: 120 s, whatever the transport. Only `timeout(None)`
873/// removes it.
874///
875/// An adapter of your own sets it too: a transport adds no total deadline
876/// of its own, so without it a server that accepts the connection and never
877/// answers holds a unary call forever. Streaming requests take
878/// [`StreamTimeouts`] instead.
879pub const DEFAULT_REQUEST_TIMEOUT: Duration = Duration::from_secs(120);
880
881/// The default limit on a success response's body that the bundled
882/// providers read whole (every non-streaming call): 256 MiB. Change it with
883/// the provider builder's `max_response_size`.
884///
885/// It is a bound against a broken or hostile server, not a size any API
886/// sends: a chat response is kilobytes, but a full batch of large embeddings
887/// in JSON (2,048 inputs of 3,072 dimensions) is over 128 MiB. Streamed
888/// responses are bounded per event instead (`max_stream_event_size`).
889pub const DEFAULT_MAX_RESPONSE_SIZE: usize = 256 * 1024 * 1024;
890
891/// The most of an error response's body [`HttpResponse::map_error`] reads:
892/// 64 KiB. An error keeps only
893/// [`ErrorContext::MAX_BODY_BYTES`](allama::ErrorContext::MAX_BODY_BYTES)
894/// of it.
895pub const MAX_ERROR_BODY_SIZE: usize = 64 * 1024;
896
897/// How long [`HttpResponse::map_error`] reads an error response's body:
898/// 30 s. A streaming request has no total deadline, only bounds on the wait
899/// for each chunk, so a body that keeps trickling would otherwise hold the
900/// call forever.
901pub const ERROR_BODY_TIMEOUT: Duration = Duration::from_secs(30);
902
903/// The `User-Agent` the bundled providers send unless the builder's
904/// `header("user-agent", ..)` replaces it: `allama/<version>`.
905pub const DEFAULT_USER_AGENT: &str = concat!("allama/", env!("CARGO_PKG_VERSION"));
906
907/// The error for a redirect (`3xx`) response, which the bundled providers
908/// never follow: an [`allama::Error::UnexpectedResponse`] with the status,
909/// the response's [`error_context`], and an
910/// [authored](allama::Error::authored) message naming the host the
911/// `Location` header points to (never its path or query, which may carry
912/// tokens). It is not retryable: the server answers the same way again.
913///
914/// [`ReqwestTransport::new`] does not follow redirects, so the bundled
915/// providers see the redirect itself and map it here, through
916/// [`HttpResponse::map_error`].
917///
918/// ```
919/// use allama::Error;
920/// use allama_http::{Headers, redirect_error};
921///
922/// let headers: Headers = [("location", "http://169.254.169.254/latest?token=x")].into_iter().collect();
923/// let err = redirect_error(307, &headers);
924/// assert!(matches!(err, Error::UnexpectedResponse { .. }));
925/// assert_eq!(err.status(), Some(307));
926/// assert!(!err.is_retryable());
927/// assert_eq!(
928/// err.to_string(),
929/// "unexpected response: HTTP 307: the server redirected the request to `169.254.169.254`; \
930/// redirects are not followed",
931/// );
932/// ```
933#[cfg_attr(not(feature = "reqwest"), doc = "[`ReqwestTransport::new`]: crate")]
934#[must_use]
935pub fn redirect_error(status: u16, headers: &Headers) -> allama::Error {
936 let target = match headers.get("location").map(config::url_host) {
937 Some(Some(host)) => format!("to `{host}`"),
938 Some(None) => "within the same host".to_owned(),
939 None => "without a `Location`".to_owned(),
940 };
941 allama::Error::unexpected_response(format!(
942 "HTTP {status}: the server redirected the request {target}; redirects are not followed"
943 ))
944 .with_context(error_context(status, headers))
945 .authored()
946}
947
948/// Whether a body stream is known to have ended, so a single chunk can be
949/// returned without a copy.
950fn body_is_done(body: &Body) -> bool {
951 body.size_hint() == (0, Some(0))
952}
953
954/// Read at most `limit` bytes of an error body within `timeout`: what was
955/// read, and why reading stopped early when it failed.
956async fn read_error_body(
957 mut body: Body,
958 limit: usize,
959 timeout: Duration,
960) -> (Vec<u8>, Option<TransportError>) {
961 let mut buffer = Vec::new();
962 let failure = {
963 let read = std::pin::pin!(async {
964 while let Some(chunk) = body.next().await {
965 let chunk = match chunk {
966 Ok(chunk) => chunk,
967 Err(error) => return Some(error),
968 };
969 let room = limit - buffer.len();
970 buffer.extend_from_slice(&chunk[..chunk.len().min(room)]);
971 if buffer.len() == limit {
972 break;
973 }
974 }
975 None
976 });
977 let deadline = futures_timer::Delay::new(timeout);
978 match futures_util::future::select(read, deadline).await {
979 futures_util::future::Either::Left((failure, _)) => failure,
980 futures_util::future::Either::Right(((), _)) => Some(
981 TransportError::new(
982 TransportErrorKind::Timeout,
983 format!("the error body did not end within {timeout:?}"),
984 )
985 .authored(),
986 ),
987 }
988 };
989 (buffer, failure)
990}
991
992/// Why a request got no (complete) response.
993#[derive(Debug)]
994pub struct TransportError {
995 kind: TransportErrorKind,
996 message: String,
997 authored: bool,
998 source: Option<Box<dyn std::error::Error + Send + Sync>>,
999}
1000
1001/// Kinds of [`TransportError`].
1002#[derive(Debug, Clone, Copy, PartialEq, Eq, Hash)]
1003#[non_exhaustive]
1004pub enum TransportErrorKind {
1005 /// The request or the body read timed out.
1006 Timeout,
1007 /// The request could not be built (invalid URL or header); nothing was
1008 /// sent.
1009 InvalidRequest,
1010 /// No response was received: connection refused, DNS or TLS failure,
1011 /// connection reset.
1012 ///
1013 /// The request may or may not have reached the provider: a connection
1014 /// reset after the request was written cannot be told apart from one
1015 /// that was never sent. Do not assume it was not processed or billed.
1016 Connect,
1017 /// The response started, but reading its body failed.
1018 Body,
1019 /// The response body was larger than the reader's limit
1020 /// ([`HttpResponse::bytes_limited`]). Unlike the other kinds, the
1021 /// response arrived whole enough to tell: it maps to
1022 /// [`allama::Error::UnexpectedResponse`], which is not retryable.
1023 TooLarge,
1024 /// Any other failure.
1025 Other,
1026}
1027
1028impl TransportError {
1029 /// Create an error.
1030 #[must_use]
1031 pub fn new(kind: TransportErrorKind, message: impl Into<String>) -> Self {
1032 Self {
1033 kind,
1034 message: message.into(),
1035 authored: false,
1036 source: None,
1037 }
1038 }
1039
1040 /// Mark the message as written by the transport's own code, with no
1041 /// text from a foreign error, the request, or the response in it, so
1042 /// [`into_error`](Self::into_error) keeps it as the
1043 /// [authored](allama::Error::authored) message.
1044 ///
1045 /// The same contract as [`allama::Error::authored`]: never call it on a
1046 /// message that embeds an HTTP client's error text, which can contain
1047 /// the request URL.
1048 #[must_use]
1049 pub fn authored(mut self) -> Self {
1050 self.authored = true;
1051 self
1052 }
1053
1054 /// Attach the underlying error. [`into_error`](Self::into_error) and
1055 /// [`into_stream_error`](Self::into_stream_error) keep it as the portable
1056 /// error's [`source`](std::error::Error::source), so a caller can
1057 /// downcast it (a TLS failure from a DNS one). Its text is printed
1058 /// wherever the error's chain is, so attach only errors whose `Display`
1059 /// holds no secret, such as a URL with a key in its query.
1060 #[must_use]
1061 pub fn with_source(mut self, source: impl std::error::Error + Send + Sync + 'static) -> Self {
1062 self.source = Some(Box::new(source));
1063 self
1064 }
1065
1066 /// What went wrong.
1067 #[must_use]
1068 pub fn kind(&self) -> TransportErrorKind {
1069 self.kind
1070 }
1071
1072 /// The portable error for a failure before or while receiving a unary
1073 /// response: [`allama::Error::Timeout`], [`allama::Error::InvalidRequest`],
1074 /// or [`allama::Error::Transport`].
1075 ///
1076 /// The constructor message is an [authored](allama::Error::authored)
1077 /// description of the [kind](Self::kind). The transport's own text is
1078 /// the [upstream message](allama::Error::with_upstream_message): an HTTP
1079 /// client's error can contain the request URL, and some APIs put the key
1080 /// in the query string, so it stays out of the redacted
1081 /// [`ErrorLog`](allama::ErrorLog). A message marked
1082 /// [`authored`](Self::authored) is the constructor message instead.
1083 ///
1084 /// ```
1085 /// use allama_http::{TransportError, TransportErrorKind};
1086 ///
1087 /// let err = TransportError::new(
1088 /// TransportErrorKind::Connect,
1089 /// "error sending request for url (https://api.example.com/v1?key=SECRET)",
1090 /// )
1091 /// .into_error();
1092 /// assert_eq!(err.as_log().to_string(), r#"kind=transport message="the request got no response""#);
1093 /// assert!(err.to_string().contains("api.example.com"));
1094 /// ```
1095 #[must_use]
1096 pub fn into_error(mut self) -> allama::Error {
1097 let source = self.source.take();
1098 if self.authored {
1099 let error = match self.kind {
1100 TransportErrorKind::Timeout => allama::Error::timeout(self.message),
1101 TransportErrorKind::InvalidRequest => allama::Error::invalid_request(self.message),
1102 TransportErrorKind::TooLarge => allama::Error::unexpected_response(self.message),
1103 _ => allama::Error::transport(self.message),
1104 };
1105 return with_source(error.authored(), source);
1106 }
1107 let error = match self.kind {
1108 TransportErrorKind::Timeout => allama::Error::timeout("the request timed out"),
1109 TransportErrorKind::InvalidRequest => {
1110 allama::Error::invalid_request("the request could not be built")
1111 }
1112 TransportErrorKind::Connect => allama::Error::transport("the request got no response"),
1113 TransportErrorKind::Body => {
1114 allama::Error::transport("reading the response body failed")
1115 }
1116 TransportErrorKind::TooLarge => {
1117 allama::Error::unexpected_response("the response body is larger than the limit")
1118 }
1119 TransportErrorKind::Other => allama::Error::transport("the transport failed"),
1120 };
1121 with_source(error.authored().with_upstream_message(self.message), source)
1122 }
1123
1124 /// The portable error for a failure while reading a streamed response,
1125 /// after it started: [`allama::Error::Timeout`] for a timeout (a stream
1126 /// timeout stays a timeout, so retry policies treat it as one), and a
1127 /// [`StreamErrorKind::Transport`](allama::StreamErrorKind::Transport)
1128 /// stream error, which is retryable, for anything else.
1129 ///
1130 /// As in [`into_error`](Self::into_error), the constructor message is
1131 /// [authored](allama::Error::authored) and the transport's own text is
1132 /// the [upstream message](allama::Error::with_upstream_message), unless
1133 /// the message is marked [`authored`](Self::authored).
1134 /// [`SseError::into_error`](sse::SseError::into_error) maps a failed body
1135 /// read with it.
1136 ///
1137 /// ```
1138 /// use allama::{Error, StreamErrorKind};
1139 /// use allama_http::{TransportError, TransportErrorKind};
1140 ///
1141 /// let err = TransportError::new(TransportErrorKind::Body, "connection reset by peer")
1142 /// .into_stream_error();
1143 /// assert!(matches!(err, Error::Stream { kind: StreamErrorKind::Transport, .. }));
1144 /// assert!(err.is_retryable());
1145 /// assert_eq!(err.upstream_message(), Some("connection reset by peer"));
1146 ///
1147 /// let err = TransportError::new(TransportErrorKind::Timeout, "idle for 30s").into_stream_error();
1148 /// assert!(matches!(err, Error::Timeout { .. }));
1149 /// ```
1150 #[must_use]
1151 pub fn into_stream_error(mut self) -> allama::Error {
1152 let source = self.source.take();
1153 if self.authored {
1154 let error = match self.kind {
1155 TransportErrorKind::Timeout => allama::Error::timeout(self.message),
1156 TransportErrorKind::TooLarge => allama::Error::stream_protocol(self.message),
1157 _ => allama::Error::stream(allama::StreamErrorKind::Transport, self.message),
1158 };
1159 return with_source(error.authored(), source);
1160 }
1161 let error = match self.kind {
1162 TransportErrorKind::Timeout => allama::Error::timeout("the stream timed out"),
1163 TransportErrorKind::TooLarge => {
1164 allama::Error::stream_protocol("the stream is larger than the limit")
1165 }
1166 _ => allama::Error::stream(
1167 allama::StreamErrorKind::Transport,
1168 "reading the stream failed",
1169 ),
1170 };
1171 with_source(error.authored().with_upstream_message(self.message), source)
1172 }
1173}
1174
1175/// Keep a [`TransportError`]'s attached cause as the portable error's
1176/// [`source`](std::error::Error::source).
1177fn with_source(
1178 error: allama::Error,
1179 source: Option<Box<dyn std::error::Error + Send + Sync>>,
1180) -> allama::Error {
1181 match source {
1182 Some(source) => error.with_source(source),
1183 None => error,
1184 }
1185}
1186
1187impl fmt::Display for TransportError {
1188 fn fmt(&self, f: &mut fmt::Formatter<'_>) -> fmt::Result {
1189 match self.kind {
1190 TransportErrorKind::Timeout => write!(f, "request timed out: {}", self.message),
1191 _ => f.write_str(&self.message),
1192 }
1193 }
1194}
1195
1196impl std::error::Error for TransportError {
1197 fn source(&self) -> Option<&(dyn std::error::Error + 'static)> {
1198 self.source
1199 .as_deref()
1200 .map(|e| e as &(dyn std::error::Error + 'static))
1201 }
1202}
1203
1204impl From<TransportError> for allama::Error {
1205 fn from(err: TransportError) -> Self {
1206 err.into_error()
1207 }
1208}
1209
1210#[cfg(all(test, not(target_arch = "wasm32")))]
1211mod tests {
1212 use super::*;
1213
1214 #[test]
1215 fn headers_are_case_insensitive() {
1216 let headers: Headers = [("Request-Id", "req_1"), ("retry-after", "2")]
1217 .into_iter()
1218 .collect();
1219 assert_eq!(headers.get("request-id"), Some("req_1"));
1220 assert_eq!(headers.get("RETRY-AFTER"), Some("2"));
1221 assert_eq!(headers.get("missing"), None);
1222 }
1223
1224 #[test]
1225 fn debug_output_redacts_credentials() {
1226 let request = HttpRequest::post("https://x", "{}")
1227 .header("Authorization", "Bearer sk-live-SECRET1")
1228 .header("x-api-key", "SECRET2")
1229 .header("x-goog-api-key", "SECRET3")
1230 .header("Proxy-Authorization", "Basic SECRET4")
1231 .header("cookie", "session=SECRET5")
1232 .header("x-gateway-token", "SECRET6")
1233 .header("X-Client-Secret", "SECRET7")
1234 .header("api-key", "SECRET8")
1235 .header("x-request-id", "req_1");
1236 let debug = format!("{request:?}");
1237 assert!(!debug.contains("SECRET"), "{debug}");
1238 assert!(debug.contains(r#""x-request-id": "req_1""#), "{debug}");
1239 assert!(debug.contains(r#""Authorization": <redacted>"#), "{debug}");
1240
1241 let response = HttpResponse::from_bytes(
1242 200,
1243 [
1244 ("set-cookie", "session=SECRET9"),
1245 ("content-type", "text/plain"),
1246 ]
1247 .into_iter()
1248 .collect(),
1249 "",
1250 );
1251 let debug = format!("{response:?}");
1252 assert!(!debug.contains("SECRET"), "{debug}");
1253 assert!(debug.contains("text/plain"), "{debug}");
1254
1255 let signed =
1256 HttpRequest::post("https://x", "{}").sensitive_header("x-signature", "SECRET10");
1257 assert_eq!(signed.headers.get("x-signature"), Some("SECRET10"));
1258 assert!(!format!("{signed:?}").contains("SECRET"));
1259 }
1260
1261 #[test]
1262 fn sensitive_headers_are_matched_by_name_and_fragment() {
1263 for name in [
1264 "AUTHORIZATION",
1265 "cookie",
1266 "Set-Cookie",
1267 "x-auth",
1268 "openai-api-key",
1269 ] {
1270 assert!(is_sensitive_header(name), "{name}");
1271 }
1272 for name in ["content-type", "x-request-id", "retry-after", "x-tenant-id"] {
1273 assert!(!is_sensitive_header(name), "{name}");
1274 }
1275 }
1276
1277 #[test]
1278 fn retry_after_reads_the_named_header_or_milliseconds() {
1279 let headers: Headers = [("x-retry", "3")].into_iter().collect();
1280 assert_eq!(headers.retry_after("x-retry"), Some(Duration::from_secs(3)));
1281 assert_eq!(headers.retry_after("retry-after"), None);
1282
1283 let headers: Headers = [("retry-after-ms", "1500")].into_iter().collect();
1284 assert_eq!(
1285 headers.retry_after("retry-after"),
1286 Some(Duration::from_millis(1500))
1287 );
1288
1289 let past: Headers = [("retry-after", "Sun, 06 Nov 1994 08:49:37 GMT")]
1290 .into_iter()
1291 .collect();
1292 assert_eq!(past.retry_after("retry-after"), Some(Duration::ZERO));
1293 }
1294
1295 #[test]
1296 fn requests_build_fluently() {
1297 let request = HttpRequest::post("https://x", "{}")
1298 .header("authorization", "Bearer k")
1299 .timeout(Some(Duration::from_secs(5)));
1300 assert_eq!(request.method.as_str(), "POST");
1301 assert_eq!(request.headers.get("Authorization"), Some("Bearer k"));
1302 assert_eq!(request.body_text(), Some("{}"));
1303 assert!(HttpRequest::get("https://x").body.is_none());
1304 }
1305
1306 fn bearer_slot() -> CredentialSlot {
1307 CredentialSlot::new("openai", CredentialPlacement::Bearer)
1308 }
1309
1310 fn options(transport: HttpRequestOptions) -> allama::RequestOptions {
1311 let mut options = allama::RequestOptions::new();
1312 options.insert(transport);
1313 options
1314 }
1315
1316 fn values<'a>(request: &'a HttpRequest, name: &str) -> Vec<&'a str> {
1317 request
1318 .headers
1319 .iter()
1320 .filter(|(key, _)| key.eq_ignore_ascii_case(name))
1321 .map(|(_, value)| value)
1322 .collect()
1323 }
1324
1325 #[test]
1326 fn request_options_replace_same_named_headers_and_add_others() {
1327 let base = || {
1328 HttpRequest::post("https://x", "{}")
1329 .header("Authorization", "Bearer provider")
1330 .header("x-tag", "one")
1331 .header("x-tag", "two")
1332 .header("content-type", "application/json")
1333 };
1334
1335 let untouched = base()
1336 .apply_options(&allama::RequestOptions::new(), &bearer_slot())
1337 .unwrap();
1338 assert_eq!(untouched, base());
1339
1340 let applied = base()
1341 .apply_options(
1342 &options(
1343 HttpRequestOptions::new()
1344 .header("X-TAG", "three")
1345 .header("x-tenant-id", "acme")
1346 .header("Idempotency-Key", "idem-1"),
1347 ),
1348 &bearer_slot(),
1349 )
1350 .unwrap();
1351 let headers: Vec<_> = applied.headers.iter().collect();
1352 assert_eq!(
1353 headers,
1354 [
1355 ("Authorization", "Bearer provider"),
1356 ("content-type", "application/json"),
1357 ("X-TAG", "three"),
1358 ("x-tenant-id", "acme"),
1359 ("Idempotency-Key", "idem-1"),
1360 ]
1361 );
1362
1363 let mut invalid = allama::RequestOptions::new();
1364 invalid.insert_json(
1365 OPTIONS_NAMESPACE,
1366 serde_json::json!({"headers": {"x-api-key": "k"}}),
1367 );
1368 assert!(matches!(
1369 base().apply_options(&invalid, &bearer_slot()).unwrap_err(),
1370 allama::Error::InvalidRequest { .. }
1371 ));
1372 }
1373
1374 #[test]
1375 fn the_scope_credential_replaces_the_configured_one_in_its_slot() {
1376 let transport = HttpRequestOptions::new()
1377 .credential("openai", Credential::api_key("TENANT-SECRET"))
1378 .credential("anthropic", Credential::api_key("FOREIGN-SECRET"));
1379
1380 let bearer = HttpRequest::post("https://x", "{}")
1381 .header("authorization", "Bearer provider")
1382 .apply_options(&options(transport.clone()), &bearer_slot())
1383 .unwrap();
1384 assert_eq!(values(&bearer, "authorization"), ["Bearer TENANT-SECRET"]);
1385
1386 let azure = CredentialSlot::new("openai", CredentialPlacement::Header("api-key".into()))
1387 .replaces("authorization");
1388 let header = HttpRequest::post("https://x", "{}")
1389 .header("authorization", "Bearer entra-token")
1390 .header("API-KEY", "configured")
1391 .apply_options(&options(transport.clone()), &azure)
1392 .unwrap();
1393 assert_eq!(values(&header, "api-key"), ["TENANT-SECRET"]);
1394 assert!(values(&header, "authorization").is_empty());
1395
1396 for request in [&bearer, &header] {
1397 let debug = format!("{request:?}");
1398 assert!(!debug.contains("SECRET"), "{debug}");
1399 assert!(
1400 !request.headers.iter().any(|(_, v)| v.contains("FOREIGN")),
1401 "{request:?}"
1402 );
1403 }
1404
1405 // A custom placement is the adapter's to sign; the configured
1406 // headers stay until it does.
1407 let custom = CredentialSlot::new("openai", CredentialPlacement::Custom);
1408 let signed = HttpRequest::post("https://x", "{}")
1409 .header("authorization", "configured")
1410 .apply_options(&options(transport.clone()), &custom)
1411 .unwrap();
1412 assert_eq!(values(&signed, "authorization"), ["configured"]);
1413 assert_eq!(
1414 transport
1415 .credential_for("openai")
1416 .map(Credential::expose_secret),
1417 Some("TENANT-SECRET")
1418 );
1419 }
1420
1421 #[test]
1422 fn header_credentials_reach_only_their_scope_and_replace_same_named_headers() {
1423 let transport = HttpRequestOptions::new()
1424 .credential(
1425 "openai",
1426 Credential::header("cf-aig-authorization", "Bearer GATEWAY-SECRET"),
1427 )
1428 .credential(
1429 "anthropic",
1430 Credential::header("x-other-gw", "OTHER-SECRET"),
1431 );
1432 let applied = HttpRequest::post("https://x", "{}")
1433 .header("authorization", "Bearer provider")
1434 .header("cf-aig-authorization", "Bearer configured-gateway")
1435 .apply_options(&options(transport), &bearer_slot())
1436 .unwrap();
1437 assert_eq!(
1438 values(&applied, "cf-aig-authorization"),
1439 ["Bearer GATEWAY-SECRET"]
1440 );
1441 assert_eq!(values(&applied, "authorization"), ["Bearer provider"]);
1442 assert!(values(&applied, "x-other-gw").is_empty());
1443 let debug = format!("{applied:?}");
1444 assert!(!debug.contains("SECRET"), "{debug}");
1445
1446 // Redacted whatever the name, once placed.
1447 let custom_name = HttpRequest::post("https://x", "{}")
1448 .apply_options(
1449 &options(
1450 HttpRequestOptions::new()
1451 .credential("openai", Credential::header("x-route", "ROUTE-SECRET")),
1452 ),
1453 &bearer_slot(),
1454 )
1455 .unwrap();
1456 assert_eq!(values(&custom_name, "x-route"), ["ROUTE-SECRET"]);
1457 assert!(!format!("{custom_name:?}").contains("ROUTE-SECRET"));
1458 }
1459
1460 #[test]
1461 fn plain_credential_headers_are_refused_as_configuration_errors() {
1462 let nimbus = CredentialSlot::new(
1463 "nimbus",
1464 CredentialPlacement::Header("x-nimbus-session".into()),
1465 )
1466 .replaces("x-nimbus-legacy");
1467 for (name, slot) in [
1468 ("Authorization", &nimbus),
1469 ("x-api-key", &nimbus),
1470 ("cf-aig-authorization", &nimbus),
1471 ("ocp-apim-subscription-key", &nimbus),
1472 ("x-amz-security-token", &nimbus),
1473 ("cookie", &nimbus),
1474 ("X-Nimbus-Session", &nimbus),
1475 ("x-nimbus-legacy", &nimbus),
1476 ] {
1477 let err = HttpRequest::post("https://x", "{}")
1478 .apply_options(
1479 &options(HttpRequestOptions::new().header(name, "PLAIN-SECRET")),
1480 slot,
1481 )
1482 .unwrap_err();
1483 assert!(
1484 matches!(err, allama::Error::Configuration { .. }),
1485 "{err:?}"
1486 );
1487 assert!(!err.is_retryable(), "{name}");
1488 let text = format!("{err} {err:?} {}", err.as_log());
1489 assert!(!text.contains("PLAIN-SECRET"), "{text}");
1490 assert!(text.contains(name), "{text}");
1491 }
1492 // Another adapter's own header is not this one's to refuse.
1493 assert!(
1494 HttpRequest::post("https://x", "{}")
1495 .apply_options(
1496 &options(HttpRequestOptions::new().header("x-nimbus-session", "v")),
1497 &bearer_slot(),
1498 )
1499 .is_ok()
1500 );
1501 }
1502
1503 #[test]
1504 fn a_required_slot_refuses_a_request_without_its_credential() {
1505 let slot = bearer_slot().required(true);
1506 for transport in [
1507 None,
1508 Some(HttpRequestOptions::new().credential("other", Credential::api_key("k"))),
1509 Some(
1510 HttpRequestOptions::new()
1511 .credential("openai", Credential::header("cf-aig-authorization", "g")),
1512 ),
1513 ] {
1514 let options = transport.map(options).unwrap_or_default();
1515 let err = HttpRequest::post("https://x", "{}")
1516 .apply_options(&options, &slot)
1517 .unwrap_err();
1518 assert!(
1519 matches!(err, allama::Error::Configuration { .. }),
1520 "{err:?}"
1521 );
1522 assert!(err.to_string().contains("`openai`"), "{err}");
1523 }
1524 let sent = HttpRequest::post("https://x", "{}")
1525 .apply_options(
1526 &options(HttpRequestOptions::new().credential("openai", Credential::api_key("k"))),
1527 &slot,
1528 )
1529 .unwrap();
1530 assert_eq!(sent.headers.get("authorization"), Some("Bearer k"));
1531 }
1532
1533 #[test]
1534 fn a_later_credential_replaces_the_same_kind_in_its_scope() {
1535 let transport = HttpRequestOptions::new()
1536 .credential("openai", Credential::api_key("first"))
1537 .credential("openai", Credential::header("x-gw", "gw-1"))
1538 .credential("OPENAI", Credential::api_key("other-scope"))
1539 .credential("openai", Credential::api_key("second"))
1540 .credential("openai", Credential::header("X-GW", "gw-2"));
1541 assert_eq!(
1542 transport
1543 .credential_for("openai")
1544 .map(Credential::expose_secret),
1545 Some("second")
1546 );
1547 let scopes: Vec<_> = transport
1548 .credentials()
1549 .map(|(scope, credential)| (scope, credential.expose_secret()))
1550 .collect();
1551 assert_eq!(
1552 scopes,
1553 [
1554 ("OPENAI", "other-scope"),
1555 ("openai", "second"),
1556 ("openai", "gw-2")
1557 ]
1558 );
1559 }
1560
1561 #[tokio::test]
1562 async fn bodies_are_read_whole_or_streamed() {
1563 let response = HttpResponse::new(
1564 200,
1565 Headers::new(),
1566 Box::pin(futures_util::stream::iter([
1567 Ok(Bytes::from_static(b"he")),
1568 Ok(Bytes::from_static(b"llo")),
1569 ])),
1570 );
1571 assert!(response.is_success());
1572 assert_eq!(response.text().await.unwrap(), "hello");
1573
1574 let failing = HttpResponse::new(
1575 200,
1576 Headers::new(),
1577 Box::pin(futures_util::stream::iter([
1578 Ok(Bytes::from_static(b"he")),
1579 Err(TransportError::new(TransportErrorKind::Body, "reset")),
1580 ])),
1581 );
1582 assert_eq!(
1583 failing.bytes().await.unwrap_err().kind(),
1584 TransportErrorKind::Body
1585 );
1586 }
1587
1588 fn chunks(parts: Vec<Result<Bytes, TransportError>>) -> Body {
1589 Box::pin(futures_util::stream::iter(parts))
1590 }
1591
1592 #[tokio::test]
1593 async fn limited_reads_stop_before_passing_the_limit() {
1594 let body = || {
1595 chunks(vec![
1596 Ok(Bytes::from_static(b"0123")),
1597 Ok(Bytes::from_static(b"4567")),
1598 Ok(Bytes::from_static(b"89")),
1599 ])
1600 };
1601 let read = |limit| HttpResponse::new(200, Headers::new(), body()).bytes_limited(limit);
1602 assert_eq!(read(10).await.unwrap(), "0123456789");
1603 assert_eq!(read(usize::MAX).await.unwrap(), "0123456789");
1604 for limit in [0, 3, 4, 9] {
1605 let err = read(limit).await.unwrap_err();
1606 assert_eq!(err.kind(), TransportErrorKind::TooLarge, "{limit}");
1607 let err = err.into_error();
1608 assert!(
1609 matches!(err, allama::Error::UnexpectedResponse { .. }),
1610 "{err:?}"
1611 );
1612 assert!(!err.is_retryable());
1613 assert!(
1614 err.as_log()
1615 .to_string()
1616 .contains(&format!("limit of {limit} bytes")),
1617 "{}",
1618 err.as_log()
1619 );
1620 }
1621 let text = HttpResponse::from_bytes(200, Headers::new(), "héllo")
1622 .text_limited(6)
1623 .await
1624 .unwrap();
1625 assert_eq!(text, "héllo");
1626 assert!(matches!(
1627 TransportError::new(TransportErrorKind::TooLarge, "x").into_stream_error(),
1628 allama::Error::Stream {
1629 kind: allama::StreamErrorKind::Protocol,
1630 ..
1631 }
1632 ));
1633 }
1634
1635 fn map_http(status: u16, _: &Headers, body: &str) -> allama::Error {
1636 status_error(status, format!("HTTP {status}"))
1637 .authored()
1638 .with_body(body.to_owned())
1639 }
1640
1641 /// An error body that never ends is read up to the cap and no
1642 /// further, so the call ends and memory stays bounded.
1643 #[tokio::test]
1644 async fn error_bodies_are_read_up_to_the_cap() {
1645 let endless: Body = Box::pin(futures_util::stream::repeat_with(|| {
1646 Ok(Bytes::from(vec![b'x'; 16 * 1024]))
1647 }));
1648 let err = HttpResponse::new(503, Headers::new(), endless)
1649 .map_error(map_http)
1650 .await;
1651 assert!(matches!(err, allama::Error::Overloaded { .. }), "{err:?}");
1652 assert_eq!(err.upstream_message(), None);
1653 let (body, failure) = read_error_body(
1654 Box::pin(futures_util::stream::repeat_with(|| {
1655 Ok(Bytes::from_static(b"abc"))
1656 })),
1657 MAX_ERROR_BODY_SIZE,
1658 ERROR_BODY_TIMEOUT,
1659 )
1660 .await;
1661 assert_eq!(body.len(), MAX_ERROR_BODY_SIZE);
1662 assert!(failure.is_none());
1663 }
1664
1665 /// A body that stalls is given up on after the time bound, keeping
1666 /// what arrived, and a failed read is recorded on the error rather than
1667 /// turned into an empty body.
1668 #[tokio::test]
1669 async fn error_body_failures_are_recorded() {
1670 let stalled: Body = Box::pin(
1671 futures_util::stream::iter([Ok(Bytes::from_static(b"{\"err"))])
1672 .chain(futures_util::stream::pending()),
1673 );
1674 let (body, failure) =
1675 read_error_body(stalled, MAX_ERROR_BODY_SIZE, Duration::from_millis(20)).await;
1676 assert_eq!(body, b"{\"err");
1677 assert_eq!(failure.unwrap().kind(), TransportErrorKind::Timeout);
1678
1679 let reset = HttpResponse::new(
1680 500,
1681 Headers::new(),
1682 chunks(vec![
1683 Ok(Bytes::from_static(b"partial")),
1684 Err(TransportError::new(
1685 TransportErrorKind::Body,
1686 "connection reset",
1687 )),
1688 ]),
1689 );
1690 let err = reset.map_error(map_http).await;
1691 assert_eq!(
1692 err.upstream_message(),
1693 Some("reading the error body failed: connection reset")
1694 );
1695 assert_eq!(err.context().and_then(|c| c.capped_body()), Some("partial"));
1696
1697 let enveloped = HttpResponse::new(
1698 500,
1699 Headers::new(),
1700 chunks(vec![Err(TransportError::new(
1701 TransportErrorKind::Body,
1702 "reset",
1703 ))]),
1704 )
1705 .map_error(|status, headers, body| {
1706 map_http(status, headers, body).with_upstream_message("upstream")
1707 })
1708 .await;
1709 assert_eq!(
1710 enveloped.upstream_message(),
1711 Some("upstream (reading the error body failed: reset)")
1712 );
1713 }
1714
1715 #[tokio::test]
1716 async fn redirects_are_unexpected_responses_naming_the_host() {
1717 let mut called = false;
1718 let headers: Headers = [
1719 (
1720 "location",
1721 "https://u:pw@Other.Example:8443/v1/x?token=SECRET",
1722 ),
1723 ("x-request-id", "req_1"),
1724 ]
1725 .into_iter()
1726 .collect();
1727 let err = HttpResponse::from_bytes(308, headers, "")
1728 .map_error(|status, headers, body| {
1729 called = true;
1730 map_http(status, headers, body)
1731 })
1732 .await;
1733 assert!(!called);
1734 assert!(
1735 matches!(err, allama::Error::UnexpectedResponse { .. }),
1736 "{err:?}"
1737 );
1738 assert_eq!(err.status(), Some(308));
1739 assert_eq!(err.request_id(), Some("req_1"));
1740 let log = err.as_log().to_string();
1741 assert!(log.contains("`other.example`"), "{log}");
1742 for text in [log, err.to_string(), format!("{err:?}")] {
1743 assert!(!text.contains("SECRET") && !text.contains("pw"), "{text}");
1744 }
1745 let relative = redirect_error(301, &[("location", "/v2/chat")].into_iter().collect());
1746 assert!(
1747 relative.to_string().contains("within the same host"),
1748 "{relative}"
1749 );
1750 let bare = redirect_error(302, &Headers::new());
1751 assert!(bare.to_string().contains("without a `Location`"), "{bare}");
1752 }
1753
1754 #[test]
1755 fn with_headers_replaces_same_named_headers() {
1756 let defaults: Headers = [("User-Agent", "a/1"), ("x-route", "eu")]
1757 .into_iter()
1758 .collect();
1759 let request = HttpRequest::post("https://x", "{}")
1760 .header("content-type", "application/json")
1761 .header("user-agent", "old")
1762 .sensitive_header("x-signed", "SECRET")
1763 .with_headers(&defaults);
1764 assert_eq!(
1765 request.headers.iter().collect::<Vec<_>>(),
1766 [
1767 ("content-type", "application/json"),
1768 ("x-signed", "SECRET"),
1769 ("User-Agent", "a/1"),
1770 ("x-route", "eu"),
1771 ]
1772 );
1773 assert!(!format!("{request:?}").contains("SECRET"));
1774 }
1775
1776 #[test]
1777 fn errors_map_to_portable_errors() {
1778 let timeout = TransportError::new(TransportErrorKind::Timeout, "after 5s");
1779 assert_eq!(timeout.to_string(), "request timed out: after 5s");
1780 assert!(matches!(
1781 allama::Error::from(timeout),
1782 allama::Error::Timeout { .. }
1783 ));
1784 assert!(matches!(
1785 TransportError::new(TransportErrorKind::InvalidRequest, "bad url").into_error(),
1786 allama::Error::InvalidRequest { .. }
1787 ));
1788 assert!(matches!(
1789 TransportError::new(TransportErrorKind::Connect, "refused").into_error(),
1790 allama::Error::Transport { .. }
1791 ));
1792 }
1793
1794 #[test]
1795 fn transport_text_stays_out_of_the_log() {
1796 for kind in [
1797 TransportErrorKind::Timeout,
1798 TransportErrorKind::InvalidRequest,
1799 TransportErrorKind::Connect,
1800 TransportErrorKind::Body,
1801 TransportErrorKind::Other,
1802 ] {
1803 let err = TransportError::new(kind, "https://x.example/v1?key=SECRET").into_error();
1804 let line = err.as_log().to_string();
1805 assert!(!line.contains("SECRET"), "{line}");
1806 assert!(line.contains("message="), "{line}");
1807 assert_eq!(
1808 err.upstream_message(),
1809 Some("https://x.example/v1?key=SECRET")
1810 );
1811 assert!(err.to_string().contains("SECRET"));
1812 }
1813 }
1814}
1815
1816/// Run on `wasm32` with `wasm-bindgen-test`
1817/// (`cargo test --target wasm32-unknown-unknown -p allama-http`).
1818#[cfg(all(test, target_arch = "wasm32"))]
1819mod wasm_tests {
1820 use wasm_bindgen_test::wasm_bindgen_test;
1821
1822 use super::*;
1823
1824 #[wasm_bindgen_test]
1825 fn retry_after_reads_the_clock() {
1826 let past: Headers = [("retry-after", "Sun, 06 Nov 1994 08:49:37 GMT")]
1827 .into_iter()
1828 .collect();
1829 assert_eq!(past.retry_after("retry-after"), Some(Duration::ZERO));
1830
1831 let future: Headers = [("retry-after", "Fri, 31 Dec 9999 23:59:59 GMT")]
1832 .into_iter()
1833 .collect();
1834 assert!(future.retry_after("retry-after").unwrap() > Duration::ZERO);
1835 }
1836}