Skip to main content

Module sse

Module sse 

Source
Expand description

Server-sent events (SSE) for streaming adapters.

Every streaming HTTP API in the bundled providers speaks SSE, so this is the framing most adapters need, first-party or not:

  • Framing. SseStream turns a response Body (or any stream of byte chunks) into SseEvents; SseDecoder is the same decoder without the stream, for callers that feed bytes themselves. Both follow the WHATWG event stream format: LF, CR and CRLF line endings (also when a chunk boundary splits a CRLF), multi-line data: fields joined with \n, a leading byte order mark, comments (: keepalive) and retry: lines skipped, and an event without a trailing blank line discarded at the end of the body. Each byte is scanned once, however the chunks split the lines, and a size cap (SseDecoder::DEFAULT_MAX_EVENT_SIZE by default) bounds what a stream can make the decoder hold.
  • Two deliberate failures. Where the standard decodes a stream with invalid UTF-8 lossily, these helpers fail (SseError::InvalidUtf8), and a line or an event past the size cap fails too (SseError::EventTooLarge). Neither is retryable: asking again streams the same bytes. SseError::into_error maps them, and a failed read, to the stream errors the variants name.
  • The driver. SseChatStream runs an adapter’s ChatFrameDecoder<SseEvent> over the events: it is the framing-neutral FramedChatStream over an SseStream (a type alias), so it ends the stream after the first error, runs the decoder’s end-of-body or failure hook once, classifies a failed read (a stream timeout stays a timeout), and never polls the body after the stream ended.
  • Vendor frames. VendorFrames (from framing) surfaces named events an adapter does not model (event: search_results) as response-level provider data under the adapter’s namespace, with one warning per event type, so they are never decoded as the adapter’s own and never dropped.

Other wire framings use the same driver and vendor frames: ndjson frames newline-delimited JSON, and any other stream of frames drives a ChatFrameDecoder.

Decoding the events’ data stays with each adapter: the helpers frame, drive and surface, they do not interpret.

use allama::{ProviderData, StreamEvent};
use allama_http::sse::{SseStream, VendorFrames};
use futures_util::{FutureExt, StreamExt, stream};

// A body whose second chunk starts in the middle of a `data:` line.
let body = stream::iter([
    Ok::<_, std::io::Error>(&b": keepalive\r\nevent: delta\r\ndata: {\"text\":"[..]),
    Ok(&b"\"Hi\"}\r\n\r\nevent: search_results\r\ndata: [1]\r\n\r\n"[..]),
]);
let events: Vec<_> = SseStream::new(body).collect().now_or_never().unwrap();
let mut vendor_frames = VendorFrames::new("demo", "demo");
let mut out = Vec::new();
for event in events {
    let event = event.unwrap();
    match event.event.as_str() {
        "delta" => assert_eq!(event.data, r#"{"text":"Hi"}"#),
        name => out.extend(vendor_frames.surface(name, &event.data)),
    }
}
// A warning, then the data under the namespace and event type.
assert!(matches!(out[0], StreamEvent::Warning { .. }));
assert_eq!(
    out[1],
    StreamEvent::response_provider_data(ProviderData::new().with_raw(
        "demo",
        "search_results",
        serde_json::json!([[1]]),
    )),
);

Re-exports§

pub use crate::framing::VendorFrames;

Structs§

SseDecoder
An incremental SSE decoder: push bytes, take complete events.
SseEvent
One SSE event.
SseStream
The SSE events of a byte stream, such as an HttpResponse body.

Enums§

SseError
Why an SseStream (or an SseDecoder, with E = Infallible) failed.

Type Aliases§

SseChatStream
A chat stream decoded from a response’s SSE events by an adapter’s ChatFrameDecoder<SseEvent>: the FramedChatStream over an SseStream, with the driver guarantees the decoder trait lists.