Expand description
Server-sent events (SSE) for streaming adapters.
Every streaming HTTP API in the bundled providers speaks SSE, so this is the framing most adapters need, first-party or not:
- Framing.
SseStreamturns a responseBody(or any stream of byte chunks) intoSseEvents;SseDecoderis the same decoder without the stream, for callers that feed bytes themselves. Both follow the WHATWG event stream format:LF,CRandCRLFline endings (also when a chunk boundary splits aCRLF), multi-linedata:fields joined with\n, a leading byte order mark, comments (: keepalive) andretry:lines skipped, and an event without a trailing blank line discarded at the end of the body. Each byte is scanned once, however the chunks split the lines, and a size cap (SseDecoder::DEFAULT_MAX_EVENT_SIZEby default) bounds what a stream can make the decoder hold. - Two deliberate failures. Where the standard decodes a stream with
invalid UTF-8 lossily, these helpers fail
(
SseError::InvalidUtf8), and a line or an event past the size cap fails too (SseError::EventTooLarge). Neither is retryable: asking again streams the same bytes.SseError::into_errormaps them, and a failed read, to the stream errors the variants name. - The driver.
SseChatStreamruns an adapter’sChatFrameDecoder<SseEvent>over the events: it is the framing-neutralFramedChatStreamover anSseStream(a type alias), so it ends the stream after the first error, runs the decoder’s end-of-body or failure hook once, classifies a failed read (a stream timeout stays a timeout), and never polls the body after the stream ended. - Vendor frames.
VendorFrames(fromframing) surfaces named events an adapter does not model (event: search_results) as response-level provider data under the adapter’s namespace, with one warning per event type, so they are never decoded as the adapter’s own and never dropped.
Other wire framings use the same driver and vendor frames:
ndjson frames newline-delimited JSON, and any other
stream of frames drives a
ChatFrameDecoder.
Decoding the events’ data stays with each adapter: the helpers frame, drive and surface, they do not interpret.
use allama::{ProviderData, StreamEvent};
use allama_http::sse::{SseStream, VendorFrames};
use futures_util::{FutureExt, StreamExt, stream};
// A body whose second chunk starts in the middle of a `data:` line.
let body = stream::iter([
Ok::<_, std::io::Error>(&b": keepalive\r\nevent: delta\r\ndata: {\"text\":"[..]),
Ok(&b"\"Hi\"}\r\n\r\nevent: search_results\r\ndata: [1]\r\n\r\n"[..]),
]);
let events: Vec<_> = SseStream::new(body).collect().now_or_never().unwrap();
let mut vendor_frames = VendorFrames::new("demo", "demo");
let mut out = Vec::new();
for event in events {
let event = event.unwrap();
match event.event.as_str() {
"delta" => assert_eq!(event.data, r#"{"text":"Hi"}"#),
name => out.extend(vendor_frames.surface(name, &event.data)),
}
}
// A warning, then the data under the namespace and event type.
assert!(matches!(out[0], StreamEvent::Warning { .. }));
assert_eq!(
out[1],
StreamEvent::response_provider_data(ProviderData::new().with_raw(
"demo",
"search_results",
serde_json::json!([[1]]),
)),
);Re-exports§
pub use crate::framing::VendorFrames;
Structs§
- SseDecoder
- An incremental SSE decoder: push bytes, take complete events.
- SseEvent
- One SSE event.
- SseStream
- The SSE events of a byte stream, such as an
HttpResponsebody.
Enums§
- SseError
- Why an
SseStream(or anSseDecoder, withE=Infallible) failed.
Type Aliases§
- SseChat
Stream - A chat stream decoded from a response’s SSE events by an adapter’s
ChatFrameDecoder<SseEvent>: theFramedChatStreamover anSseStream, with the driver guarantees the decoder trait lists.