Expand description
Runtime-neutral replay boundaries for streamed LLM responses.
This crate observes items as they pass through a GuardedStream. It does
not perform I/O, parse a transport, retain response bodies, or retry a
request. A retry is authorized only when the caller declares a
ReplayContract::KnownSafe, no output boundary was crossed, and the
classifier observed an explicit transient failure.
§Example
The guard observes an item before returning that same item. A transient failure after text has already appeared therefore cannot authorize replay:
use futures_util::{FutureExt, StreamExt};
use llm_stream_guard::{
FailureKind, GuardedStream, Observation, OutputKind, ReplayBoundary,
ReplayContract, ReplayVerdict,
};
#[derive(Clone, Copy, Debug, Eq, PartialEq)]
enum AppEvent {
Text,
TransientFailure,
}
let upstream = futures_util::stream::iter([
AppEvent::Text,
AppEvent::TransientFailure,
]);
let guarded = GuardedStream::new(
upstream,
|event: &AppEvent| match event {
AppEvent::Text => Observation::Output(OutputKind::Text),
AppEvent::TransientFailure => {
Observation::Failed(FailureKind::Transient)
}
},
ReplayContract::KnownSafe,
);
let probe = guarded.probe();
futures_util::pin_mut!(guarded);
assert_eq!(guarded.next().now_or_never(), Some(Some(AppEvent::Text)));
assert_eq!(
probe.snapshot().boundary(),
ReplayBoundary::Crossed(OutputKind::Text),
);
assert_eq!(
guarded.next().now_or_never(),
Some(Some(AppEvent::TransientFailure)),
);
assert_eq!(probe.snapshot().verdict(), ReplayVerdict::Deny);Modules§
- openai
- Conservative classification of already-framed OpenAI-compatible SSE events.
Structs§
- Guarded
Stream - A transparent stream wrapper that records replay-relevant observations.
- Probe
- A cloneable handle for reading consistent stream snapshots.
- Snapshot
- A consistent, point-in-time view of replay evidence.
Enums§
- Exit
Reason - Why observation of the stream ended without a semantic terminal item.
- Failure
Kind - How confidently a classified failure may be treated as transient.
- Observation
- A replay-relevant observation made from one stream item.
- Output
Kind - A semantic kind of model output that makes whole-request replay unsafe.
- Replay
Boundary - Whether semantic output has crossed the whole-request replay boundary.
- Replay
Contract - The caller’s independent guarantee about replaying the whole request.
- Replay
Verdict - The conservative whole-request replay decision.
- Termination
- The terminal state observed for a stream.
Traits§
- Classify
- Classifies one item emitted by an upstream stream.