Skip to main content

Crate llm_stream_guard

Crate llm_stream_guard 

Source
Expand description

Runtime-neutral replay boundaries for streamed LLM responses.

This crate observes items as they pass through a GuardedStream. It does not perform I/O, parse a transport, retain response bodies, or retry a request. A retry is authorized only when the caller declares a ReplayContract::KnownSafe, no output boundary was crossed, and the classifier observed an explicit transient failure.

§Example

The guard observes an item before returning that same item. A transient failure after text has already appeared therefore cannot authorize replay:

use futures_util::{FutureExt, StreamExt};
use llm_stream_guard::{
    FailureKind, GuardedStream, Observation, OutputKind, ReplayBoundary,
    ReplayContract, ReplayVerdict,
};

#[derive(Clone, Copy, Debug, Eq, PartialEq)]
enum AppEvent {
    Text,
    TransientFailure,
}

let upstream = futures_util::stream::iter([
    AppEvent::Text,
    AppEvent::TransientFailure,
]);
let guarded = GuardedStream::new(
    upstream,
    |event: &AppEvent| match event {
        AppEvent::Text => Observation::Output(OutputKind::Text),
        AppEvent::TransientFailure => {
            Observation::Failed(FailureKind::Transient)
        }
    },
    ReplayContract::KnownSafe,
);
let probe = guarded.probe();
futures_util::pin_mut!(guarded);

assert_eq!(guarded.next().now_or_never(), Some(Some(AppEvent::Text)));
assert_eq!(
    probe.snapshot().boundary(),
    ReplayBoundary::Crossed(OutputKind::Text),
);
assert_eq!(
    guarded.next().now_or_never(),
    Some(Some(AppEvent::TransientFailure)),
);
assert_eq!(probe.snapshot().verdict(), ReplayVerdict::Deny);

Modules§

openai
Conservative classification of already-framed OpenAI-compatible SSE events.

Structs§

GuardedStream
A transparent stream wrapper that records replay-relevant observations.
Probe
A cloneable handle for reading consistent stream snapshots.
Snapshot
A consistent, point-in-time view of replay evidence.

Enums§

ExitReason
Why observation of the stream ended without a semantic terminal item.
FailureKind
How confidently a classified failure may be treated as transient.
Observation
A replay-relevant observation made from one stream item.
OutputKind
A semantic kind of model output that makes whole-request replay unsafe.
ReplayBoundary
Whether semantic output has crossed the whole-request replay boundary.
ReplayContract
The caller’s independent guarantee about replaying the whole request.
ReplayVerdict
The conservative whole-request replay decision.
Termination
The terminal state observed for a stream.

Traits§

Classify
Classifies one item emitted by an upstream stream.