Struct StreamProcessor 

pub struct StreamProcessor<G> { /* private fields */ }

Parse backend inference chunks and produce protocol response events.

Implementations

impl<G> StreamProcessor<G>

where G: ChunkGenerator,

pub fn new(generator: G, options: ParsingOptions) -> Self

Combine a protocol event generator with output parsing options.

pub fn with_tokenizer(self, decoder: impl TokenizerDecoder + 'static) -> Self

Decode InferenceChunk::Token ids with the supplied decoder.

Ids that contribute text count as completion tokens, including the ids buffered while a multi-token character was incomplete. IDs still buffered at the end of input contribute neither text nor completion usage. Without a decoder, a token chunk fails the stream with StreamError::MissingTokenizer.

pub fn process( self, inference: impl Stream<Item = InferenceChunk> + Send, ) -> impl Stream<Item = Result<G::Chunk, StreamError>> + Send

where G::Chunk: Send,

Consume inference chunks until a finish chunk, matched stop sequence, or EOF.

A ready chunk received before output starts supplies the initial prompt usage and fingerprint. Without it, the start event uses zero prompt usage and no fingerprint. Later ready chunks do not update emitted metadata. Completion usage accumulates each processed chunk’s token count, including the entire chunk containing a stop sequence; later chunks are not read. Missing tokenizers and decoder failures yield an error and end the stream without a normal finish event.