Struct LlmProcessor 

pub struct LlmProcessor<L: LlmService> { /* private fields */ }

Wraps an LlmService: on a Frame::LlmContext (built by the user context aggregator) it runs the LLM and forwards the streamed response frames downstream. As a fallback it also runs on a bare final Transcription (wrapping it in a single-message context) so the adapter works without an aggregator in front (the direct STT → LLM → TTS fixture path). In the cascaded builder the user aggregator consumes the final transcription and emits LlmContext instead, so the LLM is triggered exactly once per turn (no double-run).

The streamed frames are pushed as they arrive (true token streaming); the whole stream is consumed before process_frame returns so a following terminal frame can’t drain the downstream TTS before the response lands. Tools pushed via Frame::UpdateSettings (target Llm) update the service’s tool set.

Implementations

impl<L: LlmService> LlmProcessor<L>

pub fn new(svc: L) -> Self

Wrap svc as a pipeline processor.

Trait Implementations

impl<L: LlmService + 'static> FrameProcessor for LlmProcessor<L>

fn name(&self) -> &str

Stable, human-readable name (observer events, error attribution, tracing).

fn start<'life0, 'life1, 'life2, 'async_trait>( &'life0 mut self, _s: &'life1 ProcessorSetup, p: &'life2 StartParams, ) -> Pin<Box<dyn Future<Output = Result<()>> + Send + 'async_trait>>

where Self: 'async_trait, 'life0: 'async_trait, 'life1: 'async_trait, 'life2: 'async_trait,

Called once when the Frame::Start frame reaches this processor, before any data frame. Open sockets / spawn provider readers here. Default: no-op.

fn can_generate_metrics(&self) -> bool

Whether this processor produces metrics (pipecat can_generate_metrics, :395). Services override to true.

fn process_frame<'life0, 'life1, 'async_trait>( &'life0 mut self, env: Envelope, link: &'life1 Link, ) -> Pin<Box<dyn Future<Output = Result<()>> + Send + 'async_trait>>

where Self: 'async_trait, 'life0: 'async_trait, 'life1: 'async_trait,

Handle one frame. Push results via link. Must not block: long work (a provider round-trip) is driven by an internally-spawned task that feeds results back as frames. The default impl forwards the frame unchanged in its direction — so a pure observer/no-op processor is process_frame = default.

fn stop<'life0, 'async_trait>( &'life0 mut self, _reason: StopReason, ) -> Pin<Box<dyn Future<Output = Result<()>> + Send + 'async_trait>>

where Self: 'async_trait, 'life0: 'async_trait,

Called on End/Stop/Cancel after the terminal frame is forwarded. Flush + close. Default: no-op.