Struct Pipeline 

pub struct Pipeline { /* private fields */ }

Top-level non-blocking pipeline. Owns all three queue handles and the gauge tracking total in-flight work.

Constructed via Self::start, which spawns one worker task per queue. Drop the Pipeline to signal shutdown — workers exit as soon as their queues drain.

Implementations

impl Pipeline

pub fn start( config: PipelineConfig, system: Arc<ConversationMemorySystem>, ) -> Pipeline

Spawn worker tasks and return a handle. Uses the current Tokio runtime via tokio::spawn — must be called from inside an async context.

system is shared with every worker so they can call into the canonical *_now helpers on ConversationMemorySystem without reimplementing the load → mutate → CAS → persist flow.

pub fn backlog(&self) -> usize

Returns a snapshot of the total number of work items currently queued across all pipeline queues. Used by the health endpoint to surface backlog pressure.

pub fn submit_embedding( &self, item: EmbeddingWorkItem, ) -> Result<(), PipelineError>

Submit an embedding-compute request. Non-blocking — returns PipelineError::Backpressure immediately if the queue is full.

pub fn submit_graph(&self, item: GraphWorkItem) -> Result<(), PipelineError>

Submit an entity-graph upsert.

pub fn submit_summary(&self, item: SummaryWorkItem) -> Result<(), PipelineError>

Submit a summary-refresh request.

pub fn embedding_queue(&self) -> &EmbeddingQueue

Borrow individual queue handles (test / advanced use).

pub fn graph_queue(&self) -> &GraphQueue

Borrow the graph queue handle (test / advanced use).

pub fn summary_queue(&self) -> &SummaryQueue

Borrow the summary queue handle (test / advanced use).