Trait LlamaEngine 

pub trait LlamaEngine: Send + Sync {
    // Required methods
    fn load_model(&self, spec: &ModelSpec) -> Result<ModelHandle>;
    fn tokenize(&self, text: &str) -> Result<Vec<TokenId>>;
    fn detokenize(&self, tokens: &[TokenId]) -> Result<String>;
    fn prefill(
        &self,
        session: &mut Session,
        tokens: &[TokenId],
    ) -> Result<PrefillResult>;
    fn decode(&self, session: &mut Session) -> Result<DecodeResult>;
    fn embed(&self, texts: &[&str]) -> Result<Vec<Vec<f32>>>;
}

The core engine trait — everything else plugs into this.

Implementations provide inference, tokenization, and embedding functionality. oxidizedRAG and oxidizedgraph depend on engine behavior, not implementation details. Swap CPU/Metal/FFI backends without changing application code.

Required Methods

fn load_model(&self, spec: &ModelSpec) -> Result<ModelHandle>

Load a model from disk given a specification.

fn tokenize(&self, text: &str) -> Result<Vec<TokenId>>

Convert text into a sequence of token IDs.

fn detokenize(&self, tokens: &[TokenId]) -> Result<String>

Convert token IDs back into text.

fn prefill( &self, session: &mut Session, tokens: &[TokenId], ) -> Result<PrefillResult>

Run the prefill phase: process prompt tokens and populate the KV cache.

fn decode(&self, session: &mut Session) -> Result<DecodeResult>

Run the decode phase: produce the next token from the model.

fn embed(&self, texts: &[&str]) -> Result<Vec<Vec<f32>>>

Generate embeddings for a batch of texts (for oxidizedRAG integration).

Implementors