Trait Model 

pub trait Model {
    // Required methods
    fn forward(&self, token_ids: &[u32], cache: &mut KvCache) -> Result<Tensor>;
    fn vocab_size(&self) -> usize;
    fn max_context(&self) -> usize;
    fn new_cache(&self) -> KvCache;
}

A causal (decoder-only) language model that can run a forward pass and produce next-token logits.

Why this trait exists with exactly one implementation

crate::model::QwenModel is the only Model today. That is not reason enough on its own to add a trait — CLAUDE.md is explicit that unused abstraction is a cost, not a virtue — but this one earns its keep on two counts: crate::generate is written against Model, not QwenModel, so it does not have to change when a second architecture (a non-GQA LLaMA checkpoint, say, or a future non-transformer architecture) arrives; and it marks precisely the seam a second architecture would implement against, the same role kopitiam_core::Device’s single Cpu variant plays for backends (see that type’s docs, which this trait’s shape deliberately echoes).

Required Methods

fn forward(&self, token_ids: &[u32], cache: &mut KvCache) -> Result<Tensor>

Runs a forward pass over token_ids — new tokens only (a multi-token prompt on the first call, ordinarily one token per call thereafter) — using and extending cache. Returns logits shaped [token_ids.len(), vocab_size]: one row of unnormalized per-vocabulary scores for every input position, in the same order as token_ids.

fn vocab_size(&self) -> usize

The vocabulary size logits’ last dimension is sized to.

fn max_context(&self) -> usize

The context window cache should be constructed with via KvCache::new to run this model without hitting kopitiam_core::Error::IndexOutOfBounds from KvCache::append.

fn new_cache(&self) -> KvCache

Builds a fresh, empty KvCache sized correctly for this model (layer count and context window). This is the only correct way to construct a cache for a given model — KvCache::new takes those two numbers as bare usizes and trusts the caller to get them right, which is fine for KvCache’s own unit tests but exactly the kind of easy-to-mismatch call crate::generate::generate (generic over any Model, not just crate::model::QwenModel) should never have to get right by hand.

Dyn Compatibility

This trait is dyn compatible.

In older versions of Rust, dyn compatibility was called "object safety".

Implementors

impl Model for QwenModel