Struct QwenModel 

pub struct QwenModel { /* private fields */ }

A loaded, ready-to-run Qwen-family transformer.

Implementations

impl QwenModel

pub fn from_loaded_model(model: &LoadedModel) -> Result<Self>

Builds a QwenModel from an already-parsed LoadedModel (kopitiam_loader::load_model’s result): resolves QwenConfig::from_metadata, loads and dequantizes every weight tensor via [crate::weights::ModelWeights::load], and precomputes RoPE’s rotation tables up to the model’s context window.

pub fn config(&self) -> &QwenConfig

pub fn backend(&self) -> &CpuBackend

Trait Implementations

impl Model for QwenModel

fn forward(&self, token_ids: &[u32], cache: &mut KvCache) -> Result<Tensor>

Runs a forward pass over token_ids — new tokens only (a multi-token prompt on the first call, ordinarily one token per call thereafter) — using and extending cache. Returns logits shaped [token_ids.len(), vocab_size]: one row of unnormalized per-vocabulary scores for every input position, in the same order as token_ids.

fn vocab_size(&self) -> usize

The vocabulary size logits’ last dimension is sized to.

fn max_context(&self) -> usize

The context window cache should be constructed with via KvCache::new to run this model without hitting kopitiam_core::Error::IndexOutOfBounds from KvCache::append.

fn new_cache(&self) -> KvCache

Builds a fresh, empty KvCache sized correctly for this model (layer count and context window). This is the only correct way to construct a cache for a given model — KvCache::new takes those two numbers as bare usizes and trusts the caller to get them right, which is fine for KvCache’s own unit tests but exactly the kind of easy-to-mismatch call crate::generate::generate (generic over any Model, not just crate::model::QwenModel) should never have to get right by hand.