Struct TextModelBase 

pub struct TextModelBase {
    pub ctx: Context,
    pub tokenizer: Tokenizer,
    pub embedding: Embedding,
    pub eos_token_id: Option<EosTokenId>,
    pub index_pos: usize,
    pub generated: usize,
    pub prompt_len: usize,
    pub blocks: Vec<Box<dyn Forwarder>>,
    pub ln_f: RmsNorm,
    pub lm_head: Linear,
    pub logits_processor: LogitsProcessor,
    pub tokens: Vec<u32>,
}

Shared base for decoder-only text models (LLaMA, Qwen2, Qwen3.5, etc.).

Contains all the state and logic that is identical across model architectures: embedding, transformer blocks, final norm, lm_head, tokenizer, sampling, and the forward/generation loop.

Fields

ctx: Context

tokenizer: Tokenizer

embedding: Embedding

eos_token_id: Option<EosTokenId>

index_pos: usize

generated: usize

prompt_len: usize

blocks: Vec<Box<dyn Forwarder>>

ln_f: RmsNorm

lm_head: Linear

logits_processor: LogitsProcessor

tokens: Vec<u32>

Implementations

impl TextModelBase

pub async fn load<B: Forwarder + 'static>( ctx: &mut Context, default_eos_token: &str, ) -> Result<Self>

Load the shared model structure from the context. default_eos_token is the model-specific fallback EOS string. The type parameter B determines which block type to use for local layers.

pub async fn forward(&mut self, x: &Tensor, idx: usize) -> Result<Tensor>

Forward pass through all blocks.

pub fn prepare_prompt(&mut self, dialog: &str) -> Result<()>

Tokenize a prompt string and set up token state for generation.

pub async fn next_token(&mut self, index: usize) -> Result<Token>

Generate the next token. Assumes prepare_prompt() has been called for the first token.

pub fn reset(&mut self)

Reset all generation state.

pub async fn goodbye(&mut self) -> Result<()>

Notify all remote blocks of session end (clears their KV caches).