Trait LanguageModelLike 

pub trait LanguageModelLike {
    // Required methods
    fn log_prob_sequence(&self, tokens: &[&str]) -> Option<f64>;
    fn vocabulary_size(&self) -> usize;
}

Trait for language models that can compute conditional log-probability over a token sequence.

Implementors expose string tokens so the trait is usable with both word-level and character-level models without a vocabulary-bridge adapter.

Required Methods

fn log_prob_sequence(&self, tokens: &[&str]) -> Option<f64>

Return Σ_t log p(tokens[t] | tokens[0..t]) for the whole sequence.

Returns None when the sequence is empty.

fn vocabulary_size(&self) -> usize

Size of the model vocabulary (used by tests / diagnostics only).

Implementors

impl LanguageModelLike for NgramModel

Implement LanguageModelLike for the wired crate::language_model::NgramModel.

Delegates to NgramModel::probability with the preceding n-1 tokens as context.

impl LanguageModelLike for NgramLM

Implement LanguageModelLike for NgramLM from language_models module.

NgramLM uses string-based context windows with Kneser-Ney smoothing.