Trait LanguageModelLike 

pub trait LanguageModelLike {
    // Required methods
    fn log_prob_sequence(&self, tokens: &[&str]) -> Option<f64>;
    fn vocabulary_size(&self) -> usize;
}

Trait for language models that can compute conditional log-probability over a token sequence.

Implementors expose string tokens so the trait is usable with both word-level and character-level models without a vocabulary-bridge adapter.

Required Methods

fn log_prob_sequence(&self, tokens: &[&str]) -> Option<f64>

Return Σ_t log p(tokens[t] | tokens[0..t]) for the whole sequence.

Returns None when the sequence is empty.

fn vocabulary_size(&self) -> usize

Size of the model vocabulary (used by tests / diagnostics only).

Dyn Compatibility

This trait is dyn compatible.

In older versions of Rust, dyn compatibility was called "object safety".

Implementors

impl LanguageModelLike for NgramLM

Implement LanguageModelLike for NgramLM from language_models module.

NgramLM uses string-based context windows with Kneser-Ney smoothing.

impl LanguageModelLike for NgramModel

Implement LanguageModelLike for the wired crate::language_model::NgramModel.

Delegates to NgramModel::probability with the preceding n-1 tokens as context.