Struct CausalLMOutput

pub struct CausalLMOutput { /* private fields */ }

Causal language model output.

Implementations

impl CausalLMOutput

pub fn new(decoder_output: DecoderOutput, logits: Tensor) -> Self

Create a causal language model output.

pub fn decoder_output(&self) -> &DecoderOutput

Get the output of the decoder used by the causal language model.

pub fn logits(&self) -> &Tensor

Get the logits of the next predicted token.

The logits are the unnormalized probabilities. Applying softmax to the logits will give the probability distribution of the next token over the vocabulary.

Trait Implementations

impl LayerOutputs for CausalLMOutput

fn layer_outputs(&self) -> &[Tensor]

Outputs of all layers.

fn embedding_layer_output(&self) -> Option<&Tensor>

Output of the embedding layer.