Module gpt2 

GPT-2 model assembly and generation.

Structs

AttnStep

One block’s attention probabilities for one decode step, read back from the device.

Gpt2

Gpt2Config

KvCache

Preallocated per-layer K/V tensors ([n_head, n_ctx, head_dim]) for incremental decode. len positions are filled; new tokens append.

LayerDetail

One block’s Q/K/V, MLP and output tensors for one step, read back from the device. Captured only for the first detail_layers blocks — the expensive kinds — while AttnStep is captured for every block.

Sampler

Draws tokens from a row of logits, carrying the sampling stream with it.

StepTrace

Everything one decode step computed, read back in a single batched round trip. A strict superset of AttnStep: attn is exactly what Gpt2::logits_step_attn_async reports for the same step.

Surprisal

Per-position surprisal from Gpt2::surprisal_async. All three vectors are the length of the input, and index 0 is a placeholder in each — nothing precedes the first token.

Enums

Sampling

Functions

top_probs

The n most likely next tokens and their probabilities, most likely first.