Module gpt2 

GPT-2 model assembly and generation.

Structs

AttnStep

One block’s attention probabilities for one decode step, read back from the device.

Gpt2

Gpt2Config

KvCache

Preallocated per-layer K/V tensors ([n_head, n_ctx, head_dim]) for incremental decode. len positions are filled; new tokens append.

LayerDetail

One block’s Q/K/V, MLP and output tensors for one step, read back from the device. Captured only for the first detail_layers blocks — the expensive kinds — while AttnStep is captured for every block.

StepTrace

Everything one decode step computed, read back in a single batched round trip. A strict superset of AttnStep: attn is exactly what Gpt2::logits_step_attn_async reports for the same step.

Enums

Sampling