Expand description
Token sampling strategies for autoregressive generation.
Supports greedy, top-k, top-p (nucleus), and temperature-scaled sampling.
Structs§
- Sampling
Config - Sampling configuration.
Functions§
- apply_
repetition_ penalty - Apply repetition penalty to logits for previously generated tokens.
- argmax
- Greedy sampling: return the token with the highest logit.
- sample
- Sample a token ID from logits using the given config.