Expand description
Sampling and logits processing interfaces
This module provides abstractions for sampling tokens from model outputs, including various sampling strategies and logits processors. These are completely separate from model execution to allow for flexible composition.
Structs§
- Greedy
Sampler - Common samplers Greedy sampler (always picks highest probability token)
- Logits
Processor Chain - Logits processor chain for composing multiple processors
- Multinomial
Sampler - Multinomial sampler for probabilistic sampling
- Repetition
Penalty Processor - Repetition penalty processor
- Sampling
Config - Complete sampling configuration
- Sampling
Config Builder - Sampling configuration builder
- Sampling
Context - Sampling context passed to logits processors and samplers
- Sampling
Stats - Sampling statistics for monitoring
- Temperature
Processor - Common logits processors Temperature scaling processor
- TopK
Processor - Top-k filtering processor
- TopP
Processor - Top-p (nucleus) filtering processor
Enums§
- Processor
Priority - Priority levels for logits processors
Traits§
- Logits
Processor - Logits processor trait for modifying raw model outputs
- Multi
Sampler - Multi-sample capability for beam search and parallel sampling
- Sampler
- Token sampler trait for selecting next token from processed logits