Expand description
Speculative decoding utilities for draft/target verification.
Structs§
- Prediction
Config - Configuration for the prediction head.
- Speculative
Candidates - Batch of speculative trees.
- Speculative
Decoder - Speculative decoder for draft/target model verification.
- Speculative
Token - Token entry in the speculative tree.
- Speculative
Tree - Speculative token tree arranged by depth levels.
- Speculative
Verification - Verification output with accepted tokens and updated cache.
- Tree
Config - Configuration for speculative tree expansion.
Enums§
- Prediction
Head Type - Prediction head choices.
- Verification
Strategy - Verification strategy for speculative decoding.