Skip to main content

Module pipeline

Module pipeline 

Source

Structsยง

SpeculativeConfig
Configuration parameters for speculative decoding.
SpeculativePipeline
Orchestrator for speculative decoding execution and streaming.