Expand description
§spec-stream-core
Core primitives, data structures, and mathematical verification algorithms for speculative decoding in high-throughput Large Language Model (LLM) inference.
Designed to be model-agnostic, zero-allocation where possible, and strictly compliant with exact output probability distributions (Leviathan et al., 2023).
Re-exports§
pub use error::CoreError;pub use logits::Logits;pub use logits::TokenId;pub use metrics::SpeculativeMetrics;pub use tokenizer::ByteLevelTokenizer;pub use tokenizer::Tokenizer;pub use tree::SpeculativeTree;pub use tree::TreeNode;pub use tree::TreeVerificationResult;pub use types::DraftProposal;pub use types::SamplingParams;pub use types::TokenCandidate;pub use types::VerificationResult;pub use types::VerificationStrategy;pub use verifier::SpeculativeVerifier;