pub struct TreeSpeculativeConfig {
pub branching_factor: usize,
pub depth: usize,
pub max_new_tokens: usize,
pub eos_token_id: Option<TokenId>,
}Expand description
Configuration for Tree-based Speculative Decoding (Medusa / Eagle style).
Fields§
§branching_factor: usizeNumber of candidate branches to expand from each node (typically 2 or 3).
depth: usizeTree exploration depth (typically 2 or 3).
max_new_tokens: usizeMaximum new tokens to generate before stopping.
eos_token_id: Option<TokenId>Optional End-of-Sequence token ID to terminate generation.
Trait Implementations§
Source§impl Clone for TreeSpeculativeConfig
impl Clone for TreeSpeculativeConfig
Source§impl Debug for TreeSpeculativeConfig
impl Debug for TreeSpeculativeConfig
Auto Trait Implementations§
impl Forget for TreeSpeculativeConfig
impl Freeze for TreeSpeculativeConfig
impl Move for TreeSpeculativeConfig
impl RefUnwindSafe for TreeSpeculativeConfig
impl Send for TreeSpeculativeConfig
impl Sync for TreeSpeculativeConfig
impl Unpin for TreeSpeculativeConfig
impl UnsafeUnpin for TreeSpeculativeConfig
impl UnwindSafe for TreeSpeculativeConfig
Blanket Implementations§
Source§impl<T> BorrowMut<T> for Twhere
T: ?Sized,
impl<T> BorrowMut<T> for Twhere
T: ?Sized,
Source§fn borrow_mut(&mut self) -> &mut T
fn borrow_mut(&mut self) -> &mut T
Mutably borrows from an owned value. Read more