pub struct SpeculativeTree {
pub nodes: Vec<TreeNode>,
}Expand description
Tree structure representing multiple branch proposals for speculative decoding.
Fields§
§nodes: Vec<TreeNode>Implementations§
Source§impl SpeculativeTree
impl SpeculativeTree
pub fn new(root_token: u32) -> SpeculativeTree
Sourcepub fn add_child(
&mut self,
parent_idx: usize,
token_id: u32,
prob: f32,
) -> Result<usize, CoreError>
pub fn add_child( &mut self, parent_idx: usize, token_id: u32, prob: f32, ) -> Result<usize, CoreError>
Adds a child branch to an existing node.
Sourcepub fn compute_tree_attention_mask(&self) -> Vec<Vec<bool>>
pub fn compute_tree_attention_mask(&self) -> Vec<Vec<bool>>
Generates the 2D Boolean causal/tree attention mask for parallel target model execution.
Returns an NxN boolean matrix where mask[i][j] == true if node i can attend to node j.
Sourcepub fn verify_greedy(
&self,
target_logits: &[Logits],
) -> Result<TreeVerificationResult, CoreError>
pub fn verify_greedy( &self, target_logits: &[Logits], ) -> Result<TreeVerificationResult, CoreError>
Verifies the entire candidate tree against the target model’s logits. Finds the longest valid path where the target model’s prediction at each parent matches the child token.
Trait Implementations§
Source§impl Clone for SpeculativeTree
impl Clone for SpeculativeTree
Source§fn clone(&self) -> SpeculativeTree
fn clone(&self) -> SpeculativeTree
Returns a duplicate of the value. Read more
1.0.0 (const: unstable) · Source§fn clone_from(&mut self, source: &Self)
fn clone_from(&mut self, source: &Self)
Performs copy-assignment from
source. Read moreSource§impl Debug for SpeculativeTree
impl Debug for SpeculativeTree
Source§impl Default for SpeculativeTree
impl Default for SpeculativeTree
Source§fn default() -> SpeculativeTree
fn default() -> SpeculativeTree
Returns the “default value” for a type. Read more
Auto Trait Implementations§
impl Forget for SpeculativeTree
impl Freeze for SpeculativeTree
impl Move for SpeculativeTree
impl RefUnwindSafe for SpeculativeTree
impl Send for SpeculativeTree
impl Sync for SpeculativeTree
impl Unpin for SpeculativeTree
impl UnsafeUnpin for SpeculativeTree
impl UnwindSafe for SpeculativeTree
Blanket Implementations§
Source§impl<T> BorrowMut<T> for Twhere
T: ?Sized,
impl<T> BorrowMut<T> for Twhere
T: ?Sized,
Source§fn borrow_mut(&mut self) -> &mut T
fn borrow_mut(&mut self) -> &mut T
Mutably borrows from an owned value. Read more