pub struct SpeculativeTree {
pub nodes: Vec<TreeNode>,
}Expand description
Tree structure representing multiple branch proposals for speculative decoding.
Fields§
§nodes: Vec<TreeNode>Implementations§
Source§impl SpeculativeTree
impl SpeculativeTree
pub fn new(root_token: TokenId) -> Self
Sourcepub fn add_child(
&mut self,
parent_idx: usize,
token_id: TokenId,
prob: f32,
) -> Result<usize, CoreError>
pub fn add_child( &mut self, parent_idx: usize, token_id: TokenId, prob: f32, ) -> Result<usize, CoreError>
Adds a child branch to an existing node.
Sourcepub fn compute_tree_attention_mask(&self) -> Vec<Vec<bool>>
pub fn compute_tree_attention_mask(&self) -> Vec<Vec<bool>>
Generates the 2D Boolean causal/tree attention mask for parallel target model execution.
Returns an NxN boolean matrix where mask[i][j] == true if node i can attend to node j.
Sourcepub fn verify_greedy(
&self,
target_logits: &[Logits],
) -> Result<TreeVerificationResult, CoreError>
pub fn verify_greedy( &self, target_logits: &[Logits], ) -> Result<TreeVerificationResult, CoreError>
Verifies the entire candidate tree against the target model’s logits. Finds the longest valid path where the target model’s prediction at each parent matches the child token.
Trait Implementations§
Source§impl Clone for SpeculativeTree
impl Clone for SpeculativeTree
Source§impl Debug for SpeculativeTree
impl Debug for SpeculativeTree
Auto Trait Implementations§
impl Forget for SpeculativeTree
impl Freeze for SpeculativeTree
impl Move for SpeculativeTree
impl RefUnwindSafe for SpeculativeTree
impl Send for SpeculativeTree
impl Sync for SpeculativeTree
impl Unpin for SpeculativeTree
impl UnsafeUnpin for SpeculativeTree
impl UnwindSafe for SpeculativeTree
Blanket Implementations§
Source§impl<T> BorrowMut<T> for Twhere
T: ?Sized,
impl<T> BorrowMut<T> for Twhere
T: ?Sized,
Source§fn borrow_mut(&mut self) -> &mut T
fn borrow_mut(&mut self) -> &mut T
Mutably borrows from an owned value. Read more