pub struct PromptLookupEngine { /* private fields */ }Expand description
Prompt Lookup speculative drafting engine.
Searches for matching n-gram suffixes in the prompt context to predict subsequent tokens.
Implementations§
Source§impl PromptLookupEngine
impl PromptLookupEngine
Sourcepub fn new(context: Vec<u32>, config: PromptLookupConfig) -> PromptLookupEngine
pub fn new(context: Vec<u32>, config: PromptLookupConfig) -> PromptLookupEngine
Creates a new prompt lookup engine initialized with a prompt context.
Sourcepub fn append_to_context(&mut self, tokens: &[u32])
pub fn append_to_context(&mut self, tokens: &[u32])
Appends generated tokens to the active context pool so the engine learns from ongoing generation.
Sourcepub fn find_candidate_tokens(&self, history: &[u32], k: usize) -> Vec<u32>
pub fn find_candidate_tokens(&self, history: &[u32], k: usize) -> Vec<u32>
Finds candidate tokens by matching the suffix of history against self.context.
Sourcepub fn draft_proposal(&self, history: &[u32], k: usize) -> DraftProposal
pub fn draft_proposal(&self, history: &[u32], k: usize) -> DraftProposal
Generates a speculative DraftProposal directly from prompt n-gram matching.
Trait Implementations§
Source§impl Clone for PromptLookupEngine
impl Clone for PromptLookupEngine
Source§fn clone(&self) -> PromptLookupEngine
fn clone(&self) -> PromptLookupEngine
Returns a duplicate of the value. Read more
1.0.0 (const: unstable) · Source§fn clone_from(&mut self, source: &Self)
fn clone_from(&mut self, source: &Self)
Performs copy-assignment from
source. Read moreSource§impl Debug for PromptLookupEngine
impl Debug for PromptLookupEngine
Source§impl ModelEngine for PromptLookupEngine
impl ModelEngine for PromptLookupEngine
Source§async fn forward_batch(
&mut self,
tokens: &[u32],
) -> Result<Vec<Logits>, EngineError>
async fn forward_batch( &mut self, tokens: &[u32], ) -> Result<Vec<Logits>, EngineError>
Computes logits for a sequence of tokens in a single parallel forward pass.
Source§async fn forward_token(&mut self, token: u32) -> Result<Logits, EngineError>
async fn forward_token(&mut self, token: u32) -> Result<Logits, EngineError>
Computes logits for a single token step.
Auto Trait Implementations§
impl Forget for PromptLookupEngine
impl Freeze for PromptLookupEngine
impl Move for PromptLookupEngine
impl RefUnwindSafe for PromptLookupEngine
impl Send for PromptLookupEngine
impl Sync for PromptLookupEngine
impl Unpin for PromptLookupEngine
impl UnsafeUnpin for PromptLookupEngine
impl UnwindSafe for PromptLookupEngine
Blanket Implementations§
Source§impl<T> BorrowMut<T> for Twhere
T: ?Sized,
impl<T> BorrowMut<T> for Twhere
T: ?Sized,
Source§fn borrow_mut(&mut self) -> &mut T
fn borrow_mut(&mut self) -> &mut T
Mutably borrows from an owned value. Read more