Struct Eagle3Session 

pub struct Eagle3Session<'ctx, 'target_model, 'draft_model> { /* private fields */ }

Owned EAGLE-3 draft session.

Drops the underlying speculative context when freed.

Both contexts are exclusively borrowed for the session lifetime. The wrapper retains no manually enforced lifetime and is neither Send nor Sync.

Implementations

impl<'ctx, 'target_model, 'draft_model> Eagle3Session<'ctx, 'target_model, 'draft_model>

pub fn new( target: &'ctx mut LlamaContext<'target_model>, draft: &'ctx mut LlamaContext<'draft_model>, n_seq: u32, n_draft_max: i32, ) -> Result<Self, Eagle3SessionError>

Construct an EAGLE-3 draft session with upstream defaults for n_min and p_min.

Equivalent to new_with_config(target, draft, Eagle3SessionConfig::new(n_seq, n_draft_max)).

Errors

Returns Eagle3SessionError::Init or Eagle3SessionError::InvalidConfig.

pub fn new_with_config( target: &'ctx mut LlamaContext<'target_model>, draft: &'ctx mut LlamaContext<'draft_model>, config: Eagle3SessionConfig, ) -> Result<Self, Eagle3SessionError>

Construct an EAGLE-3 draft session with full speculative draft parameters.

target must be a LlamaContextType::Default context over the main model. draft must be a Default context over a separate EAGLE-3 draft model trained against that target.

Errors

Returns Eagle3SessionError::Init (e.g. the draft model is not a valid EAGLE-3 model) or Eagle3SessionError::InvalidConfig.

pub fn config(&self) -> Eagle3SessionConfig

Session configuration passed at construction.

pub fn need_embd(&self) -> bool

True when the speculative backend needs post-norm embeddings on the target context (llama_set_embeddings).

pub fn need_embd_pre_norm(&self) -> bool

True when the speculative backend needs pre-norm hidden states on the target context (llama_set_embeddings_pre_norm).

Configured automatically during session init; callers normally do not need to set it manually.

pub fn n_draft_max(&self) -> i32

Configured maximum number of tokens drafted per draft call.

pub fn n_min(&self) -> i32

Configured minimum draft tokens (n_min).

pub fn p_min(&self) -> f32

Configured draft probability floor (p_min).

pub fn n_seq(&self) -> u32

Configured number of sequences.

pub fn target_context(&self) -> &LlamaContext<'target_model>

Returns shared access to the target context for reading logits, embeddings, and model metadata.

pub fn target_context_mut(&mut self) -> &mut LlamaContext<'target_model>

Returns exclusive access to the target context while this wrapper retains native pointer ownership.

pub fn draft_context(&self) -> &LlamaContext<'draft_model>

Returns shared access to the draft context for metadata inspection.

pub fn draft_context_mut(&mut self) -> &mut LlamaContext<'draft_model>

Returns exclusive access to the draft context while this wrapper retains native pointer ownership.

pub fn decode_target_and_process( &mut self, batch: &mut LlamaBatch, ) -> Result<(), Eagle3SessionError>

Decodes on the target and immediately harvests the same batch into EAGLE-3.

Errors

Returns a target crate::DecodeError or native process failure.

pub fn decode_target( &mut self, batch: &mut LlamaBatch, ) -> Result<(), Eagle3SessionError>

Decodes one batch on the exclusively held target context.

Use Self::decode_target_and_process unless mechanics must run between target decode and draft-state harvesting. This method remains available while a draft proposal is pending because that is the target verification phase; proposal creation, begin, and state access retain their stricter lifecycle checks.

Errors

Returns a target crate::DecodeError.

pub fn print_stats(&self)

Log speculative-decoding statistics (draft/accept counts and timings) via llama.cpp LOG_INF. Install a log callback with crate::log_set to capture output.

pub fn begin( &mut self, seq_id: i32, prompt: &[LlamaToken], ) -> Result<(), Eagle3SessionError>

Optional: call once at the start of a fresh generation with the prompt tokens that were just decoded into the target context.

Errors

Returns Eagle3SessionError::BadSeqId if seq_id is out of range.

pub fn process(&mut self, batch: &LlamaBatch) -> Result<(), Eagle3SessionError>

Hand the session a batch that was just decoded on the target context.

Call this after every successful target.decode(batch) so upstream can harvest the target hidden states EAGLE-3 drafts from.

Errors

Returns Eagle3SessionError::Process if the underlying call fails.

pub fn draft( &mut self, seq_id: i32, n_past: i32, id_last: LlamaToken, ) -> Result<Vec<LlamaToken>, Eagle3SessionError>

Generate up to n_draft_max speculative tokens.

n_past is the number of tokens already in the target KV cache for seq_id. id_last is the last token accepted on the target (usually the token you just sampled).

Errors

Returns Eagle3SessionError::BadSeqId if seq_id is out of range.

pub fn accept( &mut self, seq_id: i32, n_accepted: u16, ) -> Result<(), Eagle3SessionError>

Inform the session how many draft tokens the target verifier accepted.

Pass 0 when every draft was rejected.

Errors

Returns Eagle3SessionError::BadSeqId if seq_id is out of range.

pub fn is_quiescent(&self) -> bool

Returns true when every draft proposal has been completed.

pub fn speculative_state( &self, seq_id: i32, ) -> Result<Vec<u8>, Eagle3SessionError>

Captures versioned per-sequence speculative continuation state.

Target and draft context bytes are separate and must be checkpointed at the same quiescent boundary.

Errors

Returns an error for an invalid sequence, pending proposal, incomplete native support, or excessive state.

pub fn restore_speculative_state( &mut self, seq_id: i32, state: &[u8], ) -> Result<(), Eagle3SessionError>

Restores versioned per-sequence speculative continuation state.

Restore the corresponding target and draft context bytes before calling this method.

Errors

Returns an error for an invalid sequence, pending proposal, excessive input, or any version/configuration/state mismatch.

pub fn clear_target_kv_cache_seq( &mut self, seq_id: Option<u32>, p0: Option<u32>, p1: Option<u32>, ) -> Result<bool, KvCacheConversionError>

Removes a target-context KV range.

Errors

Returns a conversion error when an identifier or position exceeds native i32 bounds.

pub fn clear_draft_kv_cache_seq( &mut self, seq_id: Option<u32>, p0: Option<u32>, p1: Option<u32>, ) -> Result<bool, KvCacheConversionError>

Removes a draft-context KV range.

Errors

Returns a conversion error when an identifier or position exceeds native i32 bounds.

pub fn target_state_seq_get_size_ext( &mut self, seq_id: i32, flags: u32, ) -> usize

Returns the target context’s exact sequence-state byte count.

pub fn target_state_seq_get_data_ext( &mut self, dst: &mut [u8], seq_id: i32, flags: u32, ) -> usize

Copies target context sequence state with exact native flags.

pub fn target_state_seq_set_data_ext( &mut self, src: &[u8], seq_id: i32, flags: u32, ) -> usize

Restores target context sequence state with exact native flags.

pub fn draft_state_seq_get_size_ext(&mut self, seq_id: i32, flags: u32) -> usize

Returns the draft context’s exact sequence-state byte count.

pub fn draft_state_seq_get_data_ext( &mut self, dst: &mut [u8], seq_id: i32, flags: u32, ) -> usize

Copies draft context sequence state with exact native flags.

pub fn draft_state_seq_set_data_ext( &mut self, src: &[u8], seq_id: i32, flags: u32, ) -> usize

Restores draft context sequence state with exact native flags.

Trait Implementations

impl Debug for Eagle3Session<'_, '_, '_>

fn fmt(&self, f: &mut Formatter<'_>) -> Result

Formats the value using the given formatter.

impl Drop for Eagle3Session<'_, '_, '_>

fn drop(&mut self)

Executes the destructor for this type.

fn pin_drop(self: Pin<&mut Self>)

🔬This is a nightly-only experimental API. (pin_ergonomics)

Execute the destructor for this type, but different to Drop::drop, it requires self to be pinned.