Struct SpecDecoder

Source

pub struct SpecDecoder<D: Speculator, T: Speculator> {
    pub draft: D,
    pub target: T,
    pub n: usize,
    /* private fields */
}

Expand description

Top-level orchestrator. Holds a draft + target speculator and the lookahead window n. step() runs one full round and returns the tokens to append to the running context.

Fields§

§draft: D§target: T§n: usize

Implementations§

Source §

impl<D: Speculator, T: Speculator> SpecDecoder<D, T>

Source

pub fn new(draft: D, target: T, n: usize, seed: u64) -> Self

Source

pub fn step(&mut self, context: &[u32]) -> Vec<u32>

One speculative-decoding round. Returns the tokens that should be appended to context.

Auto Trait Implementations§

§

impl<D, T> Freeze for SpecDecoder<D, T>
where D: Freeze, T: Freeze,

§

impl<D, T> RefUnwindSafe for SpecDecoder<D, T>
where D: RefUnwindSafe, T: RefUnwindSafe,

§

impl<D, T> UnsafeUnpin for SpecDecoder<D, T>
where D: UnsafeUnpin, T: UnsafeUnpin,

§

impl<D, T> UnwindSafe for SpecDecoder<D, T>
where D: UnwindSafe, T: UnwindSafe,

Blanket Implementations§

Source §

impl<T> Any for T
where T: 'static + ?Sized,

Source §

fn type_id(&self) -> TypeId

Gets the TypeId of self. Read more

Source §

impl<T> Borrow<T> for T
where T: ?Sized,

Source §

fn borrow(&self) -> &T

Immutably borrows from an owned value. Read more

Source §

impl<T> BorrowMut<T> for T
where T: ?Sized,

Source §

fn borrow_mut(&mut self) -> &mut T

Mutably borrows from an owned value. Read more

Source §

impl<T> From<T> for T

Source §

fn from(t: T) -> T

Returns the argument unchanged.

Source §

impl<T, U> Into for T
where U: From<T>,

Source §

fn into(self) -> U

Calls U::from(self).

That is, this conversion is whatever the implementation of From<T> for U chooses to do.

Source §

impl<T> IntoEither for T

Source §

fn into_either(self, into_left: bool) -> Either<Self, Self>

Converts self into a Left variant of Either<Self, Self> if into_left is true. Converts self into a Right variant of Either<Self, Self> otherwise. Read more

Source §

fn into_either_with<F>(self, into_left: F) -> Either<Self, Self>
where F: FnOnce(&Self) -> bool,

Converts self into a Left variant of Either<Self, Self> if into_left(&self) returns true. Converts self into a Right variant of Either<Self, Self> otherwise. Read more

Source §