Struct LlamaStateSeqFlags 

pub struct LlamaStateSeqFlags(/* private fields */);

Flags for state sequence operations.

These flags control what parts of the state are included when saving/restoring sequence state.

Implementations

impl LlamaStateSeqFlags

pub const PARTIAL_ONLY: LlamaStateSeqFlags

Work only with partial states, such as SWA KV cache or recurrent cache (e.g. Mamba).

This flag is useful when you only want to save/restore the recurrent state without affecting the KV cache.

pub const ON_DEVICE: LlamaStateSeqFlags

Keep the copied data on device (GPU) memory rather than host.

Getting the state for a seq_id with this flag invalidates all prior states obtained for that seq_id with this flag set.

pub const fn empty() -> LlamaStateSeqFlags

Create an empty flags set.

pub const fn bits(&self) -> u32

Get the raw flags value.

pub const fn from_bits(bits: u32) -> LlamaStateSeqFlags

Construct from raw bits.

pub const fn contains(&self, other: LlamaStateSeqFlags) -> bool

Check if a flag is set.

Trait Implementations

impl BitOr for LlamaStateSeqFlags

type Output = LlamaStateSeqFlags

The resulting type after applying the | operator.

fn bitor(self, other: Self) -> Self

Performs the | operation.

impl Clone for LlamaStateSeqFlags

fn clone(&self) -> LlamaStateSeqFlags

Returns a duplicate of the value.

fn clone_from(&mut self, source: &Self)

Performs copy-assignment from source.

impl Copy for LlamaStateSeqFlags

impl Debug for LlamaStateSeqFlags

fn fmt(&self, f: &mut Formatter<'_>) -> Result

Formats the value using the given formatter.

impl Default for LlamaStateSeqFlags

fn default() -> Self

Returns the “default value” for a type.

impl Eq for LlamaStateSeqFlags

impl PartialEq for LlamaStateSeqFlags

fn eq(&self, other: &LlamaStateSeqFlags) -> bool

Equality operator ==.

fn ne(&self, other: &Rhs) -> bool

Inequality operator !=.

impl StructuralPartialEq for LlamaStateSeqFlags