Trait RecurrentStateHandle 

pub trait RecurrentStateHandle:
    Send
    + Sync
    + Debug {
    // Required methods
    fn request_id(&self) -> RequestId;
    fn device(&self) -> Device;
    fn num_layers(&self) -> usize;
    fn state_bytes(&self) -> usize;
    fn clone_handle(&self) -> Result<Arc<dyn RecurrentStateHandle>>;
    fn as_any(&self) -> &dyn Any;
    fn stats(&self) -> RecurrentStateHandleStats;
    fn is_valid(&self) -> bool;
    fn cache_id(&self) -> String;
}

Recurrent-state cache handle.

Required Methods

fn request_id(&self) -> RequestId

Request ID this state belongs to.

fn device(&self) -> Device

Device where the state resides.

fn num_layers(&self) -> usize

Number of model layers represented by this handle.

fn state_bytes(&self) -> usize

Approximate memory used by this state.

fn clone_handle(&self) -> Result<Arc<dyn RecurrentStateHandle>>

Clone handle reference. Implementations should not deep-copy state.

fn as_any(&self) -> &dyn Any

Downcast support for backend-specific handles.

fn stats(&self) -> RecurrentStateHandleStats

Handle statistics.

fn is_valid(&self) -> bool

Check whether state is still valid and accessible.

fn cache_id(&self) -> String

Unique identifier for this cache instance.

Dyn Compatibility

This trait is dyn compatible.

In older versions of Rust, dyn compatibility was called "object safety".

Implementors