Skip to main content

LlmBackend

Trait LlmBackend 

Source
pub trait LlmBackend:
    Backend
    + BackendGraph
    + BackendPagedKv { }
Expand description

Minimum capability set for a decoder-only LLM: the core compute trait plus paged-KV cache + graph-capture support. Every concrete backend (CUDA / Metal / CPU) satisfies this.

Dyn Compatibility§

This trait is not dyn compatible.

In older versions of Rust, dyn compatibility was called "object safety".

Implementors§