Module backend 

Embedding backend abstraction layer.

Defines the EmbedBackend trait that all embedding backends (CPU, CUDA, Metal, MLX) implement, plus the Encoding input type and BackendKind discriminant. Use load_backend to construct a backend by kind.

Modules

arch

Model architecture trait and variant enum.

blas_info

Runtime BLAS detection and optimization recommendations.

driver

Hardware-agnostic compute driver trait.

generic

Generic backend that pairs a Driver with a ModelArch.

Structs

Encoding

Pre-tokenized encoding ready for inference.

InferenceOpts

Inference optimization parameters passed through to model loading.

Enums

BackendKind

Available embedding backend implementations.

DeviceHint

Device hint passed to load_backend.

Traits

EmbedBackend

Trait for embedding backends.

Functions

detect_backends

Detect all available backends and load them.

load_backend

Construct an embedding backend of the given kind.