Struct ModelConfig 

pub struct ModelConfig {
    pub model: EmbeddingModel,
    pub cache_dir: Option<String>,
    pub max_batch_size: usize,
    pub use_gpu: bool,
    pub num_threads: Option<usize>,
    pub session_pool_size: usize,
    pub backend_override: Option<BackendKind>,
}

Configuration for model loading and inference.

Fields

model: EmbeddingModel

The embedding model to use.

cache_dir: Option<String>

Custom cache directory for model files. If None, uses HuggingFace default cache.

max_batch_size: usize

Maximum batch size for inference.

use_gpu: bool

Whether to use GPU acceleration if available.

num_threads: Option<usize>

Number of threads for CPU inference.

session_pool_size: usize

Number of parallel ONNX sessions in the session pool.

Each session holds its own ORT context. Pool members serve batches concurrently via spawn_blocking, eliminating Mutex head-of-line blocking when multiple callers embed text simultaneously. Defaults to 4; override with DAKERA_ONNX_POOL_SIZE env var at startup.

backend_override: Option<BackendKind>

Force a specific backend kind, bypassing DAKERA_BACKEND env var. Used by TieredEngine to build the fast (static) backend from the same config.

Implementations

impl ModelConfig

pub fn new(model: EmbeddingModel) -> Self

Create a new config with the specified model.

pub fn with_cache_dir(self, dir: impl Into<String>) -> Self

Set the cache directory.

pub fn with_max_batch_size(self, size: usize) -> Self

Set the maximum batch size.

pub fn with_gpu(self, use_gpu: bool) -> Self

Enable GPU acceleration.

pub fn with_num_threads(self, threads: usize) -> Self

Set the number of CPU threads.

pub fn with_session_pool_size(self, size: usize) -> Self

Set the number of parallel ONNX sessions in the pool.

Trait Implementations

impl Clone for ModelConfig

fn clone(&self) -> ModelConfig

Returns a duplicate of the value.

fn clone_from(&mut self, source: &Self)

Performs copy-assignment from source.

impl Debug for ModelConfig

fn fmt(&self, f: &mut Formatter<'_>) -> Result

Formats the value using the given formatter.

impl Default for ModelConfig

fn default() -> Self

Returns the “default value” for a type.

impl<'de> Deserialize<'de> for ModelConfig

fn deserialize<__D>(__deserializer: __D) -> Result<Self, __D::Error>

where __D: Deserializer<'de>,

Deserialize this value from the given Serde deserializer.

impl Serialize for ModelConfig

fn serialize<__S>(&self, __serializer: __S) -> Result<__S::Ok, __S::Error>

where __S: Serializer,

Serialize this value into the given Serde serializer.