rightkit-embed 0.2.2

Product-neutral local text embeddings and persistent cosine vector store.
Documentation

rightkit-embed

Local text embeddings and a durable SQLite vector store for Right Suite apps.

Embedder::load uses caller-supplied ONNX Runtime & model paths. It never downloads or silently switches models. EmbeddingGemma 300M Q4 is default, with Gemma document/query prompts, L2 normalisation, & Matryoshka prefix truncation. VectorStore stores little-endian f32 vectors in SQLite & ranks them by cosine similarity in Rust.

Licensed under MIT OR Apache-2.0.

Also (0.2.2)

  • True padded batching. embed_batch / embed_query_batch tokenize all texts, order them by length, and run max_batch rows per ONNX call (EmbedderConfig::with_max_batch(n), default 16, also capped at 16384 padded tokens per call). Rows are right-padded to the chunk maximum with a zero attention mask (token_type_ids is sent only if the model declares it), pooled per row honoring the mask, then normalized exactly like single runs. Output order equals input order. Batched and single-text results agree to cosine >= 0.9999 and max element difference <= 2e-3 (Q4 kernels accumulate differently per shape); the pooling/padding math is unit-tested to 1e-6.
  • EmbedderConfig::with_intra_threads(n) sets session intra-op threads (default 1, unchanged for existing callers).
  • EmbedderConfig::with_provider(ExecutionProvider): Cpu (default), Auto (DirectML on Windows, CoreML on macOS, CPU elsewhere), or an explicit DirectMl / CoreMl that errors instead of falling back. Auto registers the accelerator through rightkit-ort, runs a probe inference, and falls back to CPU when registration, model load or the probe fails. Embedder::provider_used() returns "cpu", "directml" or "coreml"; Embedder::fallback_reason() says why an accelerator was dropped. The accelerator support is already compiled by rightkit-ort per target OS, so there is no extra crate feature; it needs an ONNX Runtime build that includes the provider (the runtime path you pass). On accelerators, shapes are bucketed (length to 64/128/.../2048, rows to a power of two) to avoid per-shape recompiles. ExecutionProvider is now defined by this crate (adds Auto); rightkit_ort::ExecutionProvider values still convert.
  • Embedder::pipeline_id() returns rkpipe1-<32 hex>, a stable fingerprint to decide when stored vectors need re-embedding. It covers the pipeline schema version, model id and native dimension, SHA-256 of model_q4.onnx, model_q4.onnx_data, tokenizer.json and config.json, the document and query prompt templates, pooling and normalization descriptions, truncate dimension, max tokens, and Gemma 2 sentinel scrubbing. Provider, threads and batch size are excluded. The first call hashes the model files; the result is cached.