Skip to main content

DEFAULT_EMBEDDING_DIM

Constant DEFAULT_EMBEDDING_DIM 

Source
pub const DEFAULT_EMBEDDING_DIM: usize = 64;
Expand description

Default embedding vector dimensionality (v1.0.79, G42/S1).

Lowered from 384 to 64: with the LLM-only backend (v1.0.76+) each float costs ~8 autoregressive output tokens, so 384 dims ≈ 3072 tokens per vector at 50-100 tokens/s (30-60s per vector). 64 dims retain 90%+ retrieval quality for corpora under 100k memories (Matryoshka Representation Learning, arXiv 2205.13147) while cutting generation time ~6x. The historical 384 value matched multilingual-e5-small.