pocket-encoder

A bundled sentence transformer with native Rust CPU inference. No downloads, model cache, Python, service, API key or ONNX shared library at build or runtime.

let encoder = pocket_encoder::Encoder::new()?;
let vectors = encoder.embed(&["Send a document", "Transfer a file"])?;
let model = pocket_encoder::model_info();
assert_eq!(vectors[0].len(), model.dimensions);
# Ok::<(), pocket_encoder::Error>(())

Uses sentence-transformers/paraphrase-MiniLM-L3-v2, pinned at 4ca70771034acceecb2e72475f72050fcdde4ddc: three transformer layers, 17.4 million parameters, 384 coordinates, 128 WordPiece tokens including special tokens. Longer inputs are right-truncated. Chunk long documents first. Mean pooling is attention masked, then L2-normalized. No query/document prefixes. English retrieval is the intended use; multilingual quality is unmeasured.

Encoder::new() shares immutable weights through a process-wide OnceLock. embed returns one normalized vector per input in order, rejecting blank text, more than 4096 inputs, more than 64 KiB per input or 4 MiB total. It retains no input text. Calls encode inputs sequentially to bound activation memory.

Weights use symmetric 4-bit storage in groups of 32, with F16 scales and original 1-D parameters. Loading expands them to F32; inference uses Candle's Rust CPU kernels. This saves distribution size, not live parameter memory. Neural math does not call Python, a hosted model or a separately installed BLAS runtime.

Always persist the complete model_info() alongside indexed embeddings, then require exact equality before comparing vectors. Equal dimensions do not imply compatible coordinates. Quantized vectors differ from upstream F32 vectors; do not mix them. CPU/compiler differences may change low-order floating bits. Do not promise cross-platform byte-identical embeddings.

Optional exact search and compact storage:

cargo add pocket-encoder --features search
cargo run --release -p pocket-encoder --features search --example search

use pocket_encoder::Index;

let labels = ["backup", "delete"];
let index = Index::new(&[
    "backup: Save a recoverable snapshot of all project data.",
    "delete: Permanently remove a snapshot and its stored contents.",
])?;
let bytes = index.to_bytes(); // Save during development; ship with include_bytes!.
let restored = Index::from_bytes(&bytes)?;
for hit in restored.search("protect me from losing everything", 5)? {
    println!("{}: {}", labels[hit.index], hit.score);
}
# Ok::<(), pocket_encoder::Error>(())

Index stores up to 4096 vectors in one contiguous F32 allocation. It scans all rows, selects the requested best matches and sorts only those matches. Ties favor earlier rows; Neighbor::index addresses your original input order. Keep labels and source documents alongside the index. Scores are cosine similarities, not confidence probabilities. The model and retrieval quality are unchanged; exact search can still return an irrelevant first suggestion.

For existing embeddings, use Index::from_embeddings(model_info, vectors) and search_embedding(&model_info, &query_vector, limit). Both validate the complete coordinate-space identity, dimensions, finite values and unit norms. Loading from bytes performs the same checks without initializing the neural encoder. Empty corpora and zero result limits avoid inference; invalid questions and vectors are still rejected.

The version-1 format is PKTIDX01, followed by little-endian U32 metadata length and row count, JSON for the complete ModelInfo, then row-major little-endian F32 coordinates. Metadata is capped at 4 KiB. Truncated, oversized, incompatible or trailing data is rejected. Coordinates round-trip without quantization. No texts or queries are stored; this format does not authenticate artifacts or bind caller-owned labels to their vectors. The feature adds no dependencies or filesystem operations. Embedding-only builds omit the index API.

Assets include immutable source revision, SHA-256 checksums and preprocessing metadata. Reproduce them from the repository root with python3 scripts/prepare_neural.py. That is an explicit development operation, never a Cargo build step. Source: MIT. Model/tokenizer: Apache-2.0. See NOTICE and assets/MODEL-LICENSE.

The encoder retrieves similar text; it cannot authenticate documents, prove that a result answers a question, or turn retrieved instructions into authority.

pocket-encoder 0.1.1

pocket-encoder