qql-edge 0.5.0

Embedded QQL executor backed by qdrant-edge and optional local FastEmbed inference
Documentation

qql-edge

Zero-network QQL executor: qdrant-edge (in-process HNSW) + QQL runtime.

Proposition

Same language and plan path as remote Qdrant, without a server. Opt-in FastEmbed slots for dense / sparse / multi / image / cross-encoder. Cluster features fail with stable QQL-EDGE-UNSUPPORTED-* codes (not silent no-ops).

Three embedding strategies produce an [Executor] backed by [EdgeQdrant]:

Constructor Embedder Use case
local_executor() FastEmbedder (ONNX, default BGE small 384-d) Fully offline dense
local_executor_with_options() Dense + optional sparse (sparse_model), multi (multi_model), image (image_model), reranker (reranker_model) Offline dense + sparse + ColBERT + CLIP + cross-encoder
http_executor() / http_executor_with_multi() HttpEmbedder (OpenAI-compatible) Remote dense / multi APIs
custom_executor() Any Arc<dyn Embedder> GPU, ensemble, caching
list_embedding_models() — Dense + sparse (SPLADE) + multi (BGE-M3) + image (CLIP vision) catalog

FastEmbed roles (do not conflate)

FastEmbed QQL
TextEmbedding (BGE, MiniLM, CLIP text, …) Dense (TEXT)
SparseTextEmbedding (SPLADE, BGE-M3 sparse) Sparse (USING SPARSE MODEL '…') — real ONNX inference when sparse_model is set
ImageEmbedding (CLIP vision, …) Dense (IMAGE / image_model)
Bgem3Embedding (joint dense + sparse + ColBERT) Multi (MultiDense via multi_model)
TextRerank (bge-reranker, …) CROSS RERANK pair scorer (client-side; not late-interaction RERANK)

CLIP is dual-encoder dense. Multivector is ColBERT bags only. Late-interaction RERANK uses multi; cross-encoder uses CROSS RERANK + reranker_model. Sparse defaults to local wire-compatible BM25 (Qdrant qdrant/bm25-identical token IDs via qdrant-edge); set sparse_model for real SPLADE/BGE-M3 sparse inference.

Tuning the local BM25 encoder

LocalExecutorOptions { bm25_k1, bm25_b, bm25_avg_len, bm25_language, bm25_tokenizer, … } (and the FastEmbedderOptions equivalents) tune the document-side local BM25 encoder used when no sparse_model is configured. Defaults stay 1.2 / 0.75 / 256 (Qdrant qdrant/bm25); language (30 stopword lists, 17 stemmers), tokenizer (word/whitespace/prefix/ multilingual), folding, and token length limits forward to Qdrant's own pipeline. It is a client-side, write-path-only knob: query weights stay unit, server-side inference is untouched, and vectors already written keep their weights — re-ingest to apply. Invalid values fail closed with QQL-VALIDATION-CONFIG. Same knobs on HttpEmbedderOptions for the edge HTTP executor (dense via HTTP, sparse still local).

// Title-heavy corpus: average title is ~8 tokens, not 256.
let mut exec = local_executor_with_options(
    "/tmp/qql-titles",
    LocalExecutorOptions {
        bm25_avg_len: Some(8.0),
        ..Default::default()
    },
)?;
// CREATE COLLECTION titles (sparse SPARSE);
// UPSERT INTO titles VALUES {id: 1, text: 'a short title'};  // uses avg_len=8
// Offline CLIP text + vision
let mut clip = local_executor_with_options(
    "/tmp/qql-clip",
    LocalExecutorOptions {
        model: Some("ClipVitB32".into()),           // Qdrant/clip-ViT-B-32-text
        image_model: Some("clip-vision".into()),    // Qdrant/clip-ViT-B-32-vision
        ..Default::default()
    },
)?;
// CREATE COLLECTION products (image VECTOR(512, COSINE));
// UPSERT … USING IMAGE MODEL 'clip-vision' ON FIELD image INTO image;
// QUERY TEXT 'red shoe' FROM products USING image LIMIT 10;
// QUERY IMAGE '/query.jpg' FROM products USING image LIMIT 10;

Quick start

use qql_edge::{local_executor, local_executor_with_options, LocalExecutorOptions};
use qql::executor::OnError;

// Dense-only offline
let mut executor = local_executor("/tmp/qql-edge-data", false)?;
let resp = executor.execute("CREATE COLLECTION docs HYBRID", OnError::Stop).await?;
let resp = executor.execute("UPSERT INTO docs VALUES {id: 1, text: 'hello world'}", OnError::Stop).await?;
let resp = executor.execute("QUERY 'hello' FROM docs LIMIT 5;", OnError::Stop).await?;

// Dense + offline ColBERT multi + cross-encoder (downloads on first use)
let mut multi_exec = local_executor_with_options(
    "/tmp/qql-edge-multi",
    LocalExecutorOptions {
        multi_model: Some("bge-m3".into()),
        reranker_model: Some("bge-reranker-base".into()),
        ..Default::default()
    },
)?;
// CREATE … colbert VECTOR(1024, COSINE) WITH MULTIVECTOR (comparator = 'max_sim')
// QUERY … USING colbert / QUERY RERANK … USING colbert PREFETCH (…)
// QUERY CROSS RERANK TEXT 'q' MODEL 'bge-reranker-base' ON FIELD text PREFETCH (c) …

executor.close().await?; // flush before deleting the data directory

EdgeQdrant backend

[EdgeQdrant] implements QdrantOps (the unified backend trait from qql-runtime) using qdrant-edge's in-memory HNSW index. Collection data persists to disk at the configured base_path.

Storage options

  • WAL segment capacity: qdrant-edge pre-allocates each WAL segment to 32 MiB by default, which dominates the on-disk footprint of small shards. Set LocalExecutorOptions::wal_segment_capacity (bytes) or EdgeQdrant::with_wal_segment_capacity(Some(bytes)) to shrink it; the resolved value persists in edge_config.json, so later loads without the knob keep it. The Python (local_executor(..., wal_segment_mb=N)) and Node (localExecutor(dir, { walSegmentMb: N })) edge SDKs expose the same knob in whole MiB; qdrant-edge 0.8's own Python binding cannot set it.
  • Snapshot seeding: qql_edge::{unpack_snapshot, inspect_shard} wrap the engine's snapshot API (unpack into a directory, then load and count) so callers never unpack or merge archives by hand.
  • Per-vector config: CREATE COLLECTION passes through per-vector on_disk / datatype / quantization / HNSW and sparse index options; the 1.19 memory tiers map onto the engine's RAM/mmap switch (pinned → RAM, cached/cold → mmap).

Supported operations

The supported point, mutation, collection, index, query, and batch operations use the same response envelope as REST and gRPC: { "result": ..., "status": "ok", "time": 0.0 }. Unsupported clustered or server-only operations return an explicit QqlError; they are never acknowledged as no-ops.

Response normalization

Mutations return:

{ "result": { "status": "completed" }, "status": "ok", "time": 0.0 }

Queries return normalized hit arrays with id, payload, and vector keys. Batch operations return arrays under "result" — cardinality is verified against the operation count.

Features

  • fastembed-local: ONNX-based local embedding via fastembed-rs (default)
  • http-embedding: HTTP-based embedding via reqwest (for http_executor)

When neither feature is enabled, only custom_executor() is available.

fastembed-local supports Linux x86-64, Windows x86-64, and Apple Silicon macOS. ONNX Runtime does not publish the required macOS Intel artifact, so Intel Mac users should disable default features and use http-embedding or custom_executor().

Boundaries

  • Built on qdrant-edge 0.8 (retrieve API, optional score_threshold, IDF on search params)
  • No UPDATE ... SET VECTOR via batch — uses individual route dispatch
  • gRPC is not available in edge mode (no protobuf dependency)
  • Edge uses qdrant-edge's native query engine for nearest, sparse, hybrid, MMR (dense), recommendation (best_score/sum_scores), context, discover, sample, formula, relevance-feedback, and order-by queries; point-reference and text inputs that cannot be embedded locally are rejected
  • Model-based sparse inference, multivector, and CROSS RERANK require the matching models to be opted in
  • Sparse (sparse_model) defaults to local wire-compatible BM25 (Qdrant qdrant/bm25-identical token IDs via qdrant-edge); opt in with sparse_model: Some("splade".into()) for real ONNX sparse inference. The local BM25 document encoder is tunable via bm25_k1 / bm25_b / bm25_avg_len (client-side, write-path only; invalid values fail closed with QQL-VALIDATION-CONFIG)
  • IMAGE expects local filesystem paths (no remote URL fetch)
  • Query/update “batch” is fan-out, not a single native batch RPC
  • Route affinity is a remote-client transport feature (RestQdrant / GrpcQdrant and the pyqql / nqql / wasm client options); it does not apply to in-process edge

Qdrant 1.19 language surface on edge

Feature Edge
PARAMS (idf = 'global' | WHERE <filter>) Supported (qdrant-edge 0.8)
PARAMS (acorn = …, max_selectivity = …) Supported (qdrant-edge 0.8 wires ACORN into HNSW search)
QUERY … GROUP BY field [SIZE n] [LIMIT/OFFSET] Supported (qdrant-edge grouping driver; hits hydrated per output selector)
ALTER COLLECTION … WITH HNSW / WITH OPTIMIZERS / WITH VECTOR <name> (HNSW (…)) Supported (persisted via set_hnsw_config / set_optimizers_config / set_vector_hnsw_config; the per-vector diff merges over the vector's effective config and pins the merged block)
CREATE COLLECTION … WITH PARAMS (on_disk_payload = …) Supported (persisted on the shard config)
Per-vector WITH VECTOR / WITH HNSW / WITH QUANTIZATION / WITH SPARSE Supported — lowered onto EdgeVectorParams / EdgeSparseVectorParams; sparse storage is always mmap, its on_disk selects the index placement
memory = 'pinned'|'cached'|'cold' on vectors/sparse Mapped to the engine's RAM/mmap switch (pinned → RAM, cached/cold → mmap; qdrant-edge 0.8 has no memory tiers)
datatype on dense/sparse Supported (float32/float16/uint8; turbo4 is dense-only and fails closed on sparse)
Background optimization None — call optimize_collection / qql edge optimize; prevent_unoptimized hides deferred points until then
WHERE field MATCH PREFIX '…' / WHERE SLICE (total, index) Supported when the offline filter converter accepts them
SHOW QUOTAS / SET QUOTA Unsupported — cluster REST /quotas only → QQL-EDGE-UNSUPPORTED-QUOTA
SHARD / GROUP BY … LOOKUP FROM / timeout / consistency Still unsupported (table below)
-- Sparse IDF corpus works offline (edge 0.8+)
QUERY TEXT 'search' FROM docs USING sparse
  PARAMS (idf = 'global')
  LIMIT 10;

-- Grouped search with per-group size and client-side group offset
QUERY TEXT 'search' FROM docs USING dense
  GROUP BY district SIZE 3
  LIMIT 10 OFFSET 5;

-- Collection tuning the engine persists
ALTER COLLECTION docs WITH HNSW (m = 32)
  WITH OPTIMIZERS (indexing_threshold = 500);

-- Per-vector HNSW diff (other per-vector fields fail closed)
ALTER COLLECTION docs WITH VECTOR dense (HNSW (ef_construct = 200));

-- Quotas always fail-loud offline
SHOW QUOTAS;  -- QQL-EDGE-UNSUPPORTED-QUOTA

Storage & Concurrency Contract

Each data_dir storage directory requires exclusive single-process access for reading and writing segments and payload files. Attempting to initialize multiple EdgeQdrant instances pointing to the same data_dir from separate processes concurrently is not supported.

Unsupported product surface (stable codes)

Offline rejects use a fixed catalog (backend/unsupported.rs). Messages include why and use remote Qdrant when applicable:

Code Feature
QQL-EDGE-UNSUPPORTED-GROUP-LOOKUP GROUP BY … LOOKUP FROM (no lookup collection)
QQL-EDGE-UNSUPPORTED-SHARD SHARD routing or collection sharding options
QQL-EDGE-UNSUPPORTED-SHARD-KEY CREATE/DROP SHARD KEY
QQL-EDGE-UNSUPPORTED-ALTER-PARAMS ALTER COLLECTION … WITH PARAMS
QQL-EDGE-UNSUPPORTED-ALTER-QUANTIZATION ALTER COLLECTION … QUANTIZATION
QQL-EDGE-UNSUPPORTED-VECTOR-DIFF per-vector ALTER COLLECTION … WITH VECTOR <name> fields other than hnsw_config (quantization_config / on_disk / memory)
QQL-EDGE-UNSUPPORTED-SPARSE-DIFF per-sparse-vector ALTER COLLECTION … WITH SPARSE <name>
QQL-EDGE-UNSUPPORTED-COLLECTION-PARAMS create-time WITH PARAMS other than on_disk_payload
QQL-EDGE-UNSUPPORTED-OPTIMIZER-KEY OPTIMIZERS keys the engine excludes (memmap_threshold, flush_interval_sec, max_optimization_threads)
QQL-EDGE-UNSUPPORTED-TIMEOUT PARAMS (timeout = …)
QQL-EDGE-UNSUPPORTED-CONSISTENCY PARAMS (consistency = …)
QQL-EDGE-UNSUPPORTED-QUOTA SHOW QUOTAS / SET QUOTA (cluster REST /quotas only)
QQL-EDGE-UNSUPPORTED-RECOMMEND-STRATEGY RECOMMEND STRATEGY average_vector (use best_score / sum_scores)
QQL-EDGE-UNSUPPORTED-FORMULA-FUNCTION MAX / MIN / ACOSH formulas (no engine Expression variants)
QQL-EDGE-UNSUPPORTED-POINT-REF point-id query inputs without embedded vectors
QQL-EDGE-UNSUPPORTED-ROUTE unmapped REST projection

Operational errors (QQL-EDGE-SPAWN, filter convert, etc.) stay separate.

Verification

# Edge tests require the fastembed-local feature and ~1 GB model download
cargo test -p qql-edge --features fastembed-local -- --test-threads=2

# HTTP executor only
cargo test -p qql-edge --features http-embedding -- --test-threads=2