zer-lib 1.0.0

Zero-shot entity resolution pipeline for Dutch-centric data, with GPU acceleration and neural judging
Documentation

zer-lib

Zero-shot entity resolution pipeline for Dutch-centric administrative data, with GPU acceleration and neural NLI judging.

zer-lib is the top-level facade crate. It re-exports all sub-crates under a single dependency and exposes feature flags to opt into GPU backends and the neural judge.

Quick start

[dependencies]
# CPU-only, full pipeline (blocking + compare + scoring + clustering)
zer-lib = { version = "1.0", features = ["pipeline"] }

# With CUDA compute backend
zer-lib = { version = "1.0", features = ["pipeline", "cuda"] }

# With neural DeBERTa judge (CPU ORT)
zer-lib = { version = "1.0", features = ["pipeline", "judge_cpu"] }

Feature flags

Compute backends

Flag Description
cuda NVIDIA CUDA (SM 8.6+), requires CUDA Toolkit 13.1+ and nvcc
vulkan Vulkan 1.3 compute, requires slangc at build time
avx2 x86_64 AVX2 SIMD, no extra toolchain
cpu Scalar CPU fallback (always available without this flag too)

Pipeline

Flag Description
pipeline Enables Pipeline, Ingester, and async progress events

Neural judge (ORT execution providers)

Flag Description
judge_cpu CPU ORT execution provider
judge_cuda NVIDIA CUDA ORT execution provider
judge_tensorrt NVIDIA TensorRT EP (FP16, engine caching)
judge_rocm AMD ROCm ORT execution provider
judge_directml Windows DirectML ORT execution provider
judge_openvino Intel OpenVINO ORT execution provider

The judge flags are independent from the compute backend flags.

Models and datasets

Download models before using the judge:

bash scripts/download_models.sh
# or set the env var:
export ZER_MODEL_DIR=/path/to/your/models

Download datasets before running tests or examples:

bash scripts/generate_data.sh
# or set the env var:
export ZER_DATA_DIR=/path/to/your/data

Sub-crates

Crate Description
zer-core Core traits and types
zer-blocking Blocking strategies and inverted index
zer-compare Fellegi-Sunter comparison and EM scoring
zer-schema Schema inference and model registry
zer-cluster Connected-components clustering
zer-compute GPU-accelerated compute backend
zer-pipeline End-to-end pipeline orchestration
zer-judge ONNX neural NLI judge
zer-adapters Polars / Arrow data-frame adapters
zer-prof NVTX profiling annotations

License

Apache-2.0 · GitHub