revector

Declarative, versioned schema & config migrations for Qdrant — Alembic for vector collections.

revector brings ordered, reversible, database-tracked migrations to Qdrant — the piece that, unlike for relational databases, didn't exist yet. You write declarative YAML migrations, commit them next to your code, and apply or roll them back with a single static binary. No Python venv, no external state store.

Schema, not data. revector manages collection schema and config — collections, payload indexes, named vectors, aliases, and all tunable knobs. Moving points between instances is a solved problem (see qdrant/migration); that's explicitly out of scope. The one data operation revector does help with — re-embedding — is handled through an exec-hook.

Why

Qdrant collections drift. You tune hnsw_config, add a payload index, introduce a second named vector for a new model, flip quantization on. Today those changes live in ad-hoc scripts or a teammate's shell history. revector makes them:

Versioned — each change is a file with a revision and down_revision, forming an ordered chain (Alembic's model).
Tracked — applied revisions are recorded inside Qdrant itself, in a dedicated _revector_migrations collection. No external database.
Reversible (honestly) — downgrades are auto-derived where safe and refused loudly where they'd lose data, instead of pretending.
Idempotent & resumable — Qdrant has no transactional DDL and builds indexes asynchronously, so every step is safe to re-run after a failure.

Install

Once a release is cut, prebuilt binaries for Linux, macOS, and Windows are attached to each GitHub Release by cargo-dist, with installers:

# Shell (Linux/macOS) — downloads the right prebuilt binary
curl --proto '=https' --tlsv1.2 -LsSf \
  https://github.com/diegoglozano/revector/releases/latest/download/revector-installer.sh | sh

# Windows (PowerShell)
powershell -c "irm https://github.com/diegoglozano/revector/releases/latest/download/revector-installer.ps1 | iex"

Or build from source (requires Rust 1.82+):

cargo install --path .          # from a checkout
cargo build --release           # ./target/release/revector

Homebrew and crates.io are planned — see ROADMAP.md.

Quick start

# 1. Scaffold a config + migrations/ directory
revector init

# 2. Create your first migration
revector new "create products collection"
#  → migrations/1718480000_create_products_collection.yaml

# 3. Edit the file (see the format below), then apply
export REVECTOR_URL=http://localhost:6334
revector up

# 4. Inspect state
revector status

# 5. Roll back the last migration
revector down

Migration files

A migration is a YAML file with a revision id, a link to its parent, and up / optional down operation lists. Each operation names itself with an op: key.

revision: "0001_products"
down_revision: null            # null marks the base of the chain
description: create products collection

up:
  - op: create_collection
    name: products
    spec:
      vectors:
        "":                    # "" is the unnamed/default vector
          size: 768
          distance: Cosine
      hnsw_config:
        m: 16
        ef_construct: 128

  - op: create_payload_index
    collection: products
    field_name: category
    schema: keyword

# Optional. If omitted, revector auto-inverts the `up` ops in reverse order
# and refuses the downgrade if any step is irreversible.
down:
  - op: delete_collection
    name: products

Supported operations

`op:`	Effect	Auto-reversible?
`create_collection`	Create a collection from a full spec	✔ → `delete_collection`
`delete_collection`	Drop a collection	✘ (data loss)
`update_collection`	Patch `hnsw_config`, `quantization_config`, `optimizers_config`, or per-vector params in place	✘ (prior state unknown)
`create_vector`	Add a named dense vector (v1.18+)	✔ → `delete_vector`
`create_sparse_vector`	Add a named sparse vector	✔ → `delete_vector`
`delete_vector`	Drop a named vector	✘ (data loss)
`create_payload_index`	Index a payload field	✔ → `delete_payload_index`
`delete_payload_index`	Remove a payload index	✔ iff `schema:` is given
`create_alias`	Point an alias at a collection	✔ → `delete_alias`
`delete_alias`	Remove an alias	✘ (target unknown)
`switch_alias`	Atomically repoint an alias (zero-downtime swap)	✘ (prior target unknown)
`exec`	Run a shell command (the re-embedding escape hatch)	✘ unless `down` provided

When an operation isn't auto-reversible, supply an explicit down: block — then revector down uses it verbatim.

Commands

Command	Description
`revector init`	Create `migrations/` and a starter `revector.toml`.
`revector new <name>`	Scaffold a new migration chained onto the current head.
`revector status`	Show applied vs pending revisions, checksums, and reversibility.
`revector up [--to <rev>] [--dry-run]`	Apply pending migrations.
`revector down [--to <rev>] [--steps N] [--dry-run]`	Roll back migrations.
`revector to <rev> [--dry-run]`	Migrate to an exact revision (up or down).
`revector validate`	Parse all migrations and resolve the chain offline — no Qdrant connection. Good as a CI / pre-commit check.
`revector stamp <rev\|head\|base> [--dry-run]`	Mark the DB as being at a revision without running any ops — for adopting an existing collection (Alembic's `stamp`).
`revector diff <collection> --spec <file.yaml>`	Compare a declared collection spec against the live collection.

--dry-run prints the plan without touching Qdrant.

Safety

Confirmation. Rollbacks (down, and a to that moves backwards) prompt before proceeding. Pass -y / --yes to skip the prompt; in a non-interactive shell (CI) revector refuses rather than guessing, so --yes is required there.
Advisory lock. up / down / to / stamp take a lock record in the tracking collection for the duration of the run, so two concurrent runs (e.g. parallel CI jobs) don't stomp on each other. If a previous run died and left a stale lock, --force overrides it. (Best-effort — Qdrant has no compare-and-set — but it reliably catches the common case.)

Adopting an existing collection

If you already have a collection that matches an early migration, stamp lets revector take over without re-creating it:

# Tell revector the DB is already at 0002 (no operations run)
revector stamp 0002_index_and_quantize
# Now apply everything after it normally
revector up

Configuration

Settings are layered (highest precedence first): CLI flags → REVECTOR_* environment variables → revector.toml → defaults.

# revector.toml
url = "http://localhost:6334"
migrations_dir = "migrations"
# api_key = "..."                      # or REVECTOR_API_KEY
# tracking_collection = "_revector_migrations"

Setting	Env	Default
`url`	`REVECTOR_URL`	`http://localhost:6334`
`api_key`	`REVECTOR_API_KEY`	none
`migrations_dir`	`REVECTOR_MIGRATIONS_DIR`	`migrations`
`tracking_collection`	`REVECTOR_TRACKING_COLLECTION`	`_revector_migrations`

Set REVECTOR_LOG=revector=debug for verbose logging (or pass -v / -vv).

Drift detection (`diff`)

revector diff compares a declared collection spec against the live collection. It is declaration-driven: only fields you actually wrote in the spec are compared. A field you leave unset means "don't care", never "must be unset" — this avoids the classic Alembic-autogenerate false positives caused by Qdrant normalizing and defaulting config on read.

revector diff products --spec products.spec.yaml
# collection `products` has 1 difference(s):
#   vectors.<default>.size : declared 1024 | live 768

Re-embedding (the exec-hook)

Changing a vector's size or distance is structural — Qdrant can't mutate it in place. The path is: add a new named vector → re-embed points with your model → drop the old vector. The re-embedding step is the one thing a generic binary can't own, so revector shells out to your command:

up:
  - op: create_vector
    collection: products
    name: text_v2
    spec: { size: 1024, distance: Cosine }

  - op: exec
    name: re-embed with the new model
    command: "python scripts/reembed.py --collection products --target text_v2"

  - op: delete_vector          # irreversible — make this a separate, deliberate migration
    collection: products
    name: text_v1

The command runs via sh -c, inherits the environment and stdio, and a non-zero exit aborts the migration.

How state is tracked

Applied revisions are stored as points in the _revector_migrations collection (a dummy 1-d vector plus a payload of revision id, parent, checksum, and timestamp). Because the checksum of each migration file is recorded, revector refuses to proceed if a migration was edited after being applied — catching silent divergence between your files and the database.

Scope & limitations

Linear chains only (single base, single head) in v1 — branching/merging is rejected with a clear error.
Per-vector hnsw_config / quantization_config can't be set at create_vector time (Qdrant's add-vector API doesn't accept them); apply them with a follow-up update_collection step.
diff reads a standalone spec file; folding the full migration chain into a desired-state spec is future work.

Development

cargo test                       # unit + logic tests
cargo clippy --all-targets       # lints
cargo fmt                        # format

Integration tests spin up a real Qdrant via testcontainers (Docker required); they skip automatically when Docker is unavailable. To run them against an already-running Qdrant instead:

REVECTOR_TEST_URL=http://localhost:6334 cargo test --test integration

Security & supply chain

Dependency scanning. cargo-deny runs in CI (and weekly, to catch newly-published advisories) checking vulnerabilities, licenses, banned/duplicate crates, and that every dependency comes from crates.io. Config: deny.toml.
Automated updates. Dependabot opens weekly PRs for Rust deps and GitHub Actions, so security patches land promptly.
Build provenance. Release binaries carry SLSA build-provenance attestations generated by cargo-dist, so you can verify an artifact was built by this repo's CI:
```
gh attestation verify revector-x86_64-unknown-linux-gnu.tar.xz --repo diegoglozano/revector
```
Reproducibility. Cargo.lock is committed and cargo publish --locked is used, so published builds resolve to pinned versions.
Minimal runtime surface. The known advisories in the tree are confined to dev-dependencies (the test harness) and don't ship in the binary; see deny.toml for the tracked exceptions.

License

Licensed under either of MIT or Apache-2.0 at your option.

revector 0.2.0