iqdb 0.1.0

<h1 align="center">
    <img width="99" alt="Rust logo" src="https://raw.githubusercontent.com/jamesgober/rust-collection/72baabd71f00e14aa9184efcb16fa3deddda3a0a/assets/rust-logo.svg">
    <br>
    <strong>iQDB</strong>
    <br>
    <sup><sub>EMBEDDED VECTOR DATABASE FOR RUST</sub></sup>
</h1>

<p align="center">
    <a href="https://crates.io/crates/iqdb"><img alt="crates.io" src="https://img.shields.io/crates/v/iqdb.svg"></a>
    <a href="https://crates.io/crates/iqdb" alt="Download iqdb"><img alt="Crates.io Downloads" src="https://img.shields.io/crates/d/iqdb?color=%230099ff"></a>
    <a href="https://docs.rs/iqdb"><img alt="docs.rs" src="https://docs.rs/iqdb/badge.svg"></a>
    <img alt="MSRV" src="https://img.shields.io/badge/MSRV-1.75%2B-blue.svg?style=flat-square" title="Rust Version">
    <a href="https://github.com/jamesgober/iqdb/actions"><img alt="CI" src="https://github.com/jamesgober/iqdb/actions/workflows/ci.yml/badge.svg"></a>
</p>

<p align="center">
    FASTER - LIGHTER - SMARTER 
    <br>
    Intelligence-grade vector storage for AI-native applications.
    <br>
    Sub-millisecond vector search. Zero network hops. - i<b>QD</b>B
</p>

<br>

<div align="left">
    <p>
        <strong>iQDB</strong> is an <b>embedded vector database</b> for Rust, a single-process, in-application similarity-search engine designed for <em>high-dimensional</em> workloads where every microsecond on the query path matters.
        It targets the same operational shape as <code>sqlite</code> or <code>redb</code>: no daemon, no network hop, no separate runtime. Open a handle, write vectors, query nearest neighbours — all from inside your binary.
    </p>
    <p>
        The engine is built against a <b>lock-free hot path</b>, <b>allocation-free steady state</b>, and a <b>cache-aware</b> on-disk layout. Approximate-nearest-neighbour indices, exact brute-force search, payload metadata, and durable journaling are all in scope. Indices and storage are pluggable so workloads can trade recall for latency without rewriting the surrounding application.
    </p>
    <p>
        Built for <b>cross-platform</b> deployment from day one — Linux, macOS, and Windows are first-class targets, with <code>io_uring</code> on Linux and atomic file replacement everywhere else. The async surface is opt-in: synchronous embedded use stays free of executor coupling, while applications already running on Tokio can drive the same engine without bridging.
    </p>
    <br>
    <hr>
    <p>
        <strong>MSRV is 1.75+.</strong> The crate is dual-licensed under <code>Apache-2.0 OR MIT</code> at your option.
    </p>
    <blockquote>
        <strong>0.1.0 is the scaffolding milestone.</strong> The public handle (<code>Iqdb::open</code>, <code>open_in_memory</code>, <code>flush</code>, <code>close</code>) is in place and CI-green across Linux, macOS, and Windows. The query surface (<code>upsert</code>, <code>search</code>, <code>delete</code>) lands in subsequent milestones — every stub returns <a href="./src/error.rs"><code>Error::NotImplemented</code></a> until the engine is wired underneath it. The API is <b>unstable</b> until 1.0; see <a href="./CHANGELOG.md"><code>CHANGELOG.md</code></a> for the release-by-release surface.
    </blockquote>
</div>

<hr>
<br>

## Design Goals

iQDB is engineered against the <a href="./REPS.md">Rust Efficiency &amp; Performance Standards</a> (REPS). Every architectural decision is graded against the same hard constraints:

- **Embedded by default** — no daemon, no socket, no separate process. The library opens a path and gets out of the way.
- **Sub-millisecond queries** — exact and approximate search paths are budgeted in microseconds, not milliseconds, with the hot path measured under Criterion every release.
- **Lock-free hot paths** — atomic reads on the query path; writers do not contend with concurrent readers.
- **Allocation-free steady state** — buffers are pooled and reused. Per-query allocation is a design defect.
- **Cache-aware layout** — vectors land contiguous in memory and on disk; index nodes are padded to the cache line.
- **Pluggable indices** — flat, IVF, and HNSW share a common trait surface so callers can swap strategies without touching their query code.
- **Pluggable storage** — durable file-backed storage, transient in-memory storage, and the org-default `storage-io` substrate are all addressable behind the same handle.
- **Crash-safe writes** — the durable path uses write-ahead logging and atomic file replacement. A pulled power cord must never corrupt the index.
- **Tier-1 cross-platform** — Linux (x86_64, aarch64), macOS (x86_64, Apple Silicon), Windows (x86_64) all compile and pass the full test suite on every commit.

<hr>
<br>

## Status &amp; Roadmap

iQDB ships milestone-by-milestone. Each tag below corresponds to a published release; everything above the current line is shipped, everything below is planned.

| Milestone | Status | Surface delivered |
|-----------|--------|-------------------|
| `v0.1.0` — scaffolding | **current** | Crate scaffolding, lifecycle handle (`open` / `open_in_memory` / `flush` / `close`), `Error` type, integration test, criterion harness, CI matrix on all three Tier-1 platforms. |
| `v0.2.0` — vector primitives | planned | `Vector`, `DistanceMetric` (Cosine, L2, Dot), `Payload`, `RecordId`. Exact (flat) index. `upsert(id, vec)` / `get(id)` / `delete(id)` against an in-memory store. |
| `v0.3.0` — search | planned | `search(query, k)` over the flat index. Batch search. Filter-by-payload predicate. Result ranking with score + id. |
| `v0.4.0` — durable storage | planned | File-backed storage substrate. Write-ahead log, atomic-replace snapshots, crash recovery. `Iqdb::open(path)` becomes load-bearing. |
| `v0.5.0` — approximate indices | planned | IVF and HNSW indices behind the same trait the flat index implements. Build-time index selection via the builder. |
| `v0.6.0` — async surface | planned | `async`-feature-gated mirror of the public API. Driven by Tokio. Cancellation-safe. |
| `v0.7.0` — collections | planned | Named collections / namespaces with per-collection metric and dimensionality. Collection-scoped iteration. |
| `v1.0.0` — API freeze | planned | Frozen public API. SemVer guarantees. Full benchmark suite. Migration guide from 0.x. |

The per-release detail — what was added, what changed, and what was verified — lives in the [`CHANGELOG`](./CHANGELOG.md) and the per-version notes under [`docs/release/`](./docs/release/).

<hr>
<br>

## Installation

Once `0.1.0` is published, add to your `Cargo.toml`:

```toml
[dependencies]
iqdb = "0.1"
```

Pre-publish, depend on the repository directly:

```toml
[dependencies]
iqdb = { git = "https://github.com/jamesgober/iqdb", branch = "main" }
```

iQDB compiles on stable Rust **1.75** and newer. There are no required system libraries — only the Rust toolchain is needed for an out-of-the-box build.

<hr>
<br>

## Quick Start

The current `0.1.0` surface exposes the database lifecycle. Query verbs (`upsert`, `search`, `delete`) land in subsequent releases — see the roadmap above. The scaffolding compiles, lints clean, and passes the full test suite, but stubs return [`Error::NotImplemented`](./src/error.rs) until the engine lands behind them.

```rust
use iqdb::{Iqdb, Result};

fn main() -> Result<()> {
    // Open an ephemeral, in-memory instance. Never touches disk;
    // the entire dataset is dropped when the handle is dropped.
    let db = Iqdb::open_in_memory();

    // File-backed open lands in v0.4.0 — currently returns
    // Error::NotImplemented so call sites can be wired now and
    // light up automatically when the engine ships.
    // let db = Iqdb::open("./data/vectors.iqdb")?;

    // Release the handle. Cleanup happens here (sync, lock release,
    // file close — whatever the active backend needs).
    db.close()?;
    Ok(())
}
```

### Handling the staged surface

Because methods like `Iqdb::open` and `Iqdb::flush` are stubs until their respective milestones, downstream callers can safely wire them in advance and gate behaviour on the error variant:

```rust
use iqdb::{Error, Iqdb};

fn flush_if_supported(db: &Iqdb) -> Result<(), Error> {
    match db.flush() {
        Ok(()) => Ok(()),
        Err(Error::NotImplemented) => {
            // The active milestone does not yet support flushing.
            // In v0.4.0 (durable storage) this branch disappears.
            Ok(())
        }
        Err(err) => Err(err),
    }
}

let db = Iqdb::open_in_memory();
flush_if_supported(&db).unwrap();
```

<hr>
<br>

## API Overview

A dedicated `docs/API.md` reference lands alongside the query surface in a subsequent release. Until then, the rustdoc-generated API documentation at [docs.rs/iqdb](https://docs.rs/iqdb) is the canonical reference. The currently-stable items are:

- [`Iqdb`](./src/db.rs) — the top-level database handle. Owns the open backend and exposes the lifecycle verbs.
  - `Iqdb::open(path)` — open or create a file-backed database (planned for v0.4.0 — currently returns `Error::NotImplemented`).
  - `Iqdb::open_in_memory()` — open an ephemeral instance backed entirely by RAM. Never touches the filesystem.
  - `Iqdb::flush()` — flush pending writes to durable storage (planned for v0.4.0).
  - `Iqdb::close(self)` — close the handle and release all held resources.
- [`Error`](./src/error.rs) — the unified error type. `#[non_exhaustive]`; new variants are added as new failure modes appear. Always include a `_` arm when matching.
- [`Result<T>`](./src/error.rs) — alias for `core::result::Result<T, Error>`.

### Error variants

| Variant | Meaning | Recovery |
|---------|---------|----------|
| `Error::Io(std::io::Error)` | A lower-level I/O failure occurred — disk full, permission denied, missing path, etc. | Inspect the wrapped `ErrorKind`. Retry transient errors; surface permanent ones. |
| `Error::InvalidConfig(&'static str)` | Configuration supplied at open time was invalid (e.g., zero-length path, unsupported metric). | Programmer error — fix the construction site. |
| `Error::NotImplemented` | The requested operation belongs to a later milestone and has no engine behind it yet. | Either upgrade to a release where the verb is implemented, or fall back to an in-memory backend. |

The enum is `#[non_exhaustive]`. New variants will appear in minor releases as new failure modes emerge. Exhaustive `match` arms are a forward-compatibility hazard — always include `_`.

<hr>
<br>

## Examples

Self-contained examples live in [`examples/`](./examples) and are declared in `Cargo.toml`. Run them with `cargo run --example <name>`.

- **Lifecycle (`basic`)** — open an in-memory instance and close it cleanly.
  - File: [`examples/basic.rs`](./examples/basic.rs)
  - Run:
    ```sh
    cargo run --example basic --release
    ```

Additional examples land alongside their milestones — vector upsert + search in `v0.2.0` / `v0.3.0`, durable open in `v0.4.0`, approximate search in `v0.5.0`, async usage in `v0.6.0`.

<hr>
<br>

## Benchmarks

A criterion harness is wired in [`benches/scaffold.rs`](./benches/scaffold.rs). The current bench is a no-op placeholder — it exists so the harness is proven to compile and run on every CI build. Real workloads land alongside the milestones they exercise:

- **`v0.2.0`** — `upsert` throughput, `get` latency.
- **`v0.3.0`** — exact-search latency by `k` and dimensionality.
- **`v0.4.0`** — durable write throughput, recovery time.
- **`v0.5.0`** — IVF and HNSW build time, recall vs. latency.

Run the current scaffold to confirm the harness works on your machine:

```sh
cargo bench --bench scaffold
```

Criterion writes reports to `target/criterion/`. Compare runs over time to detect regressions; CI will gate merges on a regression threshold once the real benches land.

<hr>
<br>

## Testing

Coverage today is intentionally scoped to the stub surface:

- Unit tests live in `#[cfg(test)] mod tests` blocks inside each source file.
- Integration tests live in [`tests/`](./tests) — currently [`tests/smoke.rs`](./tests/smoke.rs) covers the lifecycle on the in-memory backend.
- Doc tests run as part of `cargo test` and validate every code block in the rustdoc.

```sh
# Full test sweep (unit + integration + doc tests)
cargo test

# Documentation build with no warnings (matches CI gating)
RUSTDOCFLAGS="-D warnings" cargo doc --no-deps

# Lint at the strict profile CI enforces
cargo clippy --all-targets -- -D warnings

# Formatting check (no diffs)
cargo fmt --all -- --check
```

As real engine code lands, every public path will be backed by happy / error / edge-case tests per the REPS testing mandate.

<hr>
<br>

## Cross-Platform Support

**Tier 1 targets** — every commit is built and tested on:

- Linux (`x86_64-unknown-linux-gnu`, `aarch64-unknown-linux-gnu`)
- macOS (`x86_64-apple-darwin`, `aarch64-apple-darwin`)
- Windows (`x86_64-pc-windows-msvc`)

**Platform-specific paths** — when the durable storage substrate lands, the platform-conditional code paths become observable:

- **Linux**: `io_uring` submission for batch writes; `fsync` for journal durability.
- **macOS**: `F_FULLFSYNC` for journal durability; `mmap` for read-mostly indices.
- **Windows**: `FlushFileBuffers` for journal durability; `MoveFileExW` with `MOVEFILE_REPLACE_EXISTING` for atomic snapshot replacement.

No platform is silently degraded — fallbacks are explicit and documented inline at each `#[cfg]` boundary, per the REPS *Platform-Specific Code* rule.

<hr>
<br>

## Configuration

### Feature flags

The default feature set at `0.1.0` is empty — the scaffolding has no optional code paths yet. Feature flags will be additive only (per REPS) as subsystems land:

| Feature       | Default | Planned for | Description                                            |
|---------------|---------|-------------|--------------------------------------------------------|
| *(none yet)*  |         |             | All current functionality is unconditional.            |
| `async`       | off     | v0.6.0      | Tokio-driven async mirror of the public API.           |
| `serde`       | off     | v0.2.0      | `Serialize` / `Deserialize` for `Vector` and payloads. |
| `mmap`        | off     | v0.4.0      | Memory-mapped read path for hot indices.               |
| `io-uring`    | off     | v0.4.0      | Linux-only `io_uring` submission for batch writes.     |
| `full`        | off     | post-1.0    | All stable features in one switch.                     |

```toml
# Future: pick a curated set
iqdb = { version = "0.1", features = ["serde"] }
```

### Runtime configuration

`Iqdb::open_in_memory()` takes no parameters by design. A builder (`IqdbBuilder`) is introduced in `v0.4.0` once the file-backed path lands and has more than two open-time knobs. Until then, the constructor surface is the entire configuration surface.

<hr>
<br>

## Architecture

The crate is split along strict module boundaries — each module owns one concern and exposes a single trait or type as its contract:

- `src/lib.rs` — crate root, lint profile, public re-exports.
- `src/db.rs` — the `Iqdb` handle and lifecycle verbs.
- `src/error.rs` — the `Error` enum and `Result` alias.

As milestones land, the tree grows along bounded responsibilities (`vector/`, `index/`, `storage/`, `journal/`, `query/`, `async_impl.rs`). The boundary between layers is always a trait — concrete implementations are crate-internal and gated behind `pub(crate)`.

### Compile-time guarantees

The crate root enables the strict REPS lint profile in [`src/lib.rs`](./src/lib.rs):

```text
#![deny(warnings)]
#![deny(missing_docs)]
#![deny(unsafe_op_in_unsafe_fn)]
#![deny(unused_must_use)]
#![deny(unused_results)]
#![deny(clippy::unwrap_used)]
#![deny(clippy::expect_used)]
#![deny(clippy::todo)]
#![deny(clippy::unimplemented)]
#![deny(clippy::print_stdout)]
#![deny(clippy::print_stderr)]
#![deny(clippy::dbg_macro)]
#![deny(clippy::unreachable)]
#![deny(clippy::undocumented_unsafe_blocks)]
```

Test modules locally relax the `unwrap_used` / `expect_used` lints — the strict profile is for production library code, not assertion scaffolding inside `#[cfg(test)]` blocks.

<hr>
<br>

## Contributing

Pull requests are welcome. Before opening one, please make sure the full CI gate passes locally:

```sh
cargo fmt --all -- --check
cargo clippy --all-targets -- -D warnings
cargo test
RUSTDOCFLAGS="-D warnings" cargo doc --no-deps
cargo deny check
cargo audit
```

Every contribution is expected to honour the standards in [`REPS.md`](./REPS.md) — performance, security, error handling, testing, documentation, and dependency hygiene are all enforced as merge gates, not afterthoughts. Commit messages are imperative, lowercase, and scoped to a single logical change.

<hr>
<br>

<div align="center">
    <sup>
        <span>HOME</span>
        <span>&nbsp;│&nbsp;</span>
        <a href="https://github.com/jamesgober/iqdb/blob/main/CHANGELOG.md" title="Changelog"><b>CHANGELOG</b></a>
        <span>&nbsp;│&nbsp;</span>
        <a href="https://github.com/jamesgober/iqdb/blob/main/REPS.md" title="Standards"><b>STANDARDS</b></a>
        <span>&nbsp;│&nbsp;</span>
        <a href="https://docs.rs/iqdb" title="API Reference"><b>DOCS</b></a>
    </sup>
</div>
<br>

## Links

- [Documentation (docs.rs)](https://docs.rs/iqdb)
- [Crates.io](https://crates.io/crates/iqdb)
- [Repository](https://github.com/jamesgober/iqdb)
- [Issues](https://github.com/jamesgober/iqdb/issues)
- [Changelog](./CHANGELOG.md)
- [Standards (REPS)](./REPS.md)

<br>

<hr>
<br>

<!-- LICENSE
############################################ -->
<div id="license">
    <h2>LICENSE</h2>
    <p>Licensed under either of</p>
    <b>Apache License, Version 2.0</b>: <a href="./LICENSE-APACHE">LICENSE-APACHE</a>
      &mdash; <a href="http://www.apache.org/licenses/LICENSE-2.0" target="_blank">http://www.apache.org/licenses/LICENSE-2.0</a>
    <br><br>
    <b>MIT License</b>: <a href="./LICENSE-MIT">LICENSE-MIT</a> &mdash;
    <a href="http://opensource.org/licenses/MIT" target="_blank">http://opensource.org/licenses/MIT</a>
    <br><br>
    <p>At your option. Unless you explicitly state otherwise, any contribution intentionally submitted for inclusion in the work by you, as defined in the Apache-2.0 license, shall be dual licensed as above, without any additional terms or conditions.</p>
</div>


<!-- COPYRIGHT
------------------------------>
<div align="center">
    <h2></h2>
    Copyright &copy; 2026 James Gober.
</div>