iridium-db 0.2.0

A high-performance vector-graph hybrid storage and indexing engine
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100

# Iridium Performance Release Notes

Release date: 2026-03-01  
Commit: `3029fdc`  

## Summary

This release captures optimization and performance hardening for the single-node engine, with measured validation across planning quality, hot-path runtime behavior, percentile observability, and mixed-workload reliability.

## Highlights

- Planner maturity:
  - Introduced planner stats lifecycle/versioning and freshness-based mode selection.
  - Added deterministic fallback to heuristic planning for stale/missing stats.
- Runtime hot paths:
  - Added batched cosine scoring path with Zig hook integration.
  - Preserved correctness parity through dedicated batch-vs-scalar tests.
  - Exposed runtime tuning knobs for query execution:
    - `IR_QUERY_MORSEL_SIZE`
    - `IR_QUERY_PARALLEL_WORKERS`
    - `IR_QUERY_SCAN_LIMIT_MULTIPLIER`
    - `IR_QUERY_SCAN_MIN`
- Observability and gating:
  - Replaced proxy latency behavior with true p95/p99 estimation.
  - Exposed p99 in CLI and Prometheus outputs.
  - Added SLO gate and release gate automation scripts.
- Mixed-workload reliability:
  - Added matrix, soak/failure-injection, concurrent stress, and tuning sweep harnesses.
  - Established reproducible artifact set for go/no-go decisions.

## Validation Snapshot

Source: `artifacts/release_gate_report.json`

- `alpha_ready`: `true`
- `beta_ready`: `true`
- `overall_pass`: `true`
- `soak_p95_micros`: `2469`
- `soak_p99_micros`: `2499`
- `concurrent_p95_micros`: `2182`
- `concurrent_p99_micros`: `2240`
- `concurrent_ingest_p50_eps`: `75276.01`
- `mixed_burst_ingest_eps`: `282852.09`
- `tuning_best_config`: `default`
- `tuning_best_burst_query_p99`: `2006`

## Recommended Runtime Baseline

- `IR_QUERY_MORSEL_SIZE=256`
- `IR_QUERY_PARALLEL_WORKERS=0`
- `IR_QUERY_SCAN_LIMIT_MULTIPLIER=64`
- `IR_QUERY_SCAN_MIN=512`

## Reproducibility

Run in order from repo root:

1. `bash scripts/mixed_workload_matrix.sh`
2. `bash scripts/soak_failure_check.sh`
3. `bash scripts/concurrent_stress.sh`
4. `bash scripts/tuning_sweep.sh`
5. `bash scripts/release_gate.sh`

References:

- `docs/performance/performance_runbook.md`
- `docs/performance/closeout.md`
- `docs/performance/ship_checklist.md`

## Known Risks

- Current harnesses are local single-node runs; production rollout still requires target-hardware validation.
- Release-gate parser logic depends on current JSON schema shape in generated artifacts.

## Changelog

- Added:
  - Planner stats lifecycle APIs and CBO/heuristic mode reporting in explain output.
  - Batched cosine runtime path with optional Zig kernel hook.
  - True p95/p99 metrics in CLI and Prometheus export.
  - Validation scripts:
    - `slo_gate.sh`
    - `mixed_workload_matrix.sh`
    - `soak_failure_check.sh`
    - `concurrent_stress.sh`
    - `tuning_sweep.sh`
    - `release_gate.sh`
  - Performance docs:
    - `docs/performance/performance_runbook.md`
    - `docs/performance/closeout.md`
    - `docs/performance/ship_checklist.md`
    - `docs/performance/release_notes_template.md`
- Changed:
  - Query execution defaults are now overrideable through env-driven runtime knobs.
  - Performance/testing guidance in README now includes the full performance command flow.
- Fixed:
  - Mixed-workload report generation now isolates SLO gate output from captured profile rows.
  - Mixed-workload SLO input metrics now reflect observed query runs rather than process-local counters.
- Deprecated/Removed:
  - None.