# Holger guards your artifacts at rest
**Holger, immutable artifact repository in pure Rust!**
Capabilities:
Airgap: transfer from internet ,nexus, artifacty to immutable znippy archive.
Versioned: Atomic transactions
Extreme perfomance due to Apache Arrow and its zero copy, multicore
Zero depencies to OS , shell,python. 100% rust. Compiled depencies like C based decompressors rewritten in RUST!
Embeddable on os free devices
### Artifact retrieval throughput
Random-access retrieval from artifact archives — Holger in-process (znippy 0.7.4, 2 MB RSS) vs Sonatype Nexus 3 (v3.72, container, 32 cores, 16 GB heap, 400 Jetty threads).
Same machine, same 32 cores. Nexus container stopped before Holger phase for clean isolation.
| **Artifact retrieval** | 442,482 ops/s | **10,018,754 ops/s** | 89 ops/s | 730 ops/s | 590 ops/s |
> **Holger is 13,724× faster** than Nexus at peak (16 clients = Nexus's sweet spot).
> Single-threaded Holger (442K ops/s) outperforms 32-client Nexus (590 ops/s) by **750×**.
> Nexus *degrades* past 16 clients despite 400 Jetty threads and 16 GB heap.
> Holger uses **2 MB RSS** vs Nexus's **16 GB heap** — 8,000× less memory.
#### Raw archive retrieval — in-process `get_file` (znippy 0.7.4, 32-core)
Direct random-access extraction from a 50,000-artifact znippy archive — no HTTP, no
network. This is the path `cargo xtask` benchmarks and regression-gates.
| **Rust crates** | 442,482 ops/s | **10,018,754 ops/s** |
| **Java JARs** | 285,988 ops/s | 1,912,632 ops/s |
| **Python wheels** | 308,367 ops/s | 2,083,248 ops/s |
> Backed by znippy 0.7.4's N-worker positioned-I/O read path: each `get_file` is one
> `pread` from a shared fd plus either a zero-copy skip (pre-compressed `.crate`/`.jar`)
> or an openzl decompress into a reused buffer — fully parallel across cores, no shared
> reader thread.
## Architecture
```
┌─────────────────────────────────────────────────────────────────────────┐
│ Holger Ecosystem │
├─────────────────────────────────────────────────────────────────────────┤
│ │
│ ┌─────────────────┐ ┌──────────────────┐ ┌────────────────┐ │
│ │ holger-agent │────▶│ holger-server │────▶│ Storage │ │
│ │ (CLI + daemon) │ │ (gRPC / Rust fn) │ │ (.znippy) │ │
│ │ │ │ │ │ │ │
│ │ --auth oidc │ │ OIDC validation │ │ Immutable │ │
│ └─────────────────┘ └──────────────────┘ │ Blake3 CAS │ │
│ │ │ └────────────────┘ │
│ │ │ │
│ ▼ ▼ │
│ ┌─────────────────┐ ┌──────────────────┐ │
│ │ Nexus/Registry │ │ cargo/mvn │ │
│ │ (connector) │ │ (clients) │ │
│ └─────────────────┘ └──────────────────┘ │
│ │
└─────────────────────────────────────────────────────────────────────────┘
```
## Access Modes
Holger exposes **two access modes** — no HTTP/REST:
### 1. Rust Function Bindings (embedded mode)
Use holger-server-lib directly in your Rust application:
```rust
use holger_server_lib::{read_ron_config, wire_holger};
use holger_traits::ArtifactId;
let mut holger = read_ron_config("holger.ron").unwrap();
holger.instantiate_backends().unwrap();
wire_holger(&mut holger).unwrap();
// Fetch artifact
let id = ArtifactId { namespace: None, name: "serde".into(), version: "1.0.219".into() };
let data = holger.fetch("rust-prod", &id).unwrap();
// List repositories
let repos = holger.list_repositories(); // ["rust-prod", "maven-staging"]
// Put artifact (write-enabled repos)
holger.put("rust-staging", &id, &bytes).unwrap();
```
### 2. gRPC (network mode)
Configure `exposed_endpoints` in RON → holger starts a tonic gRPC server automatically.
**Services** (defined in `proto/holger.proto`):
| `RepositoryService` | `FetchArtifact`, `ListArtifacts`, `PutArtifact`, `StreamArtifact` (server-streaming) |
| `ArchiveService` | `ListArchiveFiles`, `ArchiveInfo` |
| `AdminService` | `Health`, `ListRepositories` |
### 3. Apache Arrow Flight (batch mode) — planned
Arrow Flight as a **universal artifact service**: batch-fetch multiple artifacts (or files within artifacts) in a single call with zero-copy columnar transport.
| **Batch artifact fetch** | Request N artifacts in one Flight `DoGet` — server reads from znippy archive (already Arrow IPC), streams back as RecordBatches |
| **Files-within-artifact** | Fetch specific files from inside a .whl or .jar without downloading the whole archive (uses lzip/ljar filter) |
| **Index queries** | `GetFlightInfo` returns archive metadata (file list, sizes, checksums) as Arrow schema — zero parsing on client side |
| **Any-language client** | Python (pyarrow), Go, Java, Rust, C++ — all get native Flight clients for free |
Why Flight over raw gRPC for batch:
- znippy archives are already Arrow IPC → Flight serves them with near-zero serialization cost
- Columnar format means fetching 1000 small crates is one contiguous memory transfer, not 1000 RPCs
- `uv`/`pip`/`cargo` resolve+fetch could use a single Flight batch call instead of N HTTP GETs
**Connect with grpcurl:**
```bash
# Health check
grpcurl -plaintext localhost:50051 holger.AdminService/Health
# List repositories
grpcurl -plaintext localhost:50051 holger.AdminService/ListRepositories
# Fetch artifact
grpcurl -plaintext -d '{"repository":"rust-prod","name":"serde","version":"1.0.219"}' \
localhost:50051 holger.RepositoryService/FetchArtifact
# Stream large artifact (chunked 64KB)
grpcurl -plaintext -d '{"repository":"rust-prod","name":"tokio","version":"1.47.1"}' \
localhost:50051 holger.RepositoryService/StreamArtifact
```
## Crates
| `holger-server-lib` | Core server: RON config, wiring engine, gRPC services, auth module |
| `holger-server-cli` | Server binary (`holger-server start --config`) |
| `holger-agent-lib` | Agent logic: connectors, transfer operations |
| `holger-agent-cli` | Agent binary: airgap, push/pull between registries |
| `holger-traits` | Shared traits (`ConnectorTrait`, `RepositoryBackendTrait`) |
| `holger-nexus-connector` | Nexus/Artifactory connector (basic auth, OIDC) |
| `holger-world-connector` | crates.io + PyPI connector (reads Cargo.lock, requirements.txt, uv.lock) |
| `holger-rust-file-repository` | Rust crate repository backend (filesystem) |
| `holger-rust-znippy-repository` | Rust/Maven/Pip repository backend (znippy archive) |
| `holger-mannequin` | Dev daemon: mock PKI, OIDC, Nexus, egui GUI |
| `holger-distr` | Container/systemd distribution (Dockerfile, .service) |
## Configuration (RON)
Holger uses [RON](https://github.com/ron-rs/ron) for configuration — it can express the graph topology that TOML/YAML cannot.
### Rust crate repository (file-backed)
```ron
(
repositories: [
(
ron_name: "rust-prod",
ron_repo_type: "rust",
ron_upstreams: [],
ron_in: None,
ron_out: Some((
ron_storage_endpoint: "local-storage",
ron_exposed_endpoint: "grpc",
)),
),
],
exposed_endpoints: [
(
ron_name: "grpc",
ron_url: "0.0.0.0:50051",
),
],
storage_endpoints: [
(
ron_name: "local-storage",
ron_storage_type: "rocksdb",
ron_path: "/var/lib/holger/data",
),
],
)
```
### Rust crate repository (znippy archive)
```ron
(
repositories: [
(
ron_name: "rust-airgap",
ron_repo_type: "rust",
ron_archive_path: Some("/var/lib/holger/rust-airgap.znippy"),
ron_upstreams: [],
ron_in: None,
ron_out: Some((
ron_storage_endpoint: "znippy-storage",
ron_exposed_endpoint: "grpc",
)),
),
],
exposed_endpoints: [
(
ron_name: "grpc",
ron_url: "0.0.0.0:50051",
),
],
storage_endpoints: [
(
ron_name: "znippy-storage",
ron_storage_type: "znippy",
ron_path: "/var/lib/holger/rust-airgap",
),
],
)
```
### Python pip repository (znippy archive, PEP 503 simple index)
```ron
(
repositories: [
(
ron_name: "pip-internal",
ron_repo_type: "pip",
ron_archive_path: Some("/var/lib/holger/pip.znippy"),
ron_upstreams: [],
ron_in: None,
ron_out: Some((
ron_storage_endpoint: "pip-storage",
ron_exposed_endpoint: "grpc",
)),
),
],
exposed_endpoints: [
(
ron_name: "grpc",
ron_url: "0.0.0.0:50051",
),
],
storage_endpoints: [
(
ron_name: "pip-storage",
ron_storage_type: "znippy",
ron_path: "/var/lib/holger/pip",
),
],
)
```
The server exposes a PEP 503 simple index at `/{repo}/simple/` — point pip or uv directly at it:
```bash
pip install --index-url https://holger.internal/pip-internal/simple/ requests
uv pip install --index-url https://holger.internal/pip-internal/simple/ requests
```
### Maven repository (znippy archive)
```ron
(
repositories: [
(
ron_name: "maven-internal",
ron_repo_type: "maven3",
ron_archive_path: Some("/var/lib/holger/maven.znippy"),
ron_upstreams: [],
ron_in: None,
ron_out: Some((
ron_storage_endpoint: "maven-storage",
ron_exposed_endpoint: "grpc",
)),
),
],
exposed_endpoints: [
(
ron_name: "grpc",
ron_url: "0.0.0.0:50051",
),
],
storage_endpoints: [
(
ron_name: "maven-storage",
ron_storage_type: "znippy",
ron_path: "/var/lib/holger/maven",
),
],
)
```
## Authentication
| **OIDC** | Bearer token from token endpoint (Keycloak, AD, etc.) | Human users, CI/CD pipelines |
The agent CLI uses `--auth oidc` when talking to Holger server. Legacy Nexus connections still use `--username`/`--password` (basic auth via connector).
## Airgap Workflow
```
Internet side │ Airgapped side
│
crates.io ──┐ │
Maven ──────┤ │
PyPI ───────┤──▶ holger-agent │ holger-agent ──▶ holger-server
npm ────────┘ --to znippy │ --from znippy (gRPC / Rust fn)
│ │ ▲
▼ │ │
.znippy file ─┼───────┘
(USB/media) │
```
## Performance
Holger uses **znippy archives** (.znippy) as storage. A `.znippy` file is a valid **Arrow IPC File** — queryable directly by pyarrow/polars:
```python
# pyarrow
import pyarrow.ipc
reader = pyarrow.ipc.open_file("archive.znippy")
table = reader.read_all()
print(table.schema)
# relative_path: string, compressed: bool, uncompressed_size: uint64, chunks: list<...>
# polars
import polars as pl
df = pl.read_ipc("archive.znippy")
df.select("relative_path", "uncompressed_size", "compressed").head(10)
```
### Znippy archive benchmarks (v0.7.4 — io_uring + Gatling pipeline)
Packing source files into `.znippy` and extracting back. Files already compressed (.gz, .jar, .zip, etc.) are stored as-is (skipped). `.crate` files are gzipped tarballs — znippy compresses the *source contents* inside, not the .crate blob itself:
| text 500MB | 500 MB | 0.06 MB | 9,014× | 1,639 MB/s | 2,874 MB/s |
| single file 2GB | 2,048 MB | 0.22 MB | 9,509× | 5,802 MB/s | 3,066 MB/s |
| binary pattern 500MB | 500 MB | 0.07 MB | 7,338× | 2,242 MB/s | 2,976 MB/s |
| 100k small files (10KB) | 977 MB | 17.4 MB | 56× | 4,811 MB/s | 1,957 MB/s |
| random (incompressible) | 500 MB | 500 MB | 1.0× | 105 MB/s | 2,890 MB/s |
| mixed repo 530MB | 530 MB | 530 MB | 1.0× | 3,118 MB/s | 6,310 MB/s |
| **Real jars (4730 files, RAID)** | **5,124 MB** | — | — | **2,527 MB/s** | **9,950 MB/s** |
> **Note:** "Pack" = archiving + compression into `.znippy`. "Unpack" = decompression + extraction. Stream packer (in-memory/HTTP) numbers above; slot packer (io_uring disk reads) hits 678 MB/s on 10k×10KB synthetic. Already-compressed data (.jar, .crate) passes through at full I/O speed without re-compression.
### Host decompressors
The `host-decompressors` feature replaces miniz_oxide with custom parallel decompression:
| **linflate** | ~700 MB/s single-core | SIMD DEFLATE decoder (AVX2 match-copy) |
| **lgz** | 6,100+ MB/s multi-core | Parallel gzip decompression (full-flush segments) |
| **lgz** (speculative) | 527 MB/s | Two-pass pugz-style for arbitrary gzip streams |
| **ljar** | Multi-core | Parallel JAR/ZIP extraction (per-entry parallelism) |
| miniz_oxide (replaced) | ~190 MB/s | What most Rust projects use by default |
Single-core path is 3.7× faster than miniz_oxide. Multi-core scales linearly.
## GUI Builders
Holger has **no built-in web UI**. Instead, build your own GUI using either access mode:
### Option A: gRPC client (any language)
Use the proto definition (`proto/holger.proto`) to generate a client in any language.
**Getting OIDC tokens and TLS certs from mannequin (for GUI dev):**
Start mannequin first — it provides the complete auth infrastructure your GUI needs:
```bash
# 1. Start mannequin (mock PKI + OIDC + Nexus)
cargo run -p holger-mannequin -- serve --gui --port 9999
# 2. Get an OIDC token (accepts any credentials — it's a dev dummy)
TOKEN=$(curl -s -X POST http://127.0.0.1:9999/token \
-d 'grant_type=password&username=admin&password=anything&client_id=my-gui' \
| jq -r .access_token)
echo "Bearer token: $TOKEN"
# 3. Get a TLS client certificate (for mTLS to holger-server)
curl -s -X POST http://127.0.0.1:9999/pki/issue/client \
-H "Content-Type: application/json" \
-d '{"common_name": "my-gui-client"}' \
| jq -r '.cert_pem' > client.pem
curl -s -X POST http://127.0.0.1:9999/pki/issue/client \
-H "Content-Type: application/json" \
-d '{"common_name": "my-gui-client"}' \
| jq -r '.key_pem' > client-key.pem
# 4. Get the CA cert (to verify holger-server's TLS)
curl -s http://127.0.0.1:9999/pki/ca.pem > ca.pem
# 5. Connect to holger-server with OIDC auth
grpcurl -H "Authorization: Bearer $TOKEN" \
-cacert ca.pem -cert client.pem -key client-key.pem \
localhost:50051 holger.AdminService/ListRepositories
```
**In your GUI code (Rust + tonic):**
```rust
use tonic::transport::{Channel, ClientTlsConfig, Certificate, Identity};
use tonic::metadata::MetadataValue;
// Get token from mannequin OIDC endpoint
let token_resp: serde_json::Value = reqwest::Client::new()
.post("http://127.0.0.1:9999/token")
.form(&[("grant_type", "password"), ("username", "admin"),
("password", "dev"), ("client_id", "my-gui")])
.send().await?.json().await?;
let token = token_resp["access_token"].as_str().unwrap();
// Get CA + client cert from mannequin PKI
let ca_pem = reqwest::get("http://127.0.0.1:9999/pki/ca.pem").await?.bytes().await?;
let client_cert = reqwest::Client::new()
.post("http://127.0.0.1:9999/pki/issue/client")
.json(&serde_json::json!({"common_name": "my-gui"}))
.send().await?.json::<serde_json::Value>().await?;
// Build TLS channel
let tls = ClientTlsConfig::new()
.ca_certificate(Certificate::from_pem(&ca_pem))
.identity(Identity::from_pem(
client_cert["cert_pem"].as_str().unwrap(),
client_cert["key_pem"].as_str().unwrap(),
));
let channel = Channel::from_static("https://localhost:50051")
.tls_config(tls)?
.connect().await?;
// Attach OIDC bearer token to every request
let token_str: MetadataValue<_> = format!("Bearer {}", token).parse()?;
let mut admin = holger::admin_service_client::AdminServiceClient::with_interceptor(
channel.clone(),
move |mut req: tonic::Request<()>| {
req.metadata_mut().insert("authorization", token_str.clone());
Ok(req)
},
);
// Now use it
let repos = admin.list_repositories(holger::Empty {}).await?.into_inner();
```
**Plain examples (no auth, local dev):**
```rust
// Rust (tonic)
use tonic::transport::Channel;
let channel = Channel::from_static("http://[::1]:50051").connect().await?;
let mut admin = holger::admin_service_client::AdminServiceClient::new(channel.clone());
let mut repo = holger::repository_service_client::RepositoryServiceClient::new(channel);
// List repositories
let repos = admin.list_repositories(holger::Empty {}).await?.into_inner();
for name in &repos.names {
println!("{}", name);
}
// Fetch artifact
let artifact = repo.fetch_artifact(holger::FetchRequest {
repository: "rust-prod".into(),
name: "serde".into(),
version: "1.0.219".into(),
namespace: String::new(),
}).await?.into_inner();
// Stream large artifact (server sends 64KB chunks)
let mut stream = repo.stream_artifact(holger::FetchRequest {
repository: "rust-prod".into(),
name: "tokio".into(),
version: "1.47.1".into(),
namespace: String::new(),
}).await?.into_inner();
while let Some(chunk) = stream.message().await? {
// chunk.data: Vec<u8>, chunk.offset: u64
}
```
```python
# Python (grpcio)
import grpc
import holger_pb2, holger_pb2_grpc
channel = grpc.insecure_channel('localhost:50051')
admin = holger_pb2_grpc.AdminServiceStub(channel)
repo = holger_pb2_grpc.RepositoryServiceStub(channel)
# List repos
response = admin.ListRepositories(holger_pb2.Empty())
print(response.names)
# Fetch artifact
artifact = repo.FetchArtifact(holger_pb2.FetchRequest(
repository="rust-prod", name="serde", version="1.0.219"))
```
```go
// Go (google.golang.org/grpc)
conn, _ := grpc.Dial("localhost:50051", grpc.WithInsecure())
admin := pb.NewAdminServiceClient(conn)
repo := pb.NewRepositoryServiceClient(conn)
repos, _ := admin.ListRepositories(ctx, &pb.Empty{})
artifact, _ := repo.FetchArtifact(ctx, &pb.FetchRequest{
Repository: "rust-prod", Name: "serde", Version: "1.0.219"})
```
### Option B: Rust function bindings (same process, zero overhead)
For Rust-native GUIs (egui, iced, slint, dioxus), embed holger directly:
```rust
use holger_server_lib::{read_ron_config, wire_holger, Holger};
use holger_traits::ArtifactId;
// Initialize once at app startup
let mut holger = read_ron_config("holger.ron").unwrap();
holger.instantiate_backends().unwrap();
wire_holger(&mut holger).unwrap();
// Use from any thread — no serialization, no network
let repos = holger.list_repositories(); // Vec<&str>
let id = ArtifactId { namespace: None, name: "serde".into(), version: "1.0.219".into() };
let data: Option<Vec<u8>> = holger.fetch("rust-prod", &id).unwrap();
// Upload
holger.put("rust-staging", &id, &bytes).unwrap();
```
### gRPC services reference
| `AdminService` | `Health` | Server health + uptime |
| `AdminService` | `ListRepositories` | All configured repo names |
| `RepositoryService` | `FetchArtifact` | Download artifact (single response) |
| `RepositoryService` | `StreamArtifact` | Download artifact (64KB streaming chunks) |
| `RepositoryService` | `ListArtifacts` | List artifacts in a repository |
| `RepositoryService` | `PutArtifact` | Upload artifact |
| `ArchiveService` | `ListArchiveFiles` | List files in znippy archive |
| `ArchiveService` | `ArchiveInfo` | Archive metadata (size, entry count) |
## Development (holger-mannequin)
The `holger-mannequin` crate provides a full mock environment for development and integration testing:
```
┌─────────────────────────────────────────────┐
│ holger-mannequin │
├──────────┬──────────┬───────────┬───────────┤
│ PKI │ OIDC │ Nexus │ Auth │
│ │ │ │ │
│ CA mgmt │ /token │ proxy-only│ Users │
│ Issue │ /userinfo│ prod (4) │ Roles │
│ Regen │ Discovery│ upload │ Validate │
└──────────┴──────────┴───────────┴───────────┘
│
▼
egui GUI (--gui flag)
Live system map + request logs
```
**Subsystems:**
| **PKI** | Certificate Authority | Issue/revoke certs, CA chain |
| **OIDC** | Keycloak/Azure AD | `/token`, `/userinfo`, `/.well-known/openid-configuration` |
| **Nexus** | Sonatype Nexus | 3 repos: `proxy-only`, `prod` (4 seeded crates), `upload` |
| **Auth** | Identity provider | 4 seeded users: admin, reader, writer, ci-agent |
**Usage modes:**
```bash
# As a server (dev environment)
cargo run -p holger-mannequin -- serve --gui --port 9999
# In-process (integration tests — no network needed)
let mannequin = holger_mannequin::Mannequin::new();
let token = mannequin.oidc.issue_token("admin");
let cert = mannequin.pki.issue_client_cert("ci-agent");
# CLI (cert generation)
cargo run -p holger-mannequin -- pki generate --output ./certs
```
**egui GUI features:**
- Live system topology map (repositories ↔ storage ↔ endpoints)
- Request traffic visualization (gRPC calls in real-time)
- OIDC token issuer for quick dev testing
- Mock Nexus browser with seeded artifacts
## Quick Start
```bash
# Build everything
PROTOC=/path/to/protoc cargo build --workspace
# Run tests
PROTOC=/path/to/protoc cargo test --workspace
# Start mannequin (dev dummy with mock PKI, OIDC, Nexus + GUI)
cargo run -p holger-mannequin -- serve --gui
# Agent: download from crates.io → znippy archive
cargo run -p holger-agent -- --from world --to znippy --lockfile Cargo.lock --output ./airgap.znippy
# Agent: download Python packages (requirements.txt) → znippy archive
cargo run -p holger-agent -- --from world --lockfile-kind requirements-txt \
--lockfile requirements.txt --to znippy --output ./pip-airgap.znippy
# Agent: download Python packages (uv.lock, URLs already embedded) → znippy archive
cargo run -p holger-agent -- --from world --lockfile-kind uv-lock \
--lockfile uv.lock --to znippy --output ./pip-airgap.znippy
# Agent: push to Holger with OIDC auth
cargo run -p holger-agent -- \
--from znippy --to holger \
--input ./airgap.znippy \
--holger-url grpc://holger.internal:50051 \
--auth oidc --token-url https://keycloak/token --client-id ci-agent \
--repository rust-prod
# Start holger server (starts gRPC on configured port)
cargo run -p holger-server-cli -- start --config holger.ron
```
## Immutable Storage Model
All `.artifact` files are immutable. Holger can optionally be configured to allow **live ingest** of artifacts not found in the current archive (DEV mode only).
| V1 | Initial .artifact import | Immutable | Bootstrap, base snapshot |
| V2 | .artifact + live ingest | Appendable | DEV: allow dynamic additions |
| V3 | Promoted .artifact only | Immutable | PROD: strict airgapped enforcement |
## Status
- ✅ RON config + graph wiring engine
- ✅ gRPC server (tonic, 3 services)
- ✅ Rust function bindings API (embedded mode)
- ✅ Rust crate serving (file + znippy backends)
- ✅ Maven artifact serving (znippy backend)
- ✅ Python pip serving (znippy backend, PEP 503 simple index)
- ✅ Python pip airgap agent (requirements.txt + uv.lock → znippy)
- ✅ Airgap agent (world → znippy → nexus/holger)
- ✅ OIDC authentication
- ✅ Nexus connector (list/download/upload)
- ✅ Znippy host-decompressors (lgz, ljar, linflate)
- ✅ Container distribution (Dockerfile + systemd)
- ✅ Dev mannequin with egui system map GUI
- 🚧 Blake3 content-addressable verification
- 🚧 Authorization layer (role-based access on repos)
- 🚧 Go/npm format support
- 🚧 Live ingest proxy mode (DEV)
- 🚧 Promote workflow (DEV → PROD)
- 🚧 egui admin client for holger-server
- 🚧 Audit log (who accessed what, when)
## Testing
### Integration & Benchmark Tests
All heavy tests are `#[ignore]` — run explicitly with `-- --ignored --nocapture`.
| `test_znippy_archive_full_roundtrip` | holger-rust-znippy-repository | `cargo test --release -p holger-rust-znippy-repository --test integration_znippy_serve -- --ignored --nocapture` | Downloads ~5 GB crates from crates.io, packs into znippy, verifies byte-for-byte extraction |
| `test_all_types_download_pack_benchmark` | holger-rust-znippy-repository | (same as above) | Downloads crates+JARs+wheels (5 GB each), packs all 3, verifies, runs multi-threaded retrieval bombardment |
| `bench_znippy_throughput` | holger-rust-znippy-repository | `cargo test --release -p holger-rust-znippy-repository --test bench_znippy_throughput -- --ignored --nocapture` | Synthetic 50K-artifact benchmark (fake data) for all 3 types |
| `test_nexus_load_with_real_artifacts` | holger-agent-cli | `DOCKER_HOST=unix:///run/user/1000/podman/podman.sock cargo test --release -p holger-agent-cli --test integration_nexus_load -- --ignored --nocapture` | Starts real Sonatype Nexus 3 container, uploads all cached znippy archives, benchmarks upload/download, then runs retrieval bombardment comparing Nexus vs Holger |
### Test Cache
Heavy tests cache downloaded artifacts at `/home/rickard/work/holger_tests/`:
```
crates/ — 9,162 .crate files (4.2 GB) from crates.io
jars/ — 4,730 .jar files (5.0 GB) from Maven Central
wheels/ — 541 .whl files (5.0 GB) from PyPI
crates_archive.znippy — packed with znippy 0.7.2
jars_archive.znippy — packed with znippy 0.7.2
pip_archive.znippy — packed with znippy 0.7.2
```
If znippy version changes, delete `*.znippy` and re-run `integration_znippy_serve` to rebuild.
### Unit / Light Integration Tests
```bash
# All quick tests (no network, no containers):
cargo test --workspace
# Mannequin (mock Nexus) tests:
cargo test -p holger-agent-cli --test integration_holger -- --nocapture
cargo test -p holger-agent-cli --test integration_pip -- --nocapture
```
### Requirements
- **Podman** (or Docker) for Nexus container tests — enable socket: `systemctl --user start podman.socket`
- **Network access** for downloading real artifacts from crates.io, Maven Central, PyPI
- **~15 GB disk** for test cache
## Fan art
<img width="1024" height="1536" alt="holger" src="https://github.com/user-attachments/assets/cbc60639-0025-4437-a088-c41f8deded2e" />
*"Allfadern ser varje bit"*