Holger guards your artifacts at rest

Immutable artifact repository in pure Rust.

• Airgap: transfer from internet, Nexus, Artifactory → immutable znippy archive
• Versioned: atomic transactions, content-addressed (Blake3)
• Extreme performance: Apache Arrow zero-copy, multi-core parallel I/O
• Zero dependencies: no OS packages, no shell, no Python — 100% Rust (even decompressors rewritten from C)
• Embeddable: runs on bare-metal/OS-free devices, 2 MB RSS

Artifact retrieval throughput — Holger vs Nexus 3

Same machine (32 cores, AMD), same artifact types, same dataset size per type.
Nexus 3 (v3.72): container, 32 CPUs, 32 GB RAM, 16 GB heap, 400 Jetty threads, G1GC.
Holger (znippy 0.8.0): in-process, 2 MB RSS.

Holger is 7,972× faster on Rust crates at Nexus's sweet spot (16 clients).
12,887× faster on Java JARs, 8,106× faster on Python wheels.
Single-threaded Holger still outperforms 32-client Nexus by 805× (Rust), 677× (Java), 432× (Python).
Nexus degrades past 16 clients on all types despite 400 Jetty threads and 16 GB heap.
Holger uses 2 MB RSS vs Nexus's 16 GB heap — 8,000× less memory.

How it works

Each get_file is one pread from a shared fd plus either a zero-copy skip (pre-compressed .crate/.jar) or an openzl decompress into a reused buffer — fully parallel across cores, no shared reader thread. Backed by znippy 0.8.0's N-worker positioned-I/O read path.

Import / Export (holger-agent)

The agent CLI transfers artifacts between any combination of sources and destinations:

┌──────────────────────────────────────────────────────────────────────────────┐
│                          holger-agent transfer paths                          │
├──────────────────────────────────────────────────────────────────────────────┤
│                                                                              │
│  Sources (--from)              Destinations (--to)                            │
│  ─────────────────             ──────────────────                             │
│  world     (crates.io/PyPI/Maven Central)  ──┐                               │
│  nexus     (Sonatype Nexus, basic auth)    ──┤──▶ znippy    (.znippy archive)│
│  holger    (Holger server, OIDC/mTLS)      ──┤──▶ tar-zstd  (.tar.zst)      │
│  znippy    (.znippy archive)               ──┤──▶ directory  (flat files)    │
│  tar-zstd  (.tar.zst archive)              ──┤──▶ nexus     (push to Nexus)  │
│  directory (flat .crate/.jar/.whl files)   ──┘──▶ holger    (push to Holger) │
│                                                                              │
└──────────────────────────────────────────────────────────────────────────────┘

Examples

# ── Download from internet → znippy archive ─────────────────────────────────

# Rust crates (from Cargo.lock)
holger-agent --from world --lockfile Cargo.lock --to znippy -o rust.znippy

# Python packages (from requirements.txt)
holger-agent --from world --lockfile-kind requirements-txt \
  --lockfile requirements.txt --to znippy -o pip.znippy

# Python packages (from uv.lock — URLs already embedded)
holger-agent --from world --lockfile-kind uv-lock \
  --lockfile uv.lock --to znippy -o pip.znippy

# ── Download from internet → tar.zst archive ────────────────────────────────

# Rust crates (from Cargo.lock) — standard tar+zstd, no snappy dependency
holger-agent --from world --lockfile Cargo.lock --to tar-zstd -o rust-crates.tar.zst

# Python packages (from requirements.txt) → tar.zst
holger-agent --from world --lockfile-kind requirements-txt \
  --lockfile requirements.txt --to tar-zstd -o pip-packages.tar.zst

# Python packages (from uv.lock) → tar.zst
holger-agent --from world --lockfile-kind uv-lock \
  --lockfile uv.lock --to tar-zstd -o pip-packages.tar.zst

# ── Download from internet → directory ───────────────────────────────────────

# Rust crates to flat directory
holger-agent --from world --lockfile Cargo.lock --to directory -o ./crates/

# ── Nexus → znippy (mirror/export from Nexus) ──────────────────────────────

holger-agent --from nexus --nexus-url http://nexus.internal:8081 \
  --repository maven-releases --username admin --password secret \
  --to znippy -o maven-export.znippy

# ── Nexus → tar.zst (export from Nexus to standard archive) ─────────────────

holger-agent --from nexus --nexus-url http://nexus.internal:8081 \
  --repository maven-releases --username admin --password secret \
  --to tar-zstd -o maven-export.tar.zst

# ── Holger → tar.zst (export from Holger server) ────────────────────────────

holger-agent --from holger --holger-url grpc://holger.internal:50051 \
  --auth oidc --token-url https://keycloak/token --client-id ci-agent \
  --repository rust-prod --to tar-zstd -o rust-prod-backup.tar.zst

# ── znippy → Nexus (import into Nexus) ──────────────────────────────────────

holger-agent --from znippy --input maven-export.znippy \
  --to nexus --nexus-url http://nexus.internal:8081 \
  --repository maven-staging --username admin --password secret

# ── tar.zst → Nexus (import standard archive into Nexus) ────────────────────

holger-agent --from tar-zstd --input maven-export.tar.zst \
  --to nexus --nexus-url http://nexus.internal:8081 \
  --repository maven-staging --username admin --password secret

# ── znippy → Holger server (push to production) ─────────────────────────────

holger-agent --from znippy --input rust.znippy \
  --to holger --holger-url grpc://holger.internal:50051 \
  --auth oidc --token-url https://keycloak/token --client-id ci-agent \
  --repository rust-prod

# ── tar.zst → Holger server (push standard archive to production) ────────────

holger-agent --from tar-zstd --input rust-crates.tar.zst \
  --to holger --holger-url grpc://holger.internal:50051 \
  --auth oidc --token-url https://keycloak/token --client-id ci-agent \
  --repository rust-prod

# ── directory → Nexus (push loose files to Nexus) ────────────────────────────

holger-agent --from directory --input ./my-crates/ \
  --to nexus --nexus-url http://nexus.internal:8081 \
  --repository crates-raw --username admin --password secret

# ── directory → Holger (push loose files to Holger) ──────────────────────────

holger-agent --from directory --input ./my-crates/ \
  --to holger --holger-url grpc://holger.internal:50051 \
  --auth oidc --token-url https://keycloak/token --client-id ci-agent \
  --repository rust-staging

# ── Nexus → directory (flat file export) ─────────────────────────────────────

holger-agent --from nexus --nexus-url http://nexus.internal:8081 \
  --repository npm-hosted --username admin --password secret \
  --to directory -o ./exported-packages/

# ── Holger → directory (export from Holger to flat files) ────────────────────

holger-agent --from holger --holger-url grpc://holger.internal:50051 \
  --auth oidc --token-url https://keycloak/token --client-id ci-agent \
  --repository rust-prod --to directory -o ./exported-crates/

Airgap workflow

Internet side                    │  Airgapped side
                                 │
  crates.io ──┐                  │
  Maven ──────┤                  │
  PyPI ───────┤──▶ holger-agent  │  holger-agent ──▶ holger-server
  Nexus ──────┘    --to znippy   │  --from znippy      (gRPC / Rust fn)
                   --to tar-zstd │  --from tar-zstd
                       │        │       ▲
                       ▼        │       │
                   .znippy file ─┼───────┘  (znippy = max perf, Arrow IPC)
                   .tar.zst file ┼───────┘  (tar.zst = universal, no snappy)
                   (USB/media)   │

When to use which format:

Lockfile formats

Architecture

┌─────────────────────────────────────────────────────────────────────────┐
│                          Holger Ecosystem                                │
├─────────────────────────────────────────────────────────────────────────┤
│                                                                         │
│  ┌─────────────────┐     ┌──────────────────┐     ┌────────────────┐  │
│  │  holger-agent    │────▶│  holger-server    │────▶│  Storage       │  │
│  │  (CLI + daemon)  │     │  (gRPC / Rust fn) │     │  (.znippy)     │  │
│  │                  │     │                   │     │                │  │
│  │  --auth oidc     │     │  OIDC validation  │     │  Immutable     │  │
│  └─────────────────┘     └──────────────────┘     │  Blake3 CAS    │  │
│           │                        │               └────────────────┘  │
│           │                        │                                    │
│           ▼                        ▼                                    │
│  ┌─────────────────┐     ┌──────────────────┐                         │
│  │  Nexus/Registry  │     │  cargo/mvn        │                         │
│  │  (connector)     │     │  (clients)        │                         │
│  └─────────────────┘     └──────────────────┘                         │
│                                                                         │
└─────────────────────────────────────────────────────────────────────────┘

Access Modes

Holger exposes two access modes — no HTTP/REST:

1. Rust Function Bindings (embedded mode)

Use holger-server-lib directly in your Rust application:

use holger_server_lib::{read_ron_config, wire_holger};
use holger_traits::ArtifactId;

let mut holger = read_ron_config("holger.ron").unwrap();
holger.instantiate_backends().unwrap();
wire_holger(&mut holger).unwrap();

// Fetch artifact
let id = ArtifactId { namespace: None, name: "serde".into(), version: "1.0.219".into() };
let data = holger.fetch("rust-prod", &id).unwrap();

// List repositories
let repos = holger.list_repositories(); // ["rust-prod", "maven-staging"]

// Put artifact (write-enabled repos)
holger.put("rust-staging", &id, &bytes).unwrap();

2. gRPC (network mode)

Configure exposed_endpoints in RON → holger starts a tonic gRPC server automatically.

Services (defined in proto/holger.proto):

3. Apache Arrow Flight (batch mode) — planned

Arrow Flight as a universal artifact service: batch-fetch multiple artifacts (or files within artifacts) in a single call with zero-copy columnar transport.

Why Flight over raw gRPC for batch:

• znippy archives are already Arrow IPC → Flight serves them with near-zero serialization cost
• Columnar format means fetching 1000 small crates is one contiguous memory transfer, not 1000 RPCs
• uv/pip/cargo resolve+fetch could use a single Flight batch call instead of N HTTP GETs

Connect with grpcurl:

# Health check
grpcurl -plaintext localhost:50051 holger.AdminService/Health

# List repositories
grpcurl -plaintext localhost:50051 holger.AdminService/ListRepositories

# Fetch artifact
grpcurl -plaintext -d '{"repository":"rust-prod","name":"serde","version":"1.0.219"}' \
  localhost:50051 holger.RepositoryService/FetchArtifact

# Stream large artifact (chunked 64KB)
grpcurl -plaintext -d '{"repository":"rust-prod","name":"tokio","version":"1.47.1"}' \
  localhost:50051 holger.RepositoryService/StreamArtifact

Crates

Configuration (RON)

Holger uses RON for configuration — it can express the graph topology that TOML/YAML cannot.

Rust crate repository (file-backed)

(
    repositories: [
        (
            ron_name: "rust-prod",
            ron_repo_type: "rust",
            ron_upstreams: [],
            ron_in: None,
            ron_out: Some((
                ron_storage_endpoint: "local-storage",
                ron_exposed_endpoint: "grpc",
            )),
        ),
    ],
    exposed_endpoints: [
        (
            ron_name: "grpc",
            ron_url: "0.0.0.0:50051",
        ),
    ],
    storage_endpoints: [
        (
            ron_name: "local-storage",
            ron_storage_type: "rocksdb",
            ron_path: "/var/lib/holger/data",
        ),
    ],
)

Rust crate repository (znippy archive)

(
    repositories: [
        (
            ron_name: "rust-airgap",
            ron_repo_type: "rust",
            ron_archive_path: Some("/var/lib/holger/rust-airgap.znippy"),
            ron_upstreams: [],
            ron_in: None,
            ron_out: Some((
                ron_storage_endpoint: "znippy-storage",
                ron_exposed_endpoint: "grpc",
            )),
        ),
    ],
    exposed_endpoints: [
        (
            ron_name: "grpc",
            ron_url: "0.0.0.0:50051",
        ),
    ],
    storage_endpoints: [
        (
            ron_name: "znippy-storage",
            ron_storage_type: "znippy",
            ron_path: "/var/lib/holger/rust-airgap",
        ),
    ],
)

Python pip repository (znippy archive, PEP 503 simple index)

(
    repositories: [
        (
            ron_name: "pip-internal",
            ron_repo_type: "pip",
            ron_archive_path: Some("/var/lib/holger/pip.znippy"),
            ron_upstreams: [],
            ron_in: None,
            ron_out: Some((
                ron_storage_endpoint: "pip-storage",
                ron_exposed_endpoint: "grpc",
            )),
        ),
    ],
    exposed_endpoints: [
        (
            ron_name: "grpc",
            ron_url: "0.0.0.0:50051",
        ),
    ],
    storage_endpoints: [
        (
            ron_name: "pip-storage",
            ron_storage_type: "znippy",
            ron_path: "/var/lib/holger/pip",
        ),
    ],
)

The server exposes a PEP 503 simple index at /{repo}/simple/ — point pip or uv directly at it:

pip install --index-url https://holger.internal/pip-internal/simple/ requests
uv pip install --index-url https://holger.internal/pip-internal/simple/ requests

Maven repository (znippy archive)

(
    repositories: [
        (
            ron_name: "maven-internal",
            ron_repo_type: "maven3",
            ron_archive_path: Some("/var/lib/holger/maven.znippy"),
            ron_upstreams: [],
            ron_in: None,
            ron_out: Some((
                ron_storage_endpoint: "maven-storage",
                ron_exposed_endpoint: "grpc",
            )),
        ),
    ],
    exposed_endpoints: [
        (
            ron_name: "grpc",
            ron_url: "0.0.0.0:50051",
        ),
    ],
    storage_endpoints: [
        (
            ron_name: "maven-storage",
            ron_storage_type: "znippy",
            ron_path: "/var/lib/holger/maven",
        ),
    ],
)

Authentication

The agent CLI uses --auth oidc when talking to Holger server. Legacy Nexus connections still use --username/--password (basic auth via connector).

Performance

Holger uses znippy archives (.znippy) as storage. A .znippy file is a valid Arrow IPC File — queryable directly by pyarrow/polars:

# pyarrow
import pyarrow.ipc
reader = pyarrow.ipc.open_file("archive.znippy")
table = reader.read_all()
print(table.schema)
# relative_path: string, compressed: bool, uncompressed_size: uint64, chunks: list<...>

# polars
import polars as pl
df = pl.read_ipc("archive.znippy")
df.select("relative_path", "uncompressed_size", "compressed").head(10)

Znippy archive benchmarks (v0.7.4 — io_uring + Gatling pipeline)

Packing source files into .znippy and extracting back. Files already compressed (.gz, .jar, .zip, etc.) are stored as-is (skipped). .crate files are gzipped tarballs — znippy compresses the source contents inside, not the .crate blob itself:

Note: "Pack" = archiving + compression into .znippy. "Unpack" = decompression + extraction. Stream packer (in-memory/HTTP) numbers above; slot packer (io_uring disk reads) hits 678 MB/s on 10k×10KB synthetic. Already-compressed data (.jar, .crate) passes through at full I/O speed without re-compression.

Host decompressors

The host-decompressors feature replaces miniz_oxide with custom parallel decompression:

Single-core path is 3.7× faster than miniz_oxide. Multi-core scales linearly.

GUI Builders

Holger has no built-in web UI. Instead, build your own GUI using either access mode:

Option A: gRPC client (any language)

Use the proto definition (proto/holger.proto) to generate a client in any language.

Getting OIDC tokens and TLS certs from mannequin (for GUI dev):

Start mannequin first — it provides the complete auth infrastructure your GUI needs:

# 1. Start mannequin (mock PKI + OIDC + Nexus)
cargo run -p holger-mannequin -- serve --gui --port 9999

# 2. Get an OIDC token (accepts any credentials — it's a dev dummy)
TOKEN=$(curl -s -X POST http://127.0.0.1:9999/token \
  -d 'grant_type=password&username=admin&password=anything&client_id=my-gui' \
  | jq -r .access_token)

echo "Bearer token: $TOKEN"

# 3. Get a TLS client certificate (for mTLS to holger-server)
curl -s -X POST http://127.0.0.1:9999/pki/issue/client \
  -H "Content-Type: application/json" \
  -d '{"common_name": "my-gui-client"}' \
  | jq -r '.cert_pem' > client.pem

curl -s -X POST http://127.0.0.1:9999/pki/issue/client \
  -H "Content-Type: application/json" \
  -d '{"common_name": "my-gui-client"}' \
  | jq -r '.key_pem' > client-key.pem

# 4. Get the CA cert (to verify holger-server's TLS)
curl -s http://127.0.0.1:9999/pki/ca.pem > ca.pem

# 5. Connect to holger-server with OIDC auth
grpcurl -H "Authorization: Bearer $TOKEN" \
  -cacert ca.pem -cert client.pem -key client-key.pem \
  localhost:50051 holger.AdminService/ListRepositories

In your GUI code (Rust + tonic):

use tonic::transport::{Channel, ClientTlsConfig, Certificate, Identity};
use tonic::metadata::MetadataValue;

// Get token from mannequin OIDC endpoint
let token_resp: serde_json::Value = reqwest::Client::new()
    .post("http://127.0.0.1:9999/token")
    .form(&[("grant_type", "password"), ("username", "admin"),
            ("password", "dev"), ("client_id", "my-gui")])
    .send().await?.json().await?;
let token = token_resp["access_token"].as_str().unwrap();

// Get CA + client cert from mannequin PKI
let ca_pem = reqwest::get("http://127.0.0.1:9999/pki/ca.pem").await?.bytes().await?;
let client_cert = reqwest::Client::new()
    .post("http://127.0.0.1:9999/pki/issue/client")
    .json(&serde_json::json!({"common_name": "my-gui"}))
    .send().await?.json::<serde_json::Value>().await?;

// Build TLS channel
let tls = ClientTlsConfig::new()
    .ca_certificate(Certificate::from_pem(&ca_pem))
    .identity(Identity::from_pem(
        client_cert["cert_pem"].as_str().unwrap(),
        client_cert["key_pem"].as_str().unwrap(),
    ));

let channel = Channel::from_static("https://localhost:50051")
    .tls_config(tls)?
    .connect().await?;

// Attach OIDC bearer token to every request
let token_str: MetadataValue<_> = format!("Bearer {}", token).parse()?;
let mut admin = holger::admin_service_client::AdminServiceClient::with_interceptor(
    channel.clone(),
    move |mut req: tonic::Request<()>| {
        req.metadata_mut().insert("authorization", token_str.clone());
        Ok(req)
    },
);

// Now use it
let repos = admin.list_repositories(holger::Empty {}).await?.into_inner();

Plain examples (no auth, local dev):

// Rust (tonic)
use tonic::transport::Channel;

let channel = Channel::from_static("http://[::1]:50051").connect().await?;
let mut admin = holger::admin_service_client::AdminServiceClient::new(channel.clone());
let mut repo = holger::repository_service_client::RepositoryServiceClient::new(channel);

// List repositories
let repos = admin.list_repositories(holger::Empty {}).await?.into_inner();
for name in &repos.names {
    println!("{}", name);
}

// Fetch artifact
let artifact = repo.fetch_artifact(holger::FetchRequest {
    repository: "rust-prod".into(),
    name: "serde".into(),
    version: "1.0.219".into(),
    namespace: String::new(),
}).await?.into_inner();

// Stream large artifact (server sends 64KB chunks)
let mut stream = repo.stream_artifact(holger::FetchRequest {
    repository: "rust-prod".into(),
    name: "tokio".into(),
    version: "1.47.1".into(),
    namespace: String::new(),
}).await?.into_inner();

while let Some(chunk) = stream.message().await? {
    // chunk.data: Vec<u8>, chunk.offset: u64
}

# Python (grpcio)
import grpc
import holger_pb2, holger_pb2_grpc

channel = grpc.insecure_channel('localhost:50051')
admin = holger_pb2_grpc.AdminServiceStub(channel)
repo = holger_pb2_grpc.RepositoryServiceStub(channel)

# List repos
response = admin.ListRepositories(holger_pb2.Empty())
print(response.names)

# Fetch artifact
artifact = repo.FetchArtifact(holger_pb2.FetchRequest(
    repository="rust-prod", name="serde", version="1.0.219"))

// Go (google.golang.org/grpc)
conn, _ := grpc.Dial("localhost:50051", grpc.WithInsecure())
admin := pb.NewAdminServiceClient(conn)
repo := pb.NewRepositoryServiceClient(conn)

repos, _ := admin.ListRepositories(ctx, &pb.Empty{})
artifact, _ := repo.FetchArtifact(ctx, &pb.FetchRequest{
    Repository: "rust-prod", Name: "serde", Version: "1.0.219"})

Option B: Rust function bindings (same process, zero overhead)

For Rust-native GUIs (egui, iced, slint, dioxus), embed holger directly:

use holger_server_lib::{read_ron_config, wire_holger, Holger};
use holger_traits::ArtifactId;

// Initialize once at app startup
let mut holger = read_ron_config("holger.ron").unwrap();
holger.instantiate_backends().unwrap();
wire_holger(&mut holger).unwrap();

// Use from any thread — no serialization, no network
let repos = holger.list_repositories();  // Vec<&str>

let id = ArtifactId { namespace: None, name: "serde".into(), version: "1.0.219".into() };
let data: Option<Vec<u8>> = holger.fetch("rust-prod", &id).unwrap();

// Upload
holger.put("rust-staging", &id, &bytes).unwrap();

gRPC services reference

Development (holger-mannequin)

The holger-mannequin crate provides a full mock environment for development and integration testing:

┌─────────────────────────────────────────────┐
│              holger-mannequin                 │
├──────────┬──────────┬───────────┬───────────┤
│   PKI    │   OIDC   │   Nexus   │   Auth    │
│          │          │           │           │
│ CA mgmt  │ /token   │ proxy-only│ Users     │
│ Issue    │ /userinfo│ prod (4)  │ Roles     │
│ Regen    │ Discovery│ upload    │ Validate  │
└──────────┴──────────┴───────────┴───────────┘
         │
         ▼
    egui GUI (--gui flag)
    Live system map + request logs

Subsystems:

Usage modes:

# As a server (dev environment)
cargo run -p holger-mannequin -- serve --gui --port 9999

# In-process (integration tests — no network needed)
let mannequin = holger_mannequin::Mannequin::new();
let token = mannequin.oidc.issue_token("admin");
let cert = mannequin.pki.issue_client_cert("ci-agent");

# CLI (cert generation)
cargo run -p holger-mannequin -- pki generate --output ./certs

egui GUI features:

• Live system topology map (repositories ↔ storage ↔ endpoints)
• Request traffic visualization (gRPC calls in real-time)
• OIDC token issuer for quick dev testing
• Mock Nexus browser with seeded artifacts

Quick Start

# Build everything
PROTOC=/path/to/protoc cargo build --workspace

# Run tests
PROTOC=/path/to/protoc cargo test --workspace

# Start mannequin (dev dummy with mock PKI, OIDC, Nexus + GUI)
cargo run -p holger-mannequin -- serve --gui

# Start holger server (starts gRPC on configured port)
cargo run -p holger-server-cli -- start --config holger.ron

# Agent: see "Import / Export" section above for all transfer examples

Immutable Storage Model

All .artifact files are immutable. Holger can optionally be configured to allow live ingest of artifacts not found in the current archive (DEV mode only).

Status

• ✅ RON config + graph wiring engine
• ✅ gRPC server (tonic, 3 services)
• ✅ Rust function bindings API (embedded mode)
• ✅ Rust crate serving (file + znippy backends)
• ✅ Maven artifact serving (znippy backend)
• ✅ Python pip serving (znippy backend, PEP 503 simple index)
• ✅ Python pip airgap agent (requirements.txt + uv.lock → znippy)
• ✅ Airgap agent (world → znippy → nexus/holger)
• ✅ OIDC authentication
• ✅ Nexus connector (list/download/upload)
• ✅ Znippy host-decompressors (lgz, ljar, linflate)
• ✅ Container distribution (Dockerfile + systemd)
• ✅ Dev mannequin with egui system map GUI
• 🚧 Blake3 content-addressable verification
• 🚧 Authorization layer (role-based access on repos)
• 🚧 Go/npm format support
• 🚧 Live ingest proxy mode (DEV)
• 🚧 Promote workflow (DEV → PROD)
• 🚧 egui admin client for holger-server
• 🚧 Audit log (who accessed what, when)

Testing

Test Levels

Holger has three test levels:

1. Unit / light integration — no network, no containers, runs in seconds
2. Synthetic benchmarks — auto-generated fake data, no network, measures throughput regression
3. Heavy / container tests — spins up a real Sonatype Nexus 3 (v3.72.0) container via testcontainers crate + Podman, uploads real artifact archives, benchmarks Holger vs Nexus

Unit / Light Integration Tests

These use holger-mannequin as an in-process mock (Nexus API + OIDC + mTLS + PKI). No external services needed.

# All quick tests (no network, no containers):
cargo test --workspace

# Mannequin (mock Nexus/OIDC/mTLS) tests:
cargo test -p holger-agent-cli --test integration_holger -- --nocapture
cargo test -p holger-agent-cli --test integration_pip -- --nocapture

# tar.zst round-trip tests (mannequin mock, no network):
cargo test -p holger-agent-cli --test integration_tar_zstd -- --nocapture

Heavy Tests (Real Nexus Container)

These tests start a real Sonatype Nexus 3 container using the testcontainers Rust crate. The container lifecycle is fully managed by the test — it starts Nexus, waits for readiness, creates repositories, runs the benchmark, and tears down automatically.

How it works:

1. testcontainers pulls sonatype/nexus3:3.72.0 via Podman (or Docker)
2. Container starts with port 8081 exposed on a random host port
3. Test waits up to 180s for Nexus REST API (/service/rest/v1/status) to respond
4. Reads initial admin password from container (/nexus-data/admin.password)
5. Changes password to admin123, creates raw hosted repositories
6. Uploads cached znippy archives → benchmarks upload throughput (MB/s, files/s)
7. Downloads all artifacts back → benchmarks download throughput
8. Runs retrieval bombardment (multi-threaded random GETs) on Nexus, then same dataset on Holger znippy for comparison
9. Container is stopped and removed automatically

The tail-latency test additionally constrains Nexus to a 512 MB JVM heap (-Xms512m -Xmx512m -XX:+UseG1GC) to provoke GC pauses, measuring p99/p999 latency vs Holger's zero-GC constant-time reads.

Running heavy tests:

# Prerequisite: enable Podman socket (rootless)
systemctl --user start podman.socket

# Set DOCKER_HOST so testcontainers finds the Podman socket
export DOCKER_HOST=unix:///run/user/1000/podman/podman.sock

# Nexus load test (uploads ~14 GB, benchmarks Holger vs Nexus)
cargo test --release -p holger-agent-cli \
  --test integration_nexus_load -- --ignored --nocapture

# Tail-latency / GC pressure test (60s bombardment, p99/p999 comparison)
cargo test --release -p holger-agent-cli \
  --test integration_latency_tail -- --ignored --nocapture

Or run everything via xtask:

cargo xtask --heavy

Synthetic Benchmarks (No Network)

The bench_znippy_throughput test generates 500K fake crates/JARs/wheels in /tmp/holger_bench_throughput/ — auto-created on first run, cached for subsequent runs. Measures raw get_file() and HTTP handler ops/s for all 3 artifact types (single-thread and multi-thread).

cargo test --release -p holger-rust-znippy-repository \
  --test bench_znippy_throughput -- --ignored --nocapture

Integration Test Summary

¹ Container image must be available locally or pullable. Test data uses pre-cached archives.

Test Cache

Heavy tests cache downloaded artifacts at /home/rickard/work/holger_tests/:

crates/          — 9,162 .crate files (4.2 GB) from crates.io
jars/            — 4,730 .jar files (5.0 GB) from Maven Central
wheels/          — 541 .whl files (5.0 GB) from PyPI
crates_archive.znippy  — packed with znippy 0.7.4
jars_archive.znippy    — packed with znippy 0.7.4
pip_archive.znippy     — packed with znippy 0.7.4

If files don't exist, tests create them automatically — they download from crates.io / Maven Central / PyPI on first run. Subsequent runs reuse the cached archives. If znippy version changes, delete *.znippy and re-run to rebuild.

Requirements

Release

Pre-release gate: cargo xtask --heavy

You MUST run cargo xtask --heavy before publishing any crate. This is the full release gate — it runs every test level including a real Nexus 3 container benchmark. If it passes, the code is safe to publish. If it fails, do not release.

# Prerequisites
systemctl --user start podman.socket
export DOCKER_HOST=unix:///run/user/1000/podman/podman.sock

# THE release command — runs everything:
cargo xtask --heavy

What cargo xtask --heavy does (in order):

cargo xtask (without --heavy) skips steps 5–7 — use it for quick local iteration, but never for release.

Bump versions and publish

# 1. Bump version in all Cargo.toml files (workspace members share versions)
#    holger-traits, holger-*-repository, holger-server-*, holger-agent-*

# 2. Run the FULL release gate (mandatory — do not skip)
cargo xtask --heavy

# 3. Publish crates in dependency order (traits first, then backends, then server/agent)
cargo publish -p holger-traits
cargo publish -p holger-rust-file-repository
cargo publish -p holger-rust-znippy-repository
cargo publish -p holger-maven-znippy-repository
cargo publish -p holger-python-znippy-repository
cargo publish -p holger-nexus-connector
cargo publish -p holger-world-connector
cargo publish -p holger-server-lib
cargo publish -p holger-server-cli
cargo publish -p holger-agent-lib
cargo publish -p holger-agent-cli
cargo publish -p holger-mannequin

# 4. Tag and push
git tag v0.X.Y
git push && git push --tags

Publish order matters

Crates.io requires dependencies to be published first:

holger-traits                    (no internal deps)
  ├── holger-rust-file-repository
  ├── holger-rust-znippy-repository
  ├── holger-maven-znippy-repository
  ├── holger-python-znippy-repository
  ├── holger-nexus-connector
  ├── holger-world-connector
  ├── holger-server-lib          (depends on traits + all repos)
  │     └── holger-server-cli
  ├── holger-agent-lib           (depends on traits + connectors)
  │     └── holger-agent-cli
  └── holger-mannequin

Fan art

"Allfadern ser varje bit"