Holger

Holger guards your artifacts at rest. Immutable and silly-high-speed for your developers.
"Allfadern ser varje bit"

A secure, Rust-based artifact repository server with airgap transfer capabilities. Holger ingests language-specific package trees and serves them via gRPC or direct Rust function bindings — like Artifactory or Nexus, but with immutable append-only storage, strong authentication (OIDC), and a purpose-built airgap workflow using Znippy archives.

Artifact retrieval throughput

Random-access retrieval from znippy archives vs Sonatype Nexus 3 (v3.72, container, 64-core machine, 4GB heap).
Holger numbers use the in-process API (no network); Nexus numbers are over HTTP.

Holger is ~2.3× faster than Nexus at full concurrency for Rust crates (in-process vs HTTP).
Both scale linearly with threads. Nexus does not degrade under higher concurrency in this test.
The key advantage of Holger is not raw HTTP throughput but the in-process API — embedding the repository directly in your Rust application with zero network overhead.

Architecture

┌─────────────────────────────────────────────────────────────────────────┐
│                          Holger Ecosystem                                │
├─────────────────────────────────────────────────────────────────────────┤
│                                                                         │
│  ┌─────────────────┐     ┌──────────────────┐     ┌────────────────┐  │
│  │  holger-agent    │────▶│  holger-server    │────▶│  Storage       │  │
│  │  (CLI + daemon)  │     │  (gRPC / Rust fn) │     │  (.znippy)     │  │
│  │                  │     │                   │     │                │  │
│  │  --auth oidc     │     │  OIDC validation  │     │  Immutable     │  │
│  └─────────────────┘     └──────────────────┘     │  Blake3 CAS    │  │
│           │                        │               └────────────────┘  │
│           │                        │                                    │
│           ▼                        ▼                                    │
│  ┌─────────────────┐     ┌──────────────────┐                         │
│  │  Nexus/Registry  │     │  cargo/mvn        │                         │
│  │  (connector)     │     │  (clients)        │                         │
│  └─────────────────┘     └──────────────────┘                         │
│                                                                         │
└─────────────────────────────────────────────────────────────────────────┘

Access Modes

Holger exposes two access modes — no HTTP/REST:

1. Rust Function Bindings (embedded mode)

Use holger-server-lib directly in your Rust application:

use holger_server_lib::{read_ron_config, wire_holger};
use holger_traits::ArtifactId;

let mut holger = read_ron_config("holger.ron").unwrap();
holger.instantiate_backends().unwrap();
wire_holger(&mut holger).unwrap();

// Fetch artifact
let id = ArtifactId { namespace: None, name: "serde".into(), version: "1.0.219".into() };
let data = holger.fetch("rust-prod", &id).unwrap();

// List repositories
let repos = holger.list_repositories(); // ["rust-prod", "maven-staging"]

// Put artifact (write-enabled repos)
holger.put("rust-staging", &id, &bytes).unwrap();

2. gRPC (network mode)

Configure exposed_endpoints in RON → holger starts a tonic gRPC server automatically.

Services (defined in proto/holger.proto):

Connect with grpcurl:

# Health check
grpcurl -plaintext localhost:50051 holger.AdminService/Health

# List repositories
grpcurl -plaintext localhost:50051 holger.AdminService/ListRepositories

# Fetch artifact
grpcurl -plaintext -d '{"repository":"rust-prod","name":"serde","version":"1.0.219"}' \
  localhost:50051 holger.RepositoryService/FetchArtifact

# Stream large artifact (chunked 64KB)
grpcurl -plaintext -d '{"repository":"rust-prod","name":"tokio","version":"1.47.1"}' \
  localhost:50051 holger.RepositoryService/StreamArtifact

Crates

Configuration (RON)

Holger uses RON for configuration — it can express the graph topology that TOML/YAML cannot.

Rust crate repository (file-backed)

(
    repositories: [
        (
            ron_name: "rust-prod",
            ron_repo_type: "rust",
            ron_upstreams: [],
            ron_in: None,
            ron_out: Some((
                ron_storage_endpoint: "local-storage",
                ron_exposed_endpoint: "grpc",
            )),
        ),
    ],
    exposed_endpoints: [
        (
            ron_name: "grpc",
            ron_url: "0.0.0.0:50051",
        ),
    ],
    storage_endpoints: [
        (
            ron_name: "local-storage",
            ron_storage_type: "rocksdb",
            ron_path: "/var/lib/holger/data",
        ),
    ],
)

Rust crate repository (znippy archive)

(
    repositories: [
        (
            ron_name: "rust-airgap",
            ron_repo_type: "rust",
            ron_archive_path: Some("/var/lib/holger/rust-airgap.znippy"),
            ron_upstreams: [],
            ron_in: None,
            ron_out: Some((
                ron_storage_endpoint: "znippy-storage",
                ron_exposed_endpoint: "grpc",
            )),
        ),
    ],
    exposed_endpoints: [
        (
            ron_name: "grpc",
            ron_url: "0.0.0.0:50051",
        ),
    ],
    storage_endpoints: [
        (
            ron_name: "znippy-storage",
            ron_storage_type: "znippy",
            ron_path: "/var/lib/holger/rust-airgap",
        ),
    ],
)

Python pip repository (znippy archive, PEP 503 simple index)

(
    repositories: [
        (
            ron_name: "pip-internal",
            ron_repo_type: "pip",
            ron_archive_path: Some("/var/lib/holger/pip.znippy"),
            ron_upstreams: [],
            ron_in: None,
            ron_out: Some((
                ron_storage_endpoint: "pip-storage",
                ron_exposed_endpoint: "grpc",
            )),
        ),
    ],
    exposed_endpoints: [
        (
            ron_name: "grpc",
            ron_url: "0.0.0.0:50051",
        ),
    ],
    storage_endpoints: [
        (
            ron_name: "pip-storage",
            ron_storage_type: "znippy",
            ron_path: "/var/lib/holger/pip",
        ),
    ],
)

The server exposes a PEP 503 simple index at /{repo}/simple/ — point pip or uv directly at it:

pip install --index-url https://holger.internal/pip-internal/simple/ requests
uv pip install --index-url https://holger.internal/pip-internal/simple/ requests

Maven repository (znippy archive)

(
    repositories: [
        (
            ron_name: "maven-internal",
            ron_repo_type: "maven3",
            ron_archive_path: Some("/var/lib/holger/maven.znippy"),
            ron_upstreams: [],
            ron_in: None,
            ron_out: Some((
                ron_storage_endpoint: "maven-storage",
                ron_exposed_endpoint: "grpc",
            )),
        ),
    ],
    exposed_endpoints: [
        (
            ron_name: "grpc",
            ron_url: "0.0.0.0:50051",
        ),
    ],
    storage_endpoints: [
        (
            ron_name: "maven-storage",
            ron_storage_type: "znippy",
            ron_path: "/var/lib/holger/maven",
        ),
    ],
)

Authentication

The agent CLI uses --auth oidc when talking to Holger server. Legacy Nexus connections still use --username/--password (basic auth via connector).

Airgap Workflow

Internet side                    │  Airgapped side
                                 │
  crates.io ──┐                  │
  Maven ──────┤                  │
  PyPI ───────┤──▶ holger-agent  │  holger-agent ──▶ holger-server
  npm ────────┘    --to znippy   │  --from znippy      (gRPC / Rust fn)
                       │         │       ▲
                       ▼         │       │
                   .znippy file ─┼───────┘
                   (USB/media)   │

Performance

Holger uses znippy archives (.znippy) as storage. A .znippy file is a valid Arrow IPC File — queryable directly by pyarrow/polars:

# pyarrow
import pyarrow.ipc
reader = pyarrow.ipc.open_file("archive.znippy")
table = reader.read_all()
print(table.schema)
# relative_path: string, compressed: bool, uncompressed_size: uint64, chunks: list<...>

# polars
import polars as pl
df = pl.read_ipc("archive.znippy")
df.select("relative_path", "uncompressed_size", "compressed").head(10)

Znippy archive benchmarks

Packing source files into .znippy and extracting back. Files already compressed (.gz, .jar, .zip, etc.) are stored as-is (skipped). .crate files are gzipped tarballs — znippy compresses the source contents inside, not the .crate blob itself:

Note: "Pack" = archiving + compression into .znippy. "Unpack" = decompression + extraction. Already-compressed data (random, .jar, .crate blobs) passes through at ~1.3 GB/s without re-compression.

Host decompressors

The host-decompressors feature replaces miniz_oxide with custom parallel decompression:

Single-core path is 3.7× faster than miniz_oxide. Multi-core scales linearly.

GUI Builders

Holger has no built-in web UI. Instead, build your own GUI using either access mode:

Option A: gRPC client (any language)

Use the proto definition (proto/holger.proto) to generate a client in any language.

Getting OIDC tokens and TLS certs from mannequin (for GUI dev):

Start mannequin first — it provides the complete auth infrastructure your GUI needs:

# 1. Start mannequin (mock PKI + OIDC + Nexus)
cargo run -p holger-mannequin -- serve --gui --port 9999

# 2. Get an OIDC token (accepts any credentials — it's a dev dummy)
TOKEN=$(curl -s -X POST http://127.0.0.1:9999/token \
  -d 'grant_type=password&username=admin&password=anything&client_id=my-gui' \
  | jq -r .access_token)

echo "Bearer token: $TOKEN"

# 3. Get a TLS client certificate (for mTLS to holger-server)
curl -s -X POST http://127.0.0.1:9999/pki/issue/client \
  -H "Content-Type: application/json" \
  -d '{"common_name": "my-gui-client"}' \
  | jq -r '.cert_pem' > client.pem

curl -s -X POST http://127.0.0.1:9999/pki/issue/client \
  -H "Content-Type: application/json" \
  -d '{"common_name": "my-gui-client"}' \
  | jq -r '.key_pem' > client-key.pem

# 4. Get the CA cert (to verify holger-server's TLS)
curl -s http://127.0.0.1:9999/pki/ca.pem > ca.pem

# 5. Connect to holger-server with OIDC auth
grpcurl -H "Authorization: Bearer $TOKEN" \
  -cacert ca.pem -cert client.pem -key client-key.pem \
  localhost:50051 holger.AdminService/ListRepositories

In your GUI code (Rust + tonic):

use tonic::transport::{Channel, ClientTlsConfig, Certificate, Identity};
use tonic::metadata::MetadataValue;

// Get token from mannequin OIDC endpoint
let token_resp: serde_json::Value = reqwest::Client::new()
    .post("http://127.0.0.1:9999/token")
    .form(&[("grant_type", "password"), ("username", "admin"),
            ("password", "dev"), ("client_id", "my-gui")])
    .send().await?.json().await?;
let token = token_resp["access_token"].as_str().unwrap();

// Get CA + client cert from mannequin PKI
let ca_pem = reqwest::get("http://127.0.0.1:9999/pki/ca.pem").await?.bytes().await?;
let client_cert = reqwest::Client::new()
    .post("http://127.0.0.1:9999/pki/issue/client")
    .json(&serde_json::json!({"common_name": "my-gui"}))
    .send().await?.json::<serde_json::Value>().await?;

// Build TLS channel
let tls = ClientTlsConfig::new()
    .ca_certificate(Certificate::from_pem(&ca_pem))
    .identity(Identity::from_pem(
        client_cert["cert_pem"].as_str().unwrap(),
        client_cert["key_pem"].as_str().unwrap(),
    ));

let channel = Channel::from_static("https://localhost:50051")
    .tls_config(tls)?
    .connect().await?;

// Attach OIDC bearer token to every request
let token_str: MetadataValue<_> = format!("Bearer {}", token).parse()?;
let mut admin = holger::admin_service_client::AdminServiceClient::with_interceptor(
    channel.clone(),
    move |mut req: tonic::Request<()>| {
        req.metadata_mut().insert("authorization", token_str.clone());
        Ok(req)
    },
);

// Now use it
let repos = admin.list_repositories(holger::Empty {}).await?.into_inner();

Plain examples (no auth, local dev):

// Rust (tonic)
use tonic::transport::Channel;

let channel = Channel::from_static("http://[::1]:50051").connect().await?;
let mut admin = holger::admin_service_client::AdminServiceClient::new(channel.clone());
let mut repo = holger::repository_service_client::RepositoryServiceClient::new(channel);

// List repositories
let repos = admin.list_repositories(holger::Empty {}).await?.into_inner();
for name in &repos.names {
    println!("{}", name);
}

// Fetch artifact
let artifact = repo.fetch_artifact(holger::FetchRequest {
    repository: "rust-prod".into(),
    name: "serde".into(),
    version: "1.0.219".into(),
    namespace: String::new(),
}).await?.into_inner();

// Stream large artifact (server sends 64KB chunks)
let mut stream = repo.stream_artifact(holger::FetchRequest {
    repository: "rust-prod".into(),
    name: "tokio".into(),
    version: "1.47.1".into(),
    namespace: String::new(),
}).await?.into_inner();

while let Some(chunk) = stream.message().await? {
    // chunk.data: Vec<u8>, chunk.offset: u64
}

# Python (grpcio)
import grpc
import holger_pb2, holger_pb2_grpc

channel = grpc.insecure_channel('localhost:50051')
admin = holger_pb2_grpc.AdminServiceStub(channel)
repo = holger_pb2_grpc.RepositoryServiceStub(channel)

# List repos
response = admin.ListRepositories(holger_pb2.Empty())
print(response.names)

# Fetch artifact
artifact = repo.FetchArtifact(holger_pb2.FetchRequest(
    repository="rust-prod", name="serde", version="1.0.219"))

// Go (google.golang.org/grpc)
conn, _ := grpc.Dial("localhost:50051", grpc.WithInsecure())
admin := pb.NewAdminServiceClient(conn)
repo := pb.NewRepositoryServiceClient(conn)

repos, _ := admin.ListRepositories(ctx, &pb.Empty{})
artifact, _ := repo.FetchArtifact(ctx, &pb.FetchRequest{
    Repository: "rust-prod", Name: "serde", Version: "1.0.219"})

Option B: Rust function bindings (same process, zero overhead)

For Rust-native GUIs (egui, iced, slint, dioxus), embed holger directly:

use holger_server_lib::{read_ron_config, wire_holger, Holger};
use holger_traits::ArtifactId;

// Initialize once at app startup
let mut holger = read_ron_config("holger.ron").unwrap();
holger.instantiate_backends().unwrap();
wire_holger(&mut holger).unwrap();

// Use from any thread — no serialization, no network
let repos = holger.list_repositories();  // Vec<&str>

let id = ArtifactId { namespace: None, name: "serde".into(), version: "1.0.219".into() };
let data: Option<Vec<u8>> = holger.fetch("rust-prod", &id).unwrap();

// Upload
holger.put("rust-staging", &id, &bytes).unwrap();

gRPC services reference

Development (holger-mannequin)

The holger-mannequin crate provides a full mock environment for development and integration testing:

┌─────────────────────────────────────────────┐
│              holger-mannequin                 │
├──────────┬──────────┬───────────┬───────────┤
│   PKI    │   OIDC   │   Nexus   │   Auth    │
│          │          │           │           │
│ CA mgmt  │ /token   │ proxy-only│ Users     │
│ Issue    │ /userinfo│ prod (4)  │ Roles     │
│ Regen    │ Discovery│ upload    │ Validate  │
└──────────┴──────────┴───────────┴───────────┘
         │
         ▼
    egui GUI (--gui flag)
    Live system map + request logs

Subsystems:

Usage modes:

# As a server (dev environment)
cargo run -p holger-mannequin -- serve --gui --port 9999

# In-process (integration tests — no network needed)
let mannequin = holger_mannequin::Mannequin::new();
let token = mannequin.oidc.issue_token("admin");
let cert = mannequin.pki.issue_client_cert("ci-agent");

# CLI (cert generation)
cargo run -p holger-mannequin -- pki generate --output ./certs

egui GUI features:

• Live system topology map (repositories ↔ storage ↔ endpoints)
• Request traffic visualization (gRPC calls in real-time)
• OIDC token issuer for quick dev testing
• Mock Nexus browser with seeded artifacts

Quick Start

# Build everything
PROTOC=/path/to/protoc cargo build --workspace

# Run tests
PROTOC=/path/to/protoc cargo test --workspace

# Start mannequin (dev dummy with mock PKI, OIDC, Nexus + GUI)
cargo run -p holger-mannequin -- serve --gui

# Agent: download from crates.io → znippy archive
cargo run -p holger-agent -- --from world --to znippy --lockfile Cargo.lock --output ./airgap.znippy

# Agent: download Python packages (requirements.txt) → znippy archive
cargo run -p holger-agent -- --from world --lockfile-kind requirements-txt \
  --lockfile requirements.txt --to znippy --output ./pip-airgap.znippy

# Agent: download Python packages (uv.lock, URLs already embedded) → znippy archive
cargo run -p holger-agent -- --from world --lockfile-kind uv-lock \
  --lockfile uv.lock --to znippy --output ./pip-airgap.znippy

# Agent: push to Holger with OIDC auth
cargo run -p holger-agent -- \
  --from znippy --to holger \
  --input ./airgap.znippy \
  --holger-url grpc://holger.internal:50051 \
  --auth oidc --token-url https://keycloak/token --client-id ci-agent \
  --repository rust-prod

# Start holger server (starts gRPC on configured port)
cargo run -p holger-server-cli -- start --config holger.ron

Immutable Storage Model

All .artifact files are immutable. Holger can optionally be configured to allow live ingest of artifacts not found in the current archive (DEV mode only).

Status

• ✅ RON config + graph wiring engine
• ✅ gRPC server (tonic, 3 services)
• ✅ Rust function bindings API (embedded mode)
• ✅ Rust crate serving (file + znippy backends)
• ✅ Maven artifact serving (znippy backend)
• ✅ Python pip serving (znippy backend, PEP 503 simple index)
• ✅ Python pip airgap agent (requirements.txt + uv.lock → znippy)
• ✅ Airgap agent (world → znippy → nexus/holger)
• ✅ OIDC authentication
• ✅ Nexus connector (list/download/upload)
• ✅ Znippy host-decompressors (lgz, ljar, linflate)
• ✅ Container distribution (Dockerfile + systemd)
• ✅ Dev mannequin with egui system map GUI
• 🚧 Blake3 content-addressable verification
• 🚧 Authorization layer (role-based access on repos)
• 🚧 Go/npm format support
• 🚧 Live ingest proxy mode (DEV)
• 🚧 Promote workflow (DEV → PROD)
• 🚧 egui admin client for holger-server
• 🚧 Audit log (who accessed what, when)

Testing

Integration & Benchmark Tests

All heavy tests are #[ignore] — run explicitly with -- --ignored --nocapture.

Test Cache

Heavy tests cache downloaded artifacts at /home/rickard/work/holger_tests/:

crates/          — 9,162 .crate files (4.2 GB) from crates.io
jars/            — 4,730 .jar files (5.0 GB) from Maven Central
wheels/          — 541 .whl files (5.0 GB) from PyPI
crates_archive.znippy  — packed with znippy 0.6.0
jars_archive.znippy    — packed with znippy 0.6.0
pip_archive.znippy     — packed with znippy 0.6.0

If znippy version changes, delete *.znippy and re-run integration_znippy_serve to rebuild.

Unit / Light Integration Tests

# All quick tests (no network, no containers):
cargo test --workspace

# Mannequin (mock Nexus) tests:
cargo test -p holger-agent-cli --test integration_holger -- --nocapture
cargo test -p holger-agent-cli --test integration_pip -- --nocapture

Requirements

• Podman (or Docker) for Nexus container tests — enable socket: systemctl --user start podman.socket
• Network access for downloading real artifacts from crates.io, Maven Central, PyPI
• ~15 GB disk for test cache

Fan art