kontor-crypto 0.1.6

Kontor Proof-of-Retrievability system for decentralized storage
Documentation

Kontor Proof-of-Retrievability (PoR)

Crates.io CI License: MIT

⚠️ WARNING: This code is unaudited and experimental. Use at your own risk.

This project implements a Proof-of-Retrievability (PoR) system designed to provide economically enforceable guarantees that Storage Nodes are actually storing the data they have committed to. It is a core component for decentralized storage metaprotocols where a network of Indexers (Verifiers) must continuously audit Storage Nodes (Provers).

The system uses Nova recursive SNARKs via Microsoft's nova-snark library to generate constant-size (~10 kB) cryptographic proofs that a prover possesses a specific set of data. These proofs are efficient to verify (~30ms), making it feasible to enforce storage guarantees at scale.

Core Capabilities

  • Partition files into fixed 31-byte symbols for direct field element encoding.
  • Apply multi-codeword Reed-Solomon (GF(2^8)) for fault tolerance.
  • Generate Poseidon Merkle trees over all symbols (data + parity).
  • Create recursive SNARKs proving possession of randomly sampled symbols.
  • Compress proofs to constant ~10 kB size regardless of file count or challenge count.
  • Support dynamic circuit parameters with in-memory caching.
  • Reconstruct original files from partial symbol availability (≥90% per codeword).

Performance Characteristics

  • Proof Size: ~10 kB (constant across challenge count and file set within a shape).
  • Verification Time: ~50 ms for compressed SNARK verification.
  • Proving Time: Approximately linear in the number of recursive steps.

API Reference

The high-level API centers on a PorSystem object, which consolidates setup, proving, and verification concerns.

Core Interface

// Construction
struct PorSystem<'a> { /* holds reference to FileLedger */ }

impl<'a> PorSystem<'a> {
    pub fn new(ledger: &'a FileLedger) -> Self;

    // Prepares a file for proving. Symbol size (31 bytes) and codeword structure (231+24)
    // are fixed by protocol constants in config.rs.
    pub fn prepare_file(
        &self,
        data: &[u8],
        filename: &str,
    ) -> Result<(PreparedFile, FileMetadata)>;

    // Generates a single compact proof for any set of open Challenges.
    pub fn prove(
        &self,
        files: Vec<&PreparedFile>,
        challenges: &[Challenge],
    ) -> Result<Proof>;

    // Verifies a proof against the Challenges it claims to answer.
    pub fn verify(
        &self,
        proof: &Proof,
        challenges: &[Challenge],
    ) -> Result<bool>;
}

Usage Example

A complete example demonstrating the API workflow:

use kontor_crypto::api::{
    prepare_file, Challenge, FieldElement, PorSystem,
    tree_depth_from_metadata,
};
use kontor_crypto::FileLedger;

// 1. Prepare the file with fixed Reed-Solomon encoding (231 data + 24 parity symbols per codeword)
let my_data = b"This is a test file for the PoR system.";
let (prepared_file, metadata) = prepare_file(my_data, "test.dat").unwrap();

// 2. Create ledger and add the file
let mut ledger = FileLedger::new();
ledger.add_file(&metadata).unwrap();

// 3. Create PorSystem and challenge
let system = PorSystem::new(&ledger);
let num_challenges = 5;
let seed = FieldElement::from(12345u64); // Deterministic seed
let challenge = Challenge::new(metadata.clone(), 1000, num_challenges, seed, String::from("node_1"));

// 4. Generate proof using the unified API
let files = vec![&prepared_file];
let proof = system.prove(files, &[challenge.clone()]).unwrap();

// 5. Verify the proof
let is_valid = system.verify(&proof, &[challenge]).unwrap();
assert!(is_valid, "Proof verification failed!");

println!("Proof successfully generated and verified with Nova PoR API.");

CLI & Simulation

The project includes a CLI that simulates storage node operations with heterogeneous file sizes, staggered challenges, and multi-file proof aggregation.

Usage

# Default: small demo (100 files in ledger, node stores 10, 5 challenges)
cargo run

# Large-scale test with memory profiling
cargo run --features memory-profiling -- \
  --total-files-in-ledger 1000 \
  --files-stored-by-node 100 \
  --challenges-to-simulate 20 \
  --profile-memory

Flags

  • --total-files-in-ledger <N>: Network size (default: 100).
  • --files-stored-by-node <N>: Files this node stores (default: 10).
  • --challenges-to-simulate <N>: Challenges to batch (default: 5).
  • --file-size-distribution <TYPE>: "uniform", "mixed", or "large-heavy" (default: mixed).
  • --no-verify: Skip verification phase.
  • --profile-memory: Track peak memory usage.
  • -v, -vv: Increase verbosity (debug/trace).

Benchmark Suite

Run performance benchmarks with statistical analysis and CI integration via CodSpeed:

# Run all benchmarks locally
cargo bench

# For CI/CD integration with CodSpeed (optional):
cargo install cargo-codspeed --locked
cargo codspeed build
cargo codspeed run

Development

Test Suite

Run the extensive unit and integration test suite:

cargo install cargo-nextest
cargo nextest run

Git Hooks

Enable the pre-push hook to automatically run formatting, clippy, tests, and security audits:

git config core.hooksPath .githooks

Errors and Failure Modes

Key error variants surfaced at API boundaries (see KontorPoRError):

  • InvalidInput, InvalidChallengeCount, ChallengeMismatch (e.g., non-uniform num_challenges across the batch).
  • FileNotFound, FileNotInLedger, MetadataMismatch.
  • MerkleTree, Circuit, Snark.
  • Serialization, IO.

Documentation