hexz-server

HTTP and NBD server for streaming Hexz snapshot data over the network.

Overview

hexz-server provides network-facing interfaces for accessing compressed Hexz snapshots via standard protocols. It enables remote access to snapshot data without requiring clients to download entire files, making it ideal for forensics, VM hosting, and distributed data access.

Supported Protocols

HTTP Range Server

Exposes snapshots via HTTP 1.1 with range request support (RFC 7233). Clients can fetch specific byte ranges using standard HTTP GET requests with Range headers.

NBD (Network Block Device)

Allows mounting snapshots as Linux block devices using the NBD protocol. This enables transparent filesystem access and use of standard tools (mount, dd, fsck).

S3 Gateway (Planned)

Future S3-compatible API for cloud integration (not yet implemented).

Quick Examples

Starting an HTTP Server

use std::sync::Arc;
use hexz_core::File;
use hexz_core::store::local::FileBackend;
use hexz_core::algo::compression::lz4::Lz4Compressor;
use hexz_server::serve_http;

#[tokio::main]
async fn main() -> anyhow::Result<()> {
    let backend = Arc::new(FileBackend::new("snapshot.hxz".as_ref())?);
    let compressor = Box::new(Lz4Compressor::new());
    let snap = Arc::new(File::new(backend, compressor, None)?);

    // Start HTTP server on port 8080
    serve_http(snap, 8080).await?;
    Ok(())
}

Starting an NBD Server

use std::sync::Arc;
use hexz_core::File;
use hexz_core::store::local::FileBackend;
use hexz_core::algo::compression::lz4::Lz4Compressor;
use hexz_server::serve_nbd;

#[tokio::main]
async fn main() -> anyhow::Result<()> {
    let backend = Arc::new(FileBackend::new("snapshot.hxz".as_ref())?);
    let compressor = Box::new(Lz4Compressor::new());
    let snap = Arc::new(File::new(backend, compressor, None)?);

    // Start NBD server on port 10809
    serve_nbd(snap, 10809).await?;
    Ok(())
}

Command-Line Usage

# Start HTTP server (via hexz CLI)
hexz sys serve --port 8080 snapshot.st

# Start NBD server
hexz sys serve --nbd --port 10809 snapshot.st

HTTP Server

Endpoints

• GET /disk - Serves the disk stream (persistent storage)
• GET /memory - Serves the memory stream (RAM snapshot)

Range Request Support

# Fetch first 4KB of disk stream
curl -H "Range: bytes=0-4095" http://localhost:8080/disk -o chunk.bin

# Fetch 1MB starting at offset 1MB
curl -H "Range: bytes=1048576-2097151" http://localhost:8080/memory -o chunk.bin

# Fetch from offset to EOF (clamped to 32 MiB)
curl -H "Range: bytes=1048576-" http://localhost:8080/disk -o large_chunk.bin

Response Codes

• 206 Partial Content - Successful range request
• 416 Range Not Satisfiable - Invalid range or out of bounds
• 500 Internal Server Error - Snapshot read failure

Python Client Example

import requests

# Fetch a range
headers = {'Range': 'bytes=0-4095'}
response = requests.get('http://localhost:8080/disk', headers=headers)
assert response.status_code == 206

# Parse Content-Range header
content_range = response.headers['Content-Range']
print(f"Content-Range: {content_range}")

data = response.content
print(f"Fetched {len(data)} bytes")

NBD Server

Client Usage (Linux)

# Connect NBD client to server
sudo nbd-client localhost 10809 /dev/nbd0

# Mount the block device (read-only)
sudo mount -o ro /dev/nbd0 /mnt/snapshot

# Access files normally
ls -la /mnt/snapshot
cat /mnt/snapshot/important.log

# Disconnect when done
sudo umount /mnt/snapshot
sudo nbd-client -d /dev/nbd0

Use Cases

• VM Hosting: Boot VMs directly from NBD-mounted snapshots
• Forensics: Mount disk images for analysis without full download
• File Access: Browse snapshot contents with standard Linux tools

Architecture

hexz-server/
├── src/
│   ├── lib.rs          # HTTP server implementation (Axum)
│   └── nbd.rs          # NBD protocol implementation

Key design decisions:

• Axum Framework: Modern async HTTP server with excellent performance
• Tokio Runtime: Async I/O for handling thousands of concurrent connections
• Localhost-Only: Binds to 127.0.0.1 by default for security
• DoS Protection: Request size clamping (32 MiB max per request)

Performance

HTTP Server

NBD Server

Performance is primarily limited by decompression CPU time, not network bandwidth.

Security

Current Security Posture

• Localhost-only: Binds to 127.0.0.1, not accessible from network
• No authentication: Anyone with local access can read snapshots
• No TLS: Plaintext protocols (acceptable for loopback)
• DoS protection: Request size clamping to prevent memory exhaustion

Remote Access

For remote access, use:

• SSH tunnel: ssh -L 8080:localhost:8080 user@server
• VPN: WireGuard, OpenVPN, etc.
• Reverse proxy: nginx with TLS + authentication

Future Enhancements (Planned)

• TLS/HTTPS support
• Bearer token authentication
• Rate limiting per IP
• Configurable bind addresses
• Audit logging

DoS Protection

Request Size Clamping

All HTTP requests are clamped to 32 MiB (MAX_CHUNK_SIZE) to prevent:

• Memory exhaustion from huge reads
• CPU exhaustion from excessive decompression

Example:

Client requests:  Range: bytes=0-67108863   (64 MiB)
Server clamps to: Content-Range: bytes 0-33554431/total  (32 MiB)

Clients must check the Content-Range header and issue follow-up requests for remaining data.

Protocol Limitations

HTTP Server

Supported:

• Bounded ranges: bytes=0-1023
• Unbounded ranges: bytes=1024-

Not supported:

• Suffix ranges: bytes=-1024 (last 1KB)
• Multi-part ranges: bytes=0-100,200-300

Returns HTTP 416 for unsupported range types.

NBD Server

Supported:

• Read operations
• Block-level access
• Multiple concurrent clients

Not supported:

• Write operations (always read-only)
• Trim/discard commands

Development

From the repository root:

# Build server crate
cargo build -p hexz-server

# Run tests
cargo test -p hexz-server

# Run HTTP server example
cargo run -p hexz-server --example http_server

Testing

# Unit tests
cargo test -p hexz-server

# Integration tests with actual servers
cargo test -p hexz-server --test integration

Examples

The crate includes example programs:

# HTTP server example
cargo run --example http_server -- snapshot.hxz 8080

# NBD server example
cargo run --example nbd_server -- snapshot.hxz 10809

Dependencies

• axum: HTTP server framework
• tokio: Async runtime
• tower: Middleware and utilities
• hexz-core: Core snapshot engine

See Also

• hexz-core - Core engine (provides File)
• hexz-cli - CLI tool (serve command)
• User Documentation - Server configuration guides
• Project README - Main project overview