Crate ipfrs_interface 

IPFRS Interface - High-performance HTTP Gateway and API Layer

This crate provides a comprehensive API layer for IPFRS (InterPlanetary File & Reasoning System), offering multiple interfaces for accessing distributed content and computation.

Features

HTTP Gateway

• Kubo-compatible API (/api/v0/*) - Full compatibility with IPFS Kubo clients
• High-speed v1 API (/v1/*) - Optimized endpoints with batch operations
• Content gateway (/ipfs/{cid}) - Direct content retrieval with range requests
• Multi-range support - Efficient sparse downloads with HTTP 206

gRPC Interface

• BlockService - Raw block operations (Get, Put, Has, Delete, Batch)
• DagService - DAG operations (Get, Put, Resolve, Traverse)
• FileService - File operations (Add, Get, List, Pin)
• TensorService - Zero-copy tensor operations (Get, Slice, Stream)
• Streaming RPCs - Client, server, and bidirectional streaming
• Interceptors - Authentication, logging, metrics, rate limiting

WebSocket Support

• Real-time events - Block additions, peer connections, DHT queries
• Pub/sub system - Topic-based event subscriptions
• Connection management - Automatic cleanup and heartbeat

Advanced Features

• Zero-copy tensor API - Efficient ML model distribution via Safetensors
• Streaming uploads/downloads - Memory-efficient large file handling
• Batch operations - Parallel processing with atomic transactions
• Flow control - Adaptive window-based congestion control
• Resume/cancel - Robust transfer management

Security & Performance

• Authentication - JWT tokens, API keys, and OAuth2 (Authorization Code, Client Credentials, PKCE)
• Rate limiting - Token bucket algorithm with per-client limits
• CORS - Configurable cross-origin resource sharing
• Compression - Gzip, Brotli, and Deflate with tunable levels
• HTTP caching - ETag and Cache-Control for CDN optimization
• TLS/HTTPS - Production-ready SSL/TLS support

Quick Start

// Example usage (see examples/server.rs for a complete working example)
use ipfrs_interface::GatewayConfig;
use ipfrs_storage::blockstore::BlockStoreConfig;

#[tokio::main]
async fn main() -> Result<(), Box<dyn std::error::Error>> {
    // Create gateway with default configuration
    let storage_config = BlockStoreConfig::default();
    let mut gateway_config = GatewayConfig::default();
    gateway_config.storage_config = storage_config;
    gateway_config.listen_addr = "127.0.0.1:8080".to_string();

    // See /tmp/example_server.rs for complete implementation
    Ok(())
}

Examples

Upload a file (Kubo v0 API)

curl -X POST -F "file=@myfile.txt" http://localhost:8080/api/v0/add

Download via gateway

curl http://localhost:8080/ipfs/<CID>

Batch block operations (v1 API)

curl -X POST -H "Content-Type: application/json" \
  -d '{"cids":["<CID1>","<CID2>"]}' \
  http://localhost:8080/v1/block/batch/get

Get tensor slice (zero-copy)

curl "http://localhost:8080/v1/tensor/<CID>?slice=0:10,5:15"

Architecture

┌─────────────────┐
│  HTTP Clients   │ (curl, browsers, IPFS clients)
└────────┬────────┘
         │
┌────────▼────────┐
│  Axum Router    │ (HTTP/HTTPS)
├─────────────────┤
│  Middleware     │ (CORS, Auth, Rate Limit, Compression)
├─────────────────┤
│  API Handlers   │ (v0, v1, Gateway, WebSocket)
└────────┬────────┘
         │
┌────────▼────────┐
│  gRPC Server    │ (Tonic)
├─────────────────┤
│  Interceptors   │ (Auth, Logging, Metrics)
├─────────────────┤
│  Services       │ (Block, DAG, File, Tensor)
└────────┬────────┘
         │
┌────────▼────────┐
│  Core Layer     │ (ipfrs-core, ipfrs-storage, etc.)
└─────────────────┘

Performance

Target performance characteristics:

• Request latency: < 10ms (simple GET)
• Throughput: > 1GB/s (range requests)
• Concurrent connections: 10,000+
• Memory per connection: < 100KB

See Also

• Gateway - Main HTTP gateway implementation
• middleware - CORS, rate limiting, and compression
• streaming - Streaming operations and flow control
• websocket - WebSocket real-time events

Re-exports

pub use backpressure::BackpressureConfig;

pub use backpressure::BackpressureController;

pub use backpressure::BackpressureError;

pub use backpressure::BackpressurePermit;

pub use backpressure::BackpressureStream;

pub use binary_protocol::BinaryMessage;

pub use binary_protocol::ErrorResponse;

pub use binary_protocol::GetBlockRequest;

pub use binary_protocol::HasBlockRequest;

pub use binary_protocol::MessageType;

pub use binary_protocol::ProtocolError;

pub use binary_protocol::PutBlockRequest;

pub use binary_protocol::SuccessResponse;

pub use binary_protocol::PROTOCOL_VERSION;

pub use gateway::Gateway;

pub use gateway::GatewayConfig;

pub use graphql::create_schema;

pub use graphql::IpfrsSchema;

pub use middleware::cors_middleware;

pub use middleware::rate_limit_middleware;

pub use middleware::CacheConfig;

pub use middleware::CompressionConfig;

pub use middleware::CompressionLevel;

pub use middleware::CorsConfig;

pub use middleware::CorsState;

pub use middleware::RateLimitConfig;

pub use middleware::RateLimitState;

pub use mmap::MmapCache;

pub use mmap::MmapConfig;

pub use mmap::MmapError;

pub use mmap::MmapFile;

pub use oauth2::AccessToken;

pub use oauth2::AuthorizationCode;

pub use oauth2::CodeChallengeMethod;

pub use oauth2::ErrorResponse as OAuth2ErrorResponse;

pub use oauth2::GrantType;

pub use oauth2::OAuth2Client;

pub use oauth2::OAuth2ProviderConfig;

pub use oauth2::OAuth2Server;

pub use oauth2::RefreshToken;

pub use oauth2::ResponseType;

pub use oauth2::Scope;

pub use oauth2::TokenResponse;

pub use oauth2::TokenType;

pub use safetensors::SafetensorsFile;

pub use safetensors::SafetensorsHeader;

pub use safetensors::TensorData;

pub use safetensors::TensorInfo;

pub use streaming::ConcurrencyConfig;

pub use streaming::FlowControlConfig;

pub use streaming::FlowController;

pub use streaming::OperationState;

pub use streaming::OperationStatus;

pub use streaming::OperationType;

pub use streaming::ProgressEvent;

pub use streaming::ProgressTracker;

pub use streaming::ResumeToken;

pub use tensor::TensorLayout;

pub use tensor::TensorMetadata;

pub use tensor::TensorSlice;

pub use websocket::ws_handler;

pub use websocket::RealtimeEvent;

pub use websocket::SubscriptionManager;

pub use websocket::WsMessage;

pub use websocket::WsState;

pub use zerocopy::ZeroCopyBuffer;

Modules

arrow

Apache Arrow integration for zero-copy data exchange

auth

Authentication and Authorization Module

auth_handlers

Authentication API Handlers

backpressure

binary_protocol

Binary Protocol for High-Speed IPFRS Communication

ffi

FFI (Foreign Function Interface) bindings for C interoperability

gateway

HTTP Gateway for IPFRS

graphql

GraphQL API for IPFRS

metrics

Prometheus metrics for observability

metrics_middleware

Prometheus metrics middleware for Axum

middleware

HTTP Middleware for IPFRS Gateway

mmap

Memory-Mapped File Serving

oauth2

OAuth2 Authentication Module

python

Python bindings for IPFRS using PyO3

safetensors

Safetensors integration for tensor serialization

streaming

Streaming Support for IPFRS Gateway

tensor

Zero-Copy Tensor API

tls

TLS/SSL Configuration for HTTPS Support

websocket

WebSocket Support for Real-Time IPFRS Communication

zerocopy

Zero-copy buffer management