Crate ipfrs 

IPFRS - Inter-Planet File RUST System

Version: 0.3.0 “The Fast & The Wise”

A next-generation distributed file system built in Rust, combining:

• High-performance zero-copy data transport
• Semantic routing and vector search
• TensorLogic integration for distributed reasoning
• ARM-optimized implementation
• Content-addressed storage with IPLD
• Distributed networking via libp2p
• Pin management and garbage collection
• Comprehensive observability (metrics, logging, tracing)

Architecture

IPFRS follows a bi-layer architecture:

Logical Layer (The Brain)

• Semantic Router: Vector search and logic solving for content discovery
• TensorLogic: Distributed reasoning and knowledge representation
• Differentiable Storage: Gradient tracking for learning

Physical Layer (The Body)

• TensorSwap: Optimized tensor streaming protocol
• Block Storage: Sled-backed content-addressed storage
• Network Stack: libp2p with QUIC, Bitswap, and DHT
• Pin Management: Protect important content from garbage collection

Features

Content-Addressed Storage

• Store and retrieve data by content hash (CID)
• IPLD support for structured data with links
• DAG operations for hierarchical data structures

Semantic Search

• Vector similarity search using HNSW index
• Filter-based queries
• Persistent index with save/load support

Logic Programming

• TensorLogic integration for facts, rules, and inference
• Backward chaining reasoning
• Proof generation and verification
• Knowledge base persistence

Networking

• Peer-to-peer content distribution
• DHT-based content discovery
• Bitswap protocol for block exchange
• NAT traversal and relay support

Pin Management

• Direct and recursive pinning
• Automatic DAG traversal for recursive pins
• Protect content from garbage collection

Observability

• Prometheus metrics
• Structured logging with tracing
• OpenTelemetry integration
• Health checks and status monitoring

Quick Start

Basic Node Setup

use ipfrs::{Node, NodeConfig};

#[tokio::main]
async fn main() -> ipfrs::Result<()> {
    // Create and start a node with default configuration
    let config = NodeConfig::default();
    let mut node = Node::new(config)?;
    node.start().await?;

    // Node is now running and ready to use
    println!("Node started successfully!");

    // Stop the node when done
    node.stop().await?;
    Ok(())
}

Storing and Retrieving Content

// Store content
let data = Bytes::from("Hello, IPFRS!");
let block = Block::new(data.clone())?;
let cid = *block.cid();
node.put_block(&block).await?;
println!("Stored content with CID: {}", cid);

// Retrieve content
let retrieved = node.get_block(&cid).await?;
assert!(retrieved.is_some());

Pin Management

// Pin content to prevent garbage collection
node.pin_add(&cid, false, Some("important-data".to_string())).await?;

// List all pins
let pins = node.pin_ls()?;
for pin in pins {
    println!("Pinned: {} ({:?})", pin.cid, pin.pin_type);
}

// Unpin when no longer needed
node.pin_rm(&cid, false).await?;

Semantic Search

// Index content for semantic search
let embedding = vec![0.1; 768]; // Your embedding vector
node.index_content(&cid, &embedding).await?;

// Perform similarity search
let query = vec![0.2; 768];
let results = node.search_similar(&query, 10).await?;
for result in results {
    println!("Found: {} (score: {})", result.cid, result.score);
}

Logic Programming

// Add facts
let fact = Predicate::new(
    "parent".to_string(),
    vec![
        Term::Const(Constant::String("alice".to_string())),
        Term::Const(Constant::String("bob".to_string())),
    ],
);
node.add_fact(fact)?;

// Perform inference
let query = Predicate::new(
    "parent".to_string(),
    vec![
        Term::Var("X".to_string()),
        Term::Const(Constant::String("bob".to_string())),
    ],
);
let results = node.infer(&query)?;
println!("Found {} results", results.len());

Re-exports

pub use node::BlockStat;

pub use node::DagExportStats;

pub use node::DagImportStats;

pub use node::FsckResult;

pub use node::GcResult;

pub use node::Node;

pub use node::NodeConfig;

pub use node::NodeStatus;

pub use node::SemanticStats;

pub use node::StorageStats;

pub use node::TensorLogicStats;

pub use auth::AuthManager;

pub use auth::AuthToken;

pub use auth::OAuth2Config;

pub use auth::Permission;

pub use auth::Role;

pub use auth::TokenType;

pub use auth::User;

pub use tls::CertificateInfo;

pub use tls::SelfSignedCertGenerator;

pub use tls::TlsConfig;

pub use tls::TlsError;

pub use tls::TlsManager;

pub use tls::TlsVersion;

pub use diagnostics::DiagnosticAnalyzer;

pub use diagnostics::DiagnosticRecommendation;

pub use diagnostics::HealthStatus;

pub use diagnostics::NetworkDiagnostics;

pub use diagnostics::NodeDiagnostics;

pub use diagnostics::RecommendationSeverity;

pub use diagnostics::ResourceUsage;

pub use diagnostics::SemanticDiagnostics;

pub use diagnostics::StorageDiagnostics;

pub use diagnostics::TensorLogicDiagnostics;

pub use fsck::FilesystemChecker;

pub use fsck::FsckConfig;

pub use fsck::FsckResult as FsckResultDetailed;

pub use gc::GarbageCollector;

pub use gc::GcConfig;

pub use gc::GcStats;

pub use pin::PinInfo;

pub use pin::PinManager;

pub use pin::PinType;

pub use profiler::OperationStats;

pub use profiler::Profiler;

pub use repo::format_bytes;

pub use repo::BlockDistribution;

pub use repo::RepoAnalyzer;

pub use repo::RepoStats;

Modules

auth

Authentication and Authorization for IPFRS

diagnostics

Diagnostic utilities for IPFRS nodes

fsck

Filesystem Check (fsck) - Repository Integrity Verification

gc

Garbage Collection System

health

Health check and liveness/readiness probes

metrics

Metrics collection and Prometheus integration

node

IPFRS Node - Unified node implementation

pin

Pin Management System

profiler

Performance Profiling Utilities

recovery

Error recovery mechanisms

repo

Repository Statistics and Utilities

shutdown

Graceful shutdown handling

tls

TLS/SSL Support for IPFRS

tracing_setup

Distributed tracing setup with OpenTelemetry

Structs

BitswapConfig

Bitswap exchange configuration

BitswapExchange

Bitswap exchange handler with enhanced want list and peer management

BitswapStats

Bitswap statistics

Block

A content-addressed data block.

BlockStoreConfig

Block store configuration

Gateway

HTTP Gateway server

GatewayConfig

Gateway server configuration

KnowledgeBase

A knowledge base containing facts and rules

KnowledgeBaseStats

Knowledge base statistics

NetworkConfig

Network configuration

NetworkNode

IPFRS network node

Predicate

A logical predicate

Proof

Proof tree representing the derivation of a goal

QueryFilter

Query filter for semantic search

RouterConfig

Configuration for semantic router

RouterStats

Router statistics

Rule

A logical rule (Horn clause)

SearchResult

Search result entry

SemanticRouter

Semantic router combining CID-based and vector-based search

SledBlockStore

Block storage using Sled embedded database

SubscriptionManager

Manages WebSocket subscriptions and pub/sub

TensorLogicStore

Storage manager for TensorLogic IR

TensorMetadata

Tensor metadata for smart scheduling

TensorSwap

TensorSwap protocol handler

TensorSwapConfig

TensorSwap configuration extending Bitswap config

TensorSwapStats

TensorSwap statistics

TermRef

Reference to a term via CID

VectorIndex

HNSW-based vector index for semantic search

WantEntry

Want entry with metadata for scheduling

WsState

WebSocket handler state

ZeroCopyBuffer

Zero-copy buffer for efficient data transfer

Enums

Constant

Constant value types

DistanceMetric

Distance metric for vector similarity

Error

Unified error type for IPFRS operations.

Ipld

IPLD data model

Message

Message type for block exchange protocol

RealtimeEvent

Real-time event types

Term

A logical term in TensorLogic

WsMessage

WebSocket message envelope

Traits

BlockStoreTrait

Trait for block storage backends

Type Aliases

Cid

A Cid that contains a multihash with an allocated size of 512 bits.

PeerId

Peer identifier

Result

Convenient result type for IPFRS operations.

Substitution

Variable substitution (bindings)