Module semantic_dht 

Semantic DHT - Vector-based content routing

This module extends the standard Kademlia DHT with semantic routing capabilities, allowing content discovery based on vector embeddings and semantic similarity rather than just content-addressed hashes.

Features

• Embedding-based Routing: Map vector embeddings to DHT keys using locality-sensitive hashing
• Semantic Queries: Find content based on semantic similarity
• Distributed ANN: Approximate nearest neighbor search across the network
• Multiple Namespaces: Support different embedding spaces (text, image, etc.)
• Adaptive Routing: Learn from query results to improve routing decisions

Design

The semantic DHT uses a two-layer architecture:

1. Embedding Layer: Maps high-dimensional embeddings to DHT keys via LSH
2. Routing Layer: Routes queries using both XOR distance (Kademlia) and semantic distance

Each peer maintains:

• A local vector index for semantic search
• A mapping from embedding regions to peer IDs
• Statistics on query success rates per region

Structs

LshConfig

LSH configuration

LshHash

Locality-Sensitive Hash for embeddings

NamespaceId

Semantic namespace identifier

SemanticDht

Semantic DHT manager

SemanticDhtConfig

Configuration for semantic DHT operations

SemanticDhtStats

Statistics for semantic DHT

SemanticNamespace

Namespace configuration

SemanticQuery

Semantic DHT query

SemanticResult

Result from a semantic query

Enums

DistanceMetric

Distance metric for vector similarity

SemanticDhtError

Errors that can occur in semantic DHT operations