Crate haagenti_network 

CDN-Based Network Streaming for HoloTensor Fragments

This module implements intelligent network streaming for loading model fragments from CDN/edge locations with:

• Range Requests: Partial downloads for progressive loading
• Parallel Fetching: Concurrent chunk downloads
• Smart Caching: ETag/Last-Modified validation
• Retry with Backoff: Resilient to transient failures
• Priority Queue: Critical fragments first
• Bandwidth Adaptation: Adjusts to network conditions

Architecture

┌─────────────────────────────────────────────────────────────────┐
│                      NetworkLoader                               │
├─────────────────────────────────────────────────────────────────┤
│                                                                  │
│  ┌──────────────┐    ┌──────────────┐    ┌──────────────┐      │
│  │   Priority   │ -> │  Scheduler   │ -> │  Fetcher     │      │
│  │   Queue      │    │  (rate limit)│    │  (parallel)  │      │
│  └──────────────┘    └──────────────┘    └──────────────┘      │
│         ↑                    ↑                   ↓              │
│         │                    │            ┌──────────────┐      │
│   Importance           Bandwidth          │    Cache     │      │
│    Weights              Monitor           │  (disk+mem)  │      │
│                                           └──────────────┘      │
│                                                  ↓              │
│                                           ┌──────────────┐      │
│                                           │   Fragment   │      │
│                                           │   Library    │      │
│                                           └──────────────┘      │
└─────────────────────────────────────────────────────────────────┘

Modules

prelude

Prelude for common imports

Structs

BandwidthMonitor

Bandwidth monitor

CacheConfig

Cache configuration

CacheEntry

Cache entry metadata

CacheStats

Cache statistics

CdnEndpoint

CDN endpoint configuration

ClientConfig

Configuration for the HTTP client

FragmentCache

Two-tier fragment cache (memory + disk)

HttpClient

HTTP client for CDN requests

LoadRequest

Request to load a fragment

NetworkConfig

Network loader configuration

NetworkLoader

Network loader for fragment streaming

PrioritizedFragment

A fragment with associated priority

PriorityQueue

Thread-safe priority queue for fragments

RangeRequest

Range request for partial downloads

RetryConfig

Retry configuration

Scheduler

Download scheduler

SchedulerConfig

Scheduler configuration

StreamingLoader

Streaming loader for continuous fragment loading

Enums

LoadResult

Result of loading a fragment

NetworkError

Errors that can occur during network operations

Priority

Priority level for fragment loading

Type Aliases

Result

Result type for network operations