Crate haagenti_fragments 

Cross-Model Fragment Sharing Library

This module implements a shared fragment library that enables:

• Deduplication of similar tensor fragments across models
• Single-load-multi-use for shared attention patterns
• Significant storage reduction (30-50% for model families)
• Faster model switching by reusing cached fragments

Key Concepts

• Fragment: A compressed tensor chunk (typically 64KB-1MB)
• Signature: Locality-sensitive hash for similarity detection
• Library: Collection of unique fragments with reference tracking
• Manifest: Per-model mapping from layer names to fragment refs

Architecture

┌─────────────────────────────────────────────────────────────┐
│                    Fragment Library                          │
├─────────────────────────────────────────────────────────────┤
│  FragmentId → Fragment (deduplicated storage)               │
│  ┌────────┐ ┌────────┐ ┌────────┐ ┌────────┐               │
│  │ frag_0 │ │ frag_1 │ │ frag_2 │ │ frag_3 │  ...          │
│  └────────┘ └────────┘ └────────┘ └────────┘               │
└─────────────────────────────────────────────────────────────┘
                   ↑           ↑           ↑
           ┌──────┴───┐   ┌───┴───┐   ┌───┴───┐
           │  SDXL    │   │ SD3.5 │   │ Flux  │
           │ manifest │   │manifest│   │manifest│
           └──────────┘   └───────┘   └───────┘

Modules

prelude

Prelude for common imports

Structs

Fragment

A compressed tensor fragment

FragmentId

Unique identifier for a fragment

FragmentLibrary

Fragment library for cross-model storage

FragmentMetadata

Metadata about a fragment

FragmentSignature

Locality-sensitive hash signature for a fragment

LayerMapping

Mapping of layer names to tensor references

LibraryConfig

Library configuration

LibraryStats

Library statistics

ModelManifest

Model manifest containing all layer-to-fragment mappings

SignatureConfig

Configuration for signature computation

SimilarityIndex

Similarity index using Locality-Sensitive Hashing

SimilarityMatch

A match found in the similarity index

SimilarityThreshold

Threshold configuration for similarity matching

TensorRef

Reference to a fragment for a specific tensor

Enums

FragmentError

Errors that can occur during fragment operations

FragmentType

Type of fragment based on neural network layer type

Constants

DEFAULT_SIMILARITY_THRESHOLD

Default similarity threshold for fragment deduplication

LIBRARY_FORMAT_VERSION

Version of the fragment library format

Type Aliases

Result

Result type for fragment operations