Expand description
TPT Model Optimizer — hardware-aware LLM compression pipeline.
§Pipeline
GGUF source
→ HardwareProfiler (benchmark GPU; cache to ~/.tpt/hardware_profile.json)
→ LayerSensitivityMap (rank layers by quantization sensitivity)
→ DomainMapper (identify neurons by domain via activation analysis)
→ SurgicalPruner (zero unwanted domain neurons)
→ MixedPrecisionAllocator (5% loss frontier per layer)
→ KvCacheCalculator (recommend context window for remaining VRAM)
→ TptfFormat writer (output .tptf — self-contained: weights + tokenizer + chat template)
Export: GgufExporter, Exl2Exporter
Quality: QualityBenchmark (perplexity + task accuracy)
Large models: StreamingLoader (70B+ via layer-by-layer mmap)Re-exports§
pub use activation_capture::load_domain_maps_from_dir;pub use activation_capture::ActivationCallback;pub use activation_capture::ActivationCapture;pub use activation_capture::ActivationMap;pub use activation_capture::LayerActivations;pub use benchmark::QualityBenchmark;pub use calibration::CalibrationGenerator;pub use domain_mapper::DomainMapper;pub use export::detect::ModelFormat;pub use import::gguf::GgufImporter;pub use import::gguf::GgufModel;pub use kv_calculator::KvCacheCalculator;pub use profiler::HardwareProfiler;pub use pruner::SurgicalPruner;pub use quant_allocator::MixedPrecisionAllocator;pub use sensitivity::LayerSensitivityMap;pub use sensitivity::SensitivityConfig;pub use streaming::StreamingLoader;pub use tptf_format::TensorBlock;pub use tptf_format::TptfHeader;pub use tptf_format::TptfWriter;
Modules§
- activation_
capture - Activation Capture — hooks for capturing intermediate activations during inference.
- benchmark
- Quality Benchmark — measures perplexity and task accuracy before/after optimization.
- calibration
- Calibration Generator — produces hard domain-specific prompts for quality evaluation.
- domain_
mapper - Domain Knowledge Mapper — identifies which neurons handle specific domains.
- export
- import
- GGUF → TPTF import pipeline.
- kv_
calculator - KV Cache Calculator — computes maximum context window from remaining VRAM.
- profiler
- Hardware profiler — benchmarks the target GPU and caches results.
- pruner
- Surgical Pruner — zeros FFN neurons associated with unwanted domains.
- quant_
allocator - Mixed-Precision Allocator — finds minimum bits per layer within quality budget.
- sensitivity
- Layer sensitivity map — fast pre-pass to rank layers by quantization sensitivity.
- streaming
- Streaming Loader — processes 70B+ models layer-by-layer via mmap.
- tptf_
format - TPTF file format — TPT’s self-contained model format.
Structs§
- Optimization
Result - Summary produced after a full optimization run.
- Optimizer
Config - Configuration for a full optimization run.