# trustformers-core
Core infrastructure crate providing fundamental abstractions and utilities for the TrustformeRS ecosystem.
## Current State
**Version 0.2.0 — Stable (2026-07-02)**
This crate is **stable and production-ready**, serving as the foundation for all other TrustformeRS components. It provides high-performance tensor operations, layer implementations, and advanced optimization techniques. ~2,353 tests pass for this crate specifically, with zero stubs or unimplemented items, and zero clippy/rustdoc warnings workspace-wide.
## Features
### Tensor Operations
- **Comprehensive tensor abstraction** supporting multiple backends
- **SciRS2 integration** for SIMD-optimized operations
- **GPU support** through multiple backends (CUDA, Metal, Vulkan, WebGPU, OpenCL, ROCm, OneAPI, XLA, RISC-V)
- **Automatic differentiation** with reverse-mode and forward-mode autodiff
- **Memory-efficient operations** with zero-copy views
### Layer Implementations
- **Core Layers**: Linear, Embedding, LayerNorm, Dropout, RMSNorm
- **Attention Mechanisms**:
- Multi-head attention (MHA) with causal masking
- FlashAttention and FlashAttention-2 for memory efficiency
- Multi-Query Attention (MQA) and Grouped-Query Attention (GQA)
- PagedAttention for KV cache management
- Optimized SDPA kernels with adaptive strategies
- **Advanced Layers**: FeedForward (SwiGLU, GeGLU), PositionalEncoding, RoPE, ALiBi
### Performance Optimizations
- **SIMD Operations**: Optimized LayerNorm, Softmax, and RoPE implementations
- **Quantization Support**: INT8, INT4, FP16, FP8, GPTQ, AWQ, GGUF/K-quants with calibration
- **Custom Kernels**: Fused operations for reduced memory bandwidth
- **Kernel Tuning**: Automatic hardware-aware kernel parameter optimization
- **Memory Management**: Adaptive pooling with LRU, LFU, ARC, and Hybrid eviction policies
- **Conv2D**: Full im2col+matmul implementation with groups and dilation
- **GPU Attention**: Scaled dot-product and flash attention (tiled online-softmax)
### Export and Interoperability
- **ONNX Export**: Complete graph construction and runtime support
- **GGML/GGUF**: Advanced quantization formats including K-quants for edge deployment
- **CoreML**: iOS/macOS deployment support
### Advanced Features
- **Evaluation Framework**: GLUE, SuperGLUE, MMLU, HellaSwag, HumanEval benchmarks
- **Monitoring**: TensorBoard integration, gradient flow analysis, activation statistics
- **Caching System**: Multiple eviction policies (LRU, LFU, ARC, Hybrid)
- **A/B Testing**: Infrastructure for model comparison
- **Model Compression**: Pruning and distillation support
- **Plugin System**: Extensible architecture for custom kernels and layers
- **Tensor Debugger**: Interactive watchpoints, NaN/Inf detection, operation tracing
### Distributed and Parallel Computing
- **Tensor Parallelism**: Column/row parallel linear layers
- **Pipeline Parallelism**: Stage-based model partitioning
- **Data Parallelism**: Multi-GPU training infrastructure
- **Communication Backends**: NCCL, MPI, Gloo support
- **Process Groups**: All-reduce, broadcast, all-gather, reduce-scatter operations
### PEFT (Parameter-Efficient Fine-Tuning)
- **LoRA**: Low-rank adaptation with weight merging
- **QLoRA**: Quantized LoRA for memory efficiency
- **Adapters**: Bottleneck adapter layers
- **Prefix Tuning**: Trainable prefix embeddings
- **Prompt Tuning**: Virtual token optimization
## Architecture
```
trustformers-core/
├── src/
│ ├── tensor/ # Tensor abstractions and operations
│ ├── layers/ # Neural network layers
│ ├── attention/ # Attention mechanisms
│ ├── quantization/ # Quantization infrastructure (largest module by API surface)
│ ├── export/ # Model export formats (ONNX, GGUF, CoreML)
│ ├── kernels/ # Custom fused and SIMD compute kernels
│ ├── hardware/ # Hardware acceleration abstraction (CUDA, Metal, Vulkan, ROCm, ...)
│ ├── compiler/ # JIT compilation, kernel fusion, graph optimization
│ ├── performance/ # Benchmarking, profiling, and optimization advisor
│ ├── monitoring/ # Profiling and analysis
│ ├── parallel/ # Distributed computing
│ ├── evaluation/ # Benchmark implementations
│ └── peft.rs # Parameter-efficient fine-tuning
```
## Usage Example
```rust
use trustformers_core::layers::{Linear, LayerNorm, MultiHeadAttention};
use trustformers_core::tensor::Tensor;
use trustformers_core::traits::Layer;
fn main() -> Result<(), Box<dyn std::error::Error>> {
// Create layers for a transformer block
let attention = MultiHeadAttention::new(768, 12, 0.1, true)?;
let norm1 = LayerNorm::new(vec![768], 1e-5)?;
let ffn = Linear::new(768, 3072, true);
let norm2 = LayerNorm::new(vec![768], 1e-5)?;
// Run the attention sub-layer's forward pass
let input = Tensor::randn(&[2, 128, 768])?;
let attended = attention.forward(input)?;
Ok(())
}
```
## Performance
- **FlashAttention**: O(N) memory complexity vs O(N²) standard
- **Quantization**: 50-75% memory reduction with INT8/INT4
- **SIMD**: 2-3x speedup on supported operations
- **PagedAttention**: Eliminates KV cache fragmentation
## Testing
The crate includes comprehensive test coverage:
- **~2,353 unit and integration tests, all passing** (this crate's approximate share of a workspace-wide run completed 2026-07-01: 18,102 passed / 0 failed / 119 skipped via `cargo nextest run --workspace --all-features`, plus 0 clippy warnings and 0 rustdoc warnings)
- Property-based testing with proptest
- Memory leak detection
- Performance benchmarks
- Cross-backend compatibility tests
- Numerical stability tests with adaptive tolerance
## Dependencies
- `scirs2-core`: SIMD operations and parallelism
- `half`: FP16/BF16 support
- `rayon`: Parallel iteration (via SciRS2)
- Various serialization and utility crates
## Public API
The crate exposes **~4,533 public API items** (structs, enums, traits, and fns across the public API, including impl blocks — broader than a prior doc revision's narrower top-level-only count) covering tensors, layers, attention, quantization, export, evaluation, monitoring, distributed computing, PEFT, kernel tuning, and memory management.
## License
Apache-2.0