Expand description
Tauformer - A GPT implementation
Re-exports§
pub use checkpoint::load_checkpoint;pub use checkpoint::load_weights;pub use checkpoint::save_checkpoint;pub use checkpoint::save_weights;pub use backend::AutoBackend;pub use backend::get_device;pub use backend::print_backend_info;pub use config::NanoChatConfig;
Modules§
- backend
- Backend selection with automatic GPU detection for Burn 0.18
- causalattention
- NanoGPT with numerically stable attention and logits
- checkpoint
- Checkpoint save/load for NanoChat models
- config
- Configuration structures for NanoChat model and training
- engine
- prelude
- Prelude module for convenient imports
- pretraining
- Domain manifold (GraphLaplacian) management for Tauformer.
- rope
- sampling
- Sampling strategies for text generation
- tauattention
- Taumode attention: uses feature-space Laplacian to compress tokens into scalar lambdas, then scores via lambda-distance instead of softmax(QK).
- taugpt
- TauGPT: GPT-like transformer that uses TauModeAttention (lambda-distance attention) instead of dot-product causal attention.
- taumode
- Taumode utilities for Tauformer.
- tokenizer
- Tokenizer for NanoChat with special token support and chat templating Mirrors the Python tokenizer.py implementation