god-graph 0.6.0-alpha

A graph-based LLM white-box optimization toolbox: topology validation, Lie group orthogonalization, tensor ring compression
Documentation 
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17

//! Transformer layer implementations
//!
//! This module provides the core building blocks for Transformer models:
//! - Multi-Head Attention
//! - Feed-Forward Networks (including SwiGLU variant)
//! - Normalization layers (LayerNorm, RMSNorm)
//! - Positional encodings (RoPE)

pub mod attention;
pub mod ffn;
pub mod norm;
pub mod embedding;

pub use attention::MultiHeadAttention;
pub use ffn::FeedForward;
pub use norm::{LayerNorm, RMSNorm};
pub use embedding::RoPE;