tenflowers-core 0.1.1

Core tensor operations and execution engine for TenfloweRS
Documentation 
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39

//! Normalization Operations
//!
//! This module provides comprehensive normalization operations for deep learning,
//! including batch normalization, layer normalization, group normalization,
//! and synchronized batch normalization. All operations support both CPU and GPU
//! execution with optimized implementations for different tensor layouts.
//!
//! # Features
//! - Batch Normalization (BatchNorm) for CNN layers
//! - Layer Normalization (LayerNorm) for transformer architectures
//! - Group Normalization (GroupNorm) for stable training with small batch sizes
//! - Synchronized Batch Normalization for distributed training
//! - GPU-optimized implementations with memory coalescing
//! - Training and inference modes with running statistics
//!
//! # Architecture
//! Each normalization type is implemented in its own module with both CPU and GPU
//! variants, providing optimal performance across different hardware configurations.
//!
//! # Modules
//! - [`mod@batch_norm`] - Standard batch normalization operations
//! - [`mod@layer_norm`] - Layer normalization for sequence models
//! - [`mod@group_norm`] - Group normalization for stable training
//! - [`mod@sync_batch_norm`] - Synchronized batch normalization for distributed training

// Re-export specialized modules
pub mod batch_norm;
pub mod group_norm;
pub mod layer_norm;
pub mod sync_batch_norm;

#[cfg(test)]
pub mod tests;

// Re-export core functions for backward compatibility
pub use batch_norm::batch_norm;
pub use group_norm::group_norm;
pub use layer_norm::layer_norm;
pub use sync_batch_norm::sync_batch_norm;