Expand description
DeepSeek Model V2 - Clean implementation using solid abstractions
This implements the DeepSeek/DeepSeek-V2 architecture including:
- DeepSeek-7B, DeepSeek-67B, DeepSeek-Coder, DeepSeek-Math
- Uses unified Tensor type from tensor_core
- Supports MoE (Mixture of Experts) architecture
Structsยง
- Deep
Seek Attention - DeepSeek attention mechanism
- Deep
Seek Config - Deep
Seek Layer - DeepSeek transformer layer
- Deep
SeekMLP - DeepSeek MLP (feed-forward network)
- Deep
Seek Model V2 - Main DeepSeek model implementation