Skip to main content

Module deepseek

Module deepseek 

Source
Expand description

DeepSeek Model V2 - Clean implementation using solid abstractions

This implements the DeepSeek/DeepSeek-V2 architecture including:

  • DeepSeek-7B, DeepSeek-67B, DeepSeek-Coder, DeepSeek-Math
  • Uses unified Tensor type from tensor_core
  • Supports MoE (Mixture of Experts) architecture

Structsยง

DeepSeekAttention
DeepSeek attention mechanism
DeepSeekConfig
DeepSeekLayer
DeepSeek transformer layer
DeepSeekMLP
DeepSeek MLP (feed-forward network)
DeepSeekModelV2
Main DeepSeek model implementation