Skip to main content

Module phi

Module phi 

Source
Expand description

Phi Model V2 - Clean implementation using solid abstractions

This implements the Phi architecture (Phi-1, Phi-2, Phi-3) which includes:

  • Microsoft Phi-1/1.5/2/3 models
  • Uses unified Tensor type from tensor_core
  • Implements Model trait from model_core

Structsยง

PhiAttention
Phi attention mechanism
PhiConfig
PhiLayer
Phi transformer layer
PhiMLP
Phi MLP (feed-forward network) - standard 2-layer MLP, not SwiGLU
PhiModelV2
Main Phi model implementation