Expand description
Microsoft Phi model implementation
The Phi series are decoder-only transformers designed for code and language tasks.
Key characteristics:
-
Decoder-only transformer architecture
-
RoPE embeddings
-
Layer normalization
-
QK normalization
-
🤗 HF Link