Expand description
StarCoder Model V2 - Clean implementation
StarCoder architecture features:
- Multi-Query Attention (MQA) - single KV head
- GPT-2 style architecture with learned position embeddings
- Fill-in-the-Middle (FIM) token support
- Larger context windows