Expand description
Mistral Model V2 - Clean implementation using solid abstractions
This implements the Mistral architecture which is used by:
- Mistral-7B, Mistral-7B-Instruct
- Key features: Sliding window attention, GQA
- Uses unified Tensor type from tensor_core
- Implements Model trait from model_core
- Supports loading via weight_loader_core
Structsยง
- Mistral
Attention - Mistral attention mechanism with sliding window
- Mistral
Config - Mistral
Layer - Mistral transformer layer
- MistralMLP
- Mistral MLP (feed-forward network)
- Mistral
Model V2 - Main Mistral model implementation