Module mixtral 

Mixtral Model V2 - Clean implementation using solid abstractions

This implements the Mixtral architecture which features:

• Mixture of Experts (MoE) with 8 experts, top-2 routing
• Sliding window attention (from Mistral)
• Grouped Query Attention (GQA)
• Uses unified Tensor type from tensor_core
• Implements Model trait from model_core

Structs

MixtralAttention

Mixtral attention mechanism with sliding window

MixtralConfig

MixtralExpert

Single expert in Mixtral MoE

MixtralLayer

Mixtral transformer layer with MoE

MixtralMoE

Mixtral Mixture of Experts layer

MixtralModelV2

Main Mixtral model implementation