Params: 309B
MiMo V2 Flash is a large-scale Mixture-of-Experts (MoE) language model developed by Xiaomi for high-speed reasoning, coding, and agentic workflows. The model features 309B total parameters with 15B activated, built on a 48-layer transformer architecture with 64 attention heads and a 4,096 hidden size. It utilizes a hybrid attention design that interleaves Sliding Window Attention and Global Attention in a 5:1 ratio, significantly reducing KV-cache memory while maintaining long-context performance. The model supports up to a 256K token context window and integrates Multi-Token Prediction (MTP) to accelerate generation throughput.
Text GenerationInstruction Following+7
