Xiaomi
MiMo V2.6 Pro is an ultra-large omnimodal Mixture-of-Experts model designed for coding agents, computer use, cybersecurity, and long-horizon tasks across text, image, video, and audio. It features 1.02T total parameters with 42B activated, using a 70-layer architecture with 6,144 hidden size, 128 attention heads, and 8 KV heads. The architecture interleaves 60 Sliding Window Attention layers with 10 Global Attention layers, using 384 routed experts with 8 activated per token. Supporting a 1M-token context, it integrates vision and audio encoders with Multi-Token Prediction for demanding agentic workloads.
Xiaomi
MiMo V2.6 Flash is a native omnimodal Mixture-of-Experts model designed for coding agents, computer use, cybersecurity, and long-horizon tasks across text, image, video, and audio. It features 309B total parameters with 15B activated, using a 48-layer architecture with 4,096 hidden size, 64 attention heads, and 8 KV heads. The architecture interleaves 39 Sliding Window Attention layers with 9 Global Attention layers, using 256 routed experts with 8 activated per token. Supporting a 1M-token context, it integrates vision and audio encoders with Multi-Token Prediction for efficient agentic inference.