Ling 3.1 Flash 1019 is a post-trained update of Ling 3.1 Flash, a hybrid-linear Mixture-of-Experts model for real-world agents, scientific reasoning, and coding. The model carries 560B total parameters and 25B active in a 72-layer stack with 4,096 hidden size, 64 attention heads, and 64 KV heads. Its 63 Kimi Delta Attention layers are interleaved with 9 gated MLA layers, and the sparse blocks select 8 of 512 routed experts alongside a shared expert. It supports a 1M-token context and includes a Multi-Token Prediction layer for agentic workloads.
Ling 3.1 Flash is a native hybrid reasoning Mixture-of-Experts model designed for general-purpose agents, office productivity, and software engineering. Building on Ling 3.0 Flash, it scales to 560B total parameters with 25B active, using a 72-layer architecture with 4,096 hidden size, 64 attention heads, and 64 KV heads. Kimi Delta Attention and gated MLA layers alternate in a 7:1 ratio, while MoE layers route each token to 8 of 512 experts plus a shared expert. Supporting a 1M-token context, it adds a Multi-Token Prediction layer for speculative decoding.
Ling 3.0 Flash Sante is a healthcare-specialized Mixture-of-Experts model built on Ling 3.0 Flash for medical reasoning, healthcare research, and tool-assisted workflows. It has 124B total parameters with 5.1B active, arranged in a 42-layer backbone with 2,560 hidden size, 32 attention heads, and 32 KV heads. Five Kimi Delta Attention layers alternate with one gated MLA layer, and each MoE layer routes tokens to 8 of 512 experts plus a shared expert. Supporting a 256K-token context, it runs in thinking mode by default with a Multi-Token Prediction layer.
Mistral Large 4 1T A52B is a natively multimodal Mixture-of-Experts model that unifies instruction following, reasoning, coding, and agentic workflows. It holds approximately 1T total parameters with 52B active, using a 72-layer architecture with 6,144 hidden size, 64 attention heads, and 64 KV heads. Sparse indexed attention runs in all layers, with rotary embeddings omitted every fourth layer, and each MoE layer activates 4 of 128 routed experts plus a shared expert. Supporting a 1M-token context, it adds a Pixtral-style vision encoder and an embedded DSpark draft head.
MiniMax M3.1 Flash is a native multimodal Mixture-of-Experts model for coding, agentic workflows, computer use, and cybersecurity across text, image, and video. Its approximately 428B parameters, 23B of them active per token, are arranged in a 60-layer architecture with 6,144 hidden size, 64 attention heads, and 4 KV heads. MiniMax Sparse Attention runs in every layer, while the MoE blocks route each token to 4 of 128 experts plus a shared expert. Supporting a 1M-token context, it combines quantization-aware training with a DSpark draft head for speculative decoding.
Step 5 Preview is a multimodal sparse Mixture-of-Experts model built for agentic software engineering, long-horizon tool use, and finance-focused knowledge work. The model carries 600B total parameters and 27B active per token across 92 layers, with a 4,096 hidden size, 64 attention heads, and 4 KV heads. Sliding-window and full-attention layers alternate in a 3:1 pattern, while each MoE layer activates 8 of 352 routed experts plus a shared expert. Supporting a 1M-token context, it adds a Perception Encoder for image input and three Multi-Token Prediction layers.
DeepSeek V4.1 Flash 1017 is an updated multimodal Mixture-of-Experts model for agentic coding, reasoning, and visual tasks, with shorter outputs and faster task execution. It has 763B total parameters with 8B active during prefill and 16B during decode, using a 40-layer Causal Encoder-Decoder with 5,120 hidden size, 64 attention heads, and 1 KV head. The architecture pairs Compressed Sparse Attention 2 with 384 routed experts and a shared expert, activating 6 per token. Supporting a 1M-token context, it adds a 32-layer vision encoder and DSpark speculative decoding.
Muse Spark Open 816B A42B is a natively multimodal Mixture-of-Experts model for coding agents, tool use, and reasoning over images, audio, and video. It features 816B total parameters with 42B active, using a 62-layer architecture with 8,192 hidden size, 128 attention heads, and 16 KV heads. Every layer routes tokens to 8 of 256 experts, while the attention stack interleaves 46 sliding-window layers with 16 full-attention layers. Supporting a 1M-token context, it adds a 1.8B-parameter Perception Encoder and an audio encoder, with adjustable reasoning effort for agentic workloads.
Nemotron 3.5 Super VL 120B A12B BF16
Nemotron 3.5 Super VL 120B A12B is a multimodal hybrid Mixture-of-Experts model built for reasoning, coding, tool use, and document and video understanding. The model has 120B total parameters and 12B active, spread across 88 layers with 4,096 hidden size, 32 attention heads, and 2 KV heads. Its stack interleaves Mamba-2, LatentMoE, and attention layers, with each MoE layer activating 22 of 512 experts plus a shared expert. A C-RADIO vision encoder processes images and video within a 1M-token context, while Multi-Token Prediction enables native speculative decoding.
Nemotron 3.5 Bolt 74B A9B BF16
Nemotron 3.5 Bolt 74B A9B is a hybrid vision-language Mixture-of-Experts model designed for local agentic tasks, coding, and visual reasoning. It features 74B total parameters with 9B active, using an 88-layer architecture with 4,096 hidden size, 32 attention heads, and 2 KV heads. The backbone interleaves Mamba-2, LatentMoE, and attention layers, with each MoE layer activating 9 routed experts and a shared expert per token. Compressed from Nemotron 3.5 Super VL, it adds a C-RADIO v4-H vision encoder and 256K-token context, with the BF16 checkpoint aimed at post-training.