StepFun Step 3.5 Flash

Step 3.5 Flash is a sparse Mixture-of-Experts (MoE) language model built for fast reasoning, coding, and agentic workflows. It has 196B total parameters with about 11B activated per token, using a 45-layer transformer with 4,096 hidden size and 64 attention heads. The architecture includes 288 experts per layer with top-8 routing and a shared expert, along with a hybrid attention pattern combining sliding window and full attention in a 3:1 ratio. It supports a 256K context window and integrates Multi-Token Prediction for high-throughput generation.