
Tencent
Hy4 is a multimodal Mixture-of-Experts model built for computer use, frontend and game development, financial analysis, and scientific research. Its language backbone holds 770B total parameters with 49B active, arranged in a 78-layer architecture with 6,144 hidden size and 64 attention heads. Gated DeepSeek Sparse Attention selects the top 2,048 tokens through IndexCache, while each of the 77 MoE layers activates 8 of 256 routed experts plus a shared expert. Supporting a 1M-token context, it pairs a 40-layer vision encoder with a 5-layer DFly2 drafter for speculative decoding.

Tencent
Hy4 preview is a flagship Mixture-of-Experts model designed for advanced reasoning, coding, agentic workflows, and efficient long-context processing. It features 770B total parameters with 49B activated per token, using 78 layers with 6,144 hidden size and 64 attention heads. The architecture combines Gated DeepSeek Sparse Attention with IndexCache, 256 routed experts and 1 shared expert, activating 8 routed experts per token, while iHC expands residual information flow across 4 streams. It includes a native Multi-Token Prediction layer for speculative decoding and supports up to a 1M-token context window.

Tencent
Hy4 Flash is a cost-efficient multimodal Mixture-of-Experts model aimed at deep search, document processing, slide creation, and computer use. It carries 220B total parameters, including 64B of Engram n-gram embeddings, with 12B active per token across a 42-layer backbone of 3,072 hidden size. Twelve sparse attention layers interleave with 30 sliding-window layers, and LatentMoE routes each token to 16 of 512 experts plus a shared expert. Self-distilled from Hy4, it supports a 1M-token context with a SigLIP2-based vision encoder and a DFly2 drafter for speculative decoding.