Hy4 is a multimodal Mixture-of-Experts model built for computer use, frontend and game development, financial analysis, and scientific research. Its language backbone holds 770B total parameters with 49B active, arranged in a 78-layer architecture with 6,144 hidden size and 64 attention heads. Gated DeepSeek Sparse Attention selects the top 2,048 tokens through IndexCache, while each of the 77 MoE layers activates 8 of 256 routed experts plus a shared expert. Supporting a 1M-token context, it pairs a 40-layer vision encoder with a 5-layer DFly2 drafter for speculative decoding.
Model specifications and capabilities are published by the model author and reproduced here from HF Model Card (tencent/Hy4-FP8). Released under Tencent Hy License. Further reading: Blog
Choose your hardware and inference engine to get deployment commands and performance benchmarks tailored to your infrastructure
1,440 GB
Enter your email to get access to this content
Benchmarks measured by Vultr on B200 · vLLM. Throughput and latency vary with concurrency, input length and engine configuration, so treat these as a comparison baseline, not a service guarantee.