Deploy and optimize large language models on NVIDIA HGX B200 GPUs with vLLM. This cookbook provides tested configurations, benchmark data, and optimization strategies for serving frontier models on NVIDIA HGX B200 infrastructure.
8 GPUs per node
per GPU 1.43 TB Total
per GPU throughput
per GPU
1.8 TB/s bidirectional
per GPU
Verified version
Verified version
This cookbook benchmarks five models that represent the architectural diversity of frontier open-source LLMs in early 2026. Each model uses a different attention mechanism and stresses the NVIDIA HGX B200's hardware subsystems differently.
Four of the five models use Mixture-of-Experts (MoE) routing: this reflects the current state of open-source LLMs. Llama 3.1 405B (July 2024) remains the only large dense open-source model; every successor (Llama 4, DeepSeek V3, GLM-5, Qwen3, Kimi K2.5) uses MoE. The exception is Nemotron Super 49B, a dense NAS-optimized transformer included for its compatibility with NVIDIA Dynamo's disaggregated serving.
What varies across our five models is more interesting than the dense/MoE distinction: five different attention mechanisms (Mamba SSM, standard MHA, Lightning linear attention, Differential Sparse Attention, Multi-Latent Attention), five different KV cache profiles, and active parameter counts spanning 3B to 49B.
