Latest Content
Qwen3 235B A22B Instruct 2507 icon

Qwen3 235B A22B Instruct 2507

NVIDIA
The Qwen3-235B-A22B-Instruct-2507 is the premier flagship Mixture-of-Experts (MoE) Causal Language Model in the Qwen3 family, built with 235B total parameters and 22B activated per forward pass. This version is optimized for high-quality instruction following, logical reasoning, and preference-aligned text generation, with substantial improvements in multilingual long-tail knowledge coverage and subjective task alignment. It supports a native 262,144-token context window and can be extended up to approximately 1,010,000 tokens, enabling large-scale document analysis, extended conversational sessions, and complex multi-source workflows.
TypeMoE LLM
CapabilitiesText Generation, Instruction Following, Reasoning, Mathematical Reasoning+5 more
Release DateApril 28, 2025
Links
Paper|Blog|HF Model Card
LicenseApache 2.0

Inference Instructions

Deploy and run this model on NVIDIA B200 GPUs using the command below. Copy the command to get started with inference.

CONSOLE
docker run --gpus all 
 --shm-size 128g 
 -p 8000:8000 
 -v ~/.cache/huggingface:/root/.cache/huggingface 
 -e HF_TOKEN='YOUR_HF_TOKEN' 
 --ipc=host 
 lmsysorg/sglang:v0.5.8-cu130 
 python3 -m sglang.launch_server 
 --model-path Qwen/Qwen3-235B-A22B-Instruct-2507 
 --host 0.0.0.0 
 --port 8000 
 --max-prefill-tokens 65536 
 --max-running-requests 1024 
 --tp 8 
 --ep 8 
 --mem-fraction-static 0.95 
 --attention-backend flashinfer 
 --trust-remote-code

Model Benchmarks

Each model was tested with a fixed input size and total token volume while increasing concurrency to measure serving performance under load.

ITL vs Concurrency

Time to First Token

Throughput Scaling

Total Tokens/sec vs Avg TTFT

Vultr Cloud GPU

NVIDIA HGX B200

Deploy NVIDIA B200 on Vultr Cloud GPU

Deploy NVIDIA B200 instancesContact Sales
How to Deploy Qwen3 235B A22B Instruct 2507 on NVIDIA GPUs | Vultr Docs