Latest Content
Qwen3 30B A3B Instruct 2507 icon

Qwen3 30B A3B Instruct 2507

NVIDIA
The Qwen3-30B-A3B-Instruct-2507 is an instruction-aligned Mixture-of-Experts (MoE) Causal Language Model built on a 48-layer transformer architecture with 29.9B non-embedding parameters, designed for efficient large-scale generation under sparse activation. It activates approximately 3.3B parameters per forward pass from a total parameter count of 30.5B, enabling efficient inference while retaining strong general-purpose performance. With native support for a 262,144-token context window, it is well-suited for long-document processing, extended conversational contexts, and large-context instruction workloads.
TypeMoE LLM
CapabilitiesText Generation, Instruction Following, Reasoning, Mathematical Reasoning+5 more
Release DateApril 28, 2025
Links
Paper|Blog|HF Model Card
LicenseApache 2.0

Inference Instructions

Deploy and run this model on NVIDIA B200 GPUs using the command below. Copy the command to get started with inference.

CONSOLE
docker run --gpus all 
 --shm-size 128g 
 -p 8000:8000 
 -v ~/.cache/huggingface:/root/.cache/huggingface 
 -e HF_TOKEN='YOUR_HF_TOKEN' 
 --ipc=host 
 lmsysorg/sglang:v0.5.8-cu130 
 python3 -m sglang.launch_server 
 --model-path Qwen/Qwen3-30B-A3B-Instruct-2507 
 --host 0.0.0.0 
 --port 8000 
 --max-prefill-tokens 65536 
 --max-running-requests 1024 
 --tp 8 
 --ep 8 
 --mem-fraction-static 0.95 
 --attention-backend flashinfer 
 --trust-remote-code

Model Benchmarks

Each model was tested with a fixed input size and total token volume while increasing concurrency to measure serving performance under load.

ITL vs Concurrency

Time to First Token

Throughput Scaling

Total Tokens/sec vs Avg TTFT

Vultr Cloud GPU

NVIDIA HGX B200

Deploy NVIDIA B200 on Vultr Cloud GPU

Deploy NVIDIA B200 instancesContact Sales
How to Deploy Qwen3 30B A3B Instruct 2507 on NVIDIA GPUs | Vultr Docs