Latest Content
Qwen3.5 122B A10B icon

Qwen3.5 122B A10B

NVIDIA
Qwen3.5-122B-A10B is a multimodal Mixture-of-Experts (MoE) large language model optimized for long-context reasoning, vision-language understanding, and scalable agentic workflows. It contains 122B total parameters with 10B active parameters, activating 8 routed experts plus 1 shared expert from 256 experts per token. The model employs a 48-layer hybrid architecture combining Gated DeltaNet and full attention layers, with 32 attention heads and a 3,072 hidden size. It supports a 262K token context window, extendable to ~1M, and integrates a 27-layer vision encoder for unified multimodal processing.
TypeVision-Language Model
CapabilitiesText Generation, Instruction Following, Reasoning, Mathematical Reasoning+6 more
Paper/Blog
Blog|HF Model Card
LicenseApache 2.0

Inference Instructions

Deploy and run this model on NVIDIA B200 GPUs using the command below. Copy the command to get started with inference.

CONSOLE
docker run -it --rm 
 --runtime=nvidia 
 --gpus all 
 --ipc=host 
 --shm-size=128g 
 -p 8000:8000 
 -v ~/.cache/huggingface:/root/.cache/huggingface 
 -e VLLM_USE_FLASHINFER_MOE_FP16=1 
 -e VLLM_FLASHINFER_MOE_BACKEND=latency 
 -e HF_TOKEN='YOUR_HF_TOKEN' 
 vllm/vllm-openai:v0.17.1 
 Qwen/Qwen3.5-122B-A10B 
  --tensor-parallel-size 8 
 --mm-encoder-tp-mode data 
 --max-model-len auto 
 --max-num-batched-tokens 65536 
 --gpu-memory-utilization 0.95 
 --enable-expert-parallel 
 --tool-call-parser qwen3_coder 
 --reasoning-parser qwen3 
 --enable-auto-tool-choice 
 --max-num-seqs 1024 
 --trust-remote-code
Note

To use VLLM_USE_FLASHINFER_MOE_FP16=1 and VLLM_FLASHINFER_MOE_BACKEND=latency for more optimal performance, need to use expert parallelism.

Model Benchmarks

Each model was tested with a fixed input size and total token volume while increasing concurrency to measure serving performance under load.

ITL vs Concurrency

Time to First Token

Throughput Scaling

Total Tokens/sec vs Avg TTFT

Vultr Cloud GPU

NVIDIA HGX B200

Deploy NVIDIA B200 on Vultr Cloud GPU

Deploy NVIDIA B200 instancesContact Sales
How to Deploy Qwen3.5 122B A10B on NVIDIA GPUs | Vultr Docs