Latest Content
Qwen3 4B Thinking 2507 icon

Qwen3 4B Thinking 2507

NVIDIA
The Qwen3-4B-Thinking-2507 is a reasoning-specialized Causal Language Model built on a 36-layer architecture with 3.6B non-embedding parameters. This updated model operates only in thinking mode and is optimized to generate explicit reasoning traces as part of its output, making it suitable for complex analytical tasks that require deeper logical decomposition. It supports a native 262,144-token context window, enabling extended deliberation over long documents, large problem states, and multi-stage reasoning workflows.
TypeDense LLM
CapabilitiesText Generation, Instruction Following, Reasoning, Mathematical Reasoning+5 more
Release DateApril 28, 2025
Links
Paper|Blog|HF Model Card
LicenseApache 2.0

Inference Instructions

Deploy and run this model on NVIDIA B200 GPUs using the command below. Copy the command to get started with inference.

CONSOLE
docker run --gpus all 
 --shm-size 64g 
 -p 8000:8000 
 -v ~/.cache/huggingface:/root/.cache/huggingface 
 -e HF_TOKEN='YOUR_HF_TOKEN' 
 --ipc=host 
 lmsysorg/sglang:v0.5.8-cu130 
 python3 -m sglang.launch_server 
 --model-path Qwen/Qwen3-4B-Thinking-2507 
 --host 0.0.0.0 
 --port 8000 
 --max-prefill-tokens 65536 
 --max-running-requests 1024 
 --enable-piecewise-cuda-graph 
 --tp 8 
 --mem-fraction-static 0.95 
 --attention-backend flashinfer 
 --trust-remote-code

Model Benchmarks

Each model was tested with a fixed input size and total token volume while increasing concurrency to measure serving performance under load.

ITL vs Concurrency

Time to First Token

Throughput Scaling

Total Tokens/sec vs Avg TTFT

Vultr Cloud GPU

NVIDIA HGX B200

Deploy NVIDIA B200 on Vultr Cloud GPU

Deploy NVIDIA B200 instancesContact Sales