Latest ContentInference Cookbook Model Library

Qwen3 4B Instruct 2507

The Qwen3-4B-Instruct-2507 is an instruction-aligned Causal Language Model built on a 36-layer architecture with 3.6B non-embedding parameters, optimized for high-quality instruction following and preference-aligned generation. Unlike the base Qwen3 models, this updated model operates exclusively in non-thinking mode and is tuned to deliver direct, well-structured responses without intermediate reasoning traces. It supports a native 262,144-token context window, enabling effective handling of very long documents, extended conversations, and large-context comprehension workloads.

Type	Dense LLM
Capabilities	Text Generation, Instruction Following, Reasoning, Mathematical Reasoning+5 more
Release Date	April 28, 2025
Links	Paper\|Blog\|HF Model Card
License	Apache 2.0

Inference Instructions

Deploy and run this model on NVIDIA B200 GPUs using the command below. Copy the command to get started with inference.

CONSOLE

docker run --gpus all 
 --shm-size 64g 
 -p 8000:8000 
 -v ~/.cache/huggingface:/root/.cache/huggingface 
 -e HF_TOKEN='YOUR_HF_TOKEN' 
 --ipc=host 
 lmsysorg/sglang:v0.5.8-cu130 
 python3 -m sglang.launch_server 
 --model-path Qwen/Qwen3-4B-Instruct-2507 
 --host 0.0.0.0 
 --port 8000 
 --max-prefill-tokens 65536 
 --max-running-requests 1024 
 --enable-piecewise-cuda-graph 
 --tp 8 
 --mem-fraction-static 0.95 
 --attention-backend flashinfer 
 --trust-remote-code