Latest ContentInference Cookbook Model Library

Sarvam 105b

Sarvam-105b is a Mixture-of-Experts (MoE) language model built for advanced reasoning, coding, and agentic tasks. It uses a 32-layer transformer with 4,096 hidden size, 64 attention heads, and a large head dimension of 576. The model includes 128 experts with top-8 routing and a shared expert, with ~10.3B active parameters per token. It supports a 128K context window with YaRN scaling and MLA-style attention. Trained with a focus on Indian languages, it delivers strong performance across multilingual, reasoning, and real-world agentic applications.

Type	MoE LLM
Capabilities	Text Generation, Instruction Following, Reasoning, Mathematical Reasoning+5 more
Group Release Date	March 5, 2026
Links	Blog\|HF Model Card
License	Apache 2.0

Inference Instructions

Deploy and run this model on NVIDIA B200 GPUs using the command below. Copy the command to get started with inference.

CONSOLE

docker run -it --rm 
 --runtime=nvidia 
 --gpus all 
 --ipc=host 
 --shm-size=128g 
 -p 8000:8000 
 -v ~/.cache/huggingface:/root/.cache/huggingface 
 -e HF_TOKEN='YOUR_HF_TOKEN' 
 -e TORCH_FLOAT32_MATMUL_PRECISION=high 
 vllm/vllm-openai:v0.18.0 
 sarvamai/sarvam-105b 
  --tensor-parallel-size 8 
 --max-model-len auto 
  --max-num-batched-tokens 65536 
 --gpu-memory-utilization 0.95 
 --max-num-seqs 1024 
 --trust-remote-code