| Type | Vision-Language Model |
| Capabilities | Text Generation, Instruction Following, Reasoning, Mathematical Reasoning+6 more |
| Release Date | 27 July, 2026 |
| Links | |
| License | Kimi-K3 |
Inference Instructions
Deploy and run this model on AMD MI355X GPUs using the command below. Copy the command to get started with inference.
CONSOLE
docker run -it --rm \ --device=/dev/kfd --privileged --device=/dev/dri --group-add video --security-opt seccomp=unconfined --security-opt apparmor=unconfined --cap-add=SYS_PTRACE --ipc=host --shm-size=128g -p 8000:8000 -v ~/.cache/huggingface:/root/.cache/huggingface -e NCCL_MIN_NCHANNELS=112 -e TORCH_NCCL_HIGH_PRIORITY=1 -e HIP_FORCE_DEV_KERNARG=1 -e TORCH_BLAS_PREFER_HIPBLASLT=1 -e VLLM_ROCM_USE_AITER=1 -e AITER_SITUV2_A8W4=1 -e VLLM_ROCM_USE_AITER_RMSNORM=0 -e VLLM_USE_BREAKABLE_CUDAGRAPH=0 -e AITER_BF16_FP8_MOE_BOUND=0 -e SAFETENSORS_FAST_GPU=1 -e HF_TOKEN='YOUR_HF_TOKEN' vllm/vllm-openai-rocm:kimi-k3 moonshotai/Kimi-K3 --port 8000 --host 0.0.0.0 --tensor-parallel-size 8 --moe-backend auto --load-format auto --mm-encoder-tp-mode data --max-model-len auto --max-num-seqs 1024 --enable-auto-tool-choice --tool-call-parser kimi_k3 --reasoning-parser kimi_k3 --gpu-memory-utilization 0.90 --trust-remote-code
Note
Requires the vllm/vllm-openai-rocm:kimi-k3 Docker image or a standard vLLM image version greater than v0.26.0.
Model Benchmarks
Each model was tested with a fixed input size and total token volume while increasing concurrency to measure serving performance under load.
ITL vs Concurrency
Time to First Token
Throughput Scaling
Total Tokens/sec vs Avg TTFT
Vultr Cloud GPU
AMD Instinct MI355X
Deploy AMD MI355X on Vultr Cloud GPU