Kimi-K2.5 (1T) Validation Testing

Concurrency Scaling

Scaling Results

Observations:

• Consistent scaling from 5 to 100 concurrent requests
• Peak scaling throughput of ~1,777 tok/s at 50 concurrent
• DEGRADED status indicates p99 latency >2x baseline (expected under concurrent load)
• 100% success rate maintained across all concurrency levels

Stress Tests

Key findings:

• Long output generation (500 tokens): 107 tok/s total, 90 output tok/s
• Long context (4K tokens): 454 tok/s with 40.1s p99 latency
• Multi-image (3 images): 491 tok/s with 66.5s p99 latency
• High concurrency vision (100 concurrent): 887 tok/s, 149 output tok/s

Saturation Testing

Extreme Load Results

Observations:

• Peak throughput of ~2,053 tok/s achieved at 500 concurrent
• Saturation begins at 300 concurrent requests
• 100% success rate maintained even under extreme load
• Consistent ~1,900-2,000 tok/s across 150-500 concurrent

Recommendations

Test Configuration

Launch Command

docker run --rm \
  --name vllm-kimi-k25 \
  --ipc=host \
  --network=host \
  --group-add video \
  --group-add render \
  --cap-add=SYS_PTRACE \
  --cap-add=CAP_SYS_ADMIN \
  --security-opt seccomp=unconfined \
  --device /dev/kfd \
  --device /dev/dri \
  -v ~/.cache/huggingface:/root/.cache/huggingface \
  --env "VLLM_ROCM_USE_AITER=0" \
  --env "VLLM_USE_TRITON_FLASH_ATTN=0" \
  rocm/vllm-dev:nightly \
  vllm serve moonshotai/Kimi-K2.5 \
  --tensor-parallel-size 4 \
  --max-model-len 32768 \
  --trust-remote-code \
  --block-size 1 \
  --mm-encoder-tp-mode data

Test Environment