DeepSeek Open-Sources DeepInfer: vLLM and SGLang Match Its Performance Within a Week
DeepSeek published DeepInfer, its own inference engine, in late October 2026. Within a week, vLLM and SGLang both shipped changes that land at or above it on the configurations each team disclosed. vLLM measured a 6% decode-throughput gap on GB200 with 131K context and 954K tokens/s per GPU on a GB300 NVL72 rack; SGLang reported up to 4% higher prefill throughput and 28.88 ms per decode step at 256 requests per GPU against DeepInfer's 31.19 ms. Every number is self-reported on different hardware, so treat them as per-config results rather than a universal ranking.
Lire l’article