DeepSeek's own inference engine for serving its models. It began as a vLLM build, borrowed scheduling ideas from SGLang, and DeepSeek published the implementation in October 2026.

Did vLLM catch up with DeepInfer?

On the configuration vLLM disclosed, close enough. With 131K context and three draft tokens per round on GB200, batch sizes 8 to 256 kept vLLM within 6% of DeepInfer on decode throughput, while prefill throughput for new tokens rose 35.6% to 43.9%.

Is SGLang faster than DeepInfer?

On SGLang's own tests, in places. Prefill throughput came in up to about 4% higher on four B300 and four GB300 servers; on 64 GB300s at 256 requests per GPU it needed 28.88 ms per decode step against 31.19 ms. At 8 requests per GPU the two were level.

Is 954K tokens per second per GPU a fair comparison?

It is a rack-level figure, not an engine-level one. vLLM reached it on a GB300 NVL72 rack with prefill-decode disaggregation and wider expert parallelism, about 7.9 times its earlier single-node result. DeepSeek's reference runs used different hardware and parallelism.

Does DeepInfer replace vLLM or SGLang?

No. DeepInfer began as a vLLM build with SGLang-inspired scheduling, and publishing it let both projects absorb the optimizations within a week.

DeepSeek Open-Sources DeepInfer: vLLM and SGLang Match Its Performance Within a Week

TL;DRDeepSeek published DeepInfer, its own inference engine, in late October 2026. Within a week, vLLM and SGLang both shipped changes that land at or above it on the configurations each team disclosed. vLLM measured a 6% decode-throughput gap on GB200 with 131K context and 954K tokens/s per GPU on a GB300 NVL72 rack; SGLang reported up to 4% higher prefill throughput and 28.88 ms per decode step at 256 requests per GPU against DeepInfer's 31.19 ms. Every number is self-reported on different hardware, so treat them as per-config results rather than a universal ranking.

When DeepSeek open-sourced DeepInfer, the open question was how much of its advantage would survive contact with the general frameworks. A week later, most of it has.

DeepInfer did not start from scratch. It began as a vLLM build and took scheduling ideas from SGLang. So when DeepSeek published the polished version, the two projects that supplied its foundation read it and shipped their own versions of the same optimizations within days. The code moved in both directions, which is the part worth watching.

What did DeepSeek actually release?

DeepInfer is the inference engine DeepSeek uses to serve its own models. It targets the DeepSeek V4 family, with the published community benchmarks centered on DeepSeek-V4.1-Flash. The code is public, and both vLLM and SGLang integrated parts of it within days rather than months.

The lineage matters more than the benchmark tables. DeepInfer began as a vLLM build with SGLang-inspired scheduling, so the three codebases now share ancestry and read each other's work directly. SGLang went further and re-ran DeepInfer on GB300 and B300 hardware to make the comparison honest. Its measurement found DeepInfer's throughput on some configurations running higher than DeepSeek's own published figures.

How close is vLLM to DeepInfer?

vLLM is close on decode and ahead on prefill throughput. The team integrated DeepSeek's open kernels and reworked both stages. The decode comparison used 131K context with three draft tokens per round on GB200; across batch sizes from 8 to 256, the gap to DeepInfer stayed at 6%. New-token throughput in the prefill stage improved 35.6% to 43.9%.

DeepSeek-V4.1-Flash on vLLM against DeepInfer. Hardware and parallelism differ between the two series, so the chart is not an engine-to-engine speedup measurement.

Three changes did most of the work. The first is DeepSelect: sparse attention has to pick top-k candidates, and the old path converted DeepGEMM's BF16 scores to FP32 before filtering. The new path filters BF16 directly, and that operator now runs about 3.7× faster. The second is a rewritten CPU communication layer that exchanges data through recursive doubling and similar schemes, cutting the number of synchronization rounds. The third is dedicated support for DSpark draft generation in the small-batch decode pattern that agents generate, executed through a full CUDA Graph.

At rack scale the numbers look different. On an NVIDIA GB300 NVL72 rack with prefill-decode disaggregation and wider expert parallelism, vLLM reached 954K tokens/s per GPU in the SemiAnalysis AgentX test, 7.9× its own earlier single-node result.

The specialized decode path on B300 raised p90 interactivity 44% to 59% depending on concurrency. Chart: SemiAnalysis AgentX, B300, TP4 on four GPUs, reproduced from vLLM's release.

Did SGLang beat DeepInfer outright?

On its own tests, in places, yes. SGLang covered both stages instead of stopping at decode. Prefill on four B300 servers and four GB300 servers came in up to about 4% above DeepInfer. Decode on 64 GB300s told a more interesting story: at 256 requests per GPU, DeepInfer needed 31.19 ms per step and SGLang needed 28.88 ms, about 7.4% shorter. At 8 requests per GPU the two were effectively level.

The engineering list is longer than vLLM's. SGLang first pulled in the Mega kernels (MegaMoE, MegaAttention, Mega-mHC, MegaGate) along with SWA bounded replay and Engram optimizations, reached parity, then kept iterating with more kernel fusion and splittable CUDA graphs to make prefill cheaper. It also shipped what DeepInfer's public version leaves out: a KV cache system, prefill-decode disaggregation, and a load-balancing module. For anyone deploying rather than benchmarking, that completeness may matter more than the headline percentage.

What does one developer's Resin build show?

That the design travels further than the two big frameworks. Eric Zhang had been building Resin, an experimental toolkit that treats the shared parts of an inference engine (scheduler, KV cache management) as a base layer and leaves the model- and hardware-specific compute to the user. Inspired by DeepInfer, he got much of the feature set and several parallel strategies running within a week.

On DeepSeek-V4.1-Flash, Resin stayed within roughly 5% to 10% of DeepInfer while also providing HTTP serving, prefix caching, and metrics. He then paired the base with a skill file and turned an agent loose on it overnight. The agent produced a GLM-5.3-Flash engine with MTP support; on B200 at EP4 it reached up to 60K tokens/s input and 1.6K tokens/s output. Resin remains open source.

Resin against DeepInfer's published EP8 decode figures at 128K context. DeepInfer's curve is an approximate digitization, not a rerun.

What should you check before trusting these numbers?

Three things decide whether a comparison like this means anything.

Hardware and parallelism. vLLM's decode comparison used GB200 at a specific TP/EP layout; its 954K figure came from a 72-GPU rack. SGLang used B300 and GB300 at different widths. None of these are the same test.
Which stage is being measured. Prefill and decode scale differently, and a framework can win one and tie the other, as SGLang does at low concurrency.
Whether the comparison is engine-to-engine at all. vLLM's own footnote says its chart is not a speedup comparison, because the parallelism settings differ between the vLLM series and DeepInfer's published series.

The takeaway is narrower than the headline suggests. The gap between a vendor-tuned engine and a general framework has closed to single digits on matched workloads, and serving completeness (KV cache, disaggregation, load balancing) now counts for as much as raw tokens per second.