Is DeepSeek V4.1 Pro in Testing? What the 64% LobeBench Result Shows
TL;DRA LobeHub evaluator reports that the current DeepSeek V4 Pro on the web completed 32 of 50 difficult agent tasks in all three attempts, up from 24 for V4 Pro-0813. The result has sparked speculation about V4.1 Pro, but the test does not establish the identity of the model behind the service. DeepSeek has not announced a V4.1 Pro release date.
By Cameron Voss and Marcus Bell
DeepSeek V4.1 Pro is back in the spotlight after an independent evaluator reported a substantial change in how the current V4 Pro performed on a private set of agent tasks. The reported gain is particularly interesting for developers: it concerns whether an agent can complete the same difficult task reliably, not just succeed once.
There is a distinction, though. The evaluation compares an observed web experience with an earlier V4 Pro version. It does not prove that DeepSeek has deployed V4.1 Pro or that a recent service interruption was caused by a model rollout.
What did the LobeBench test find?
Max For AI, a LobeHub developer and maintainer, reported results from LobeBench Hard 50, a private collection of 50 complex agent tasks. According to the evaluator, the tasks assess abilities such as understanding an assignment, using tools and continuing until the work is complete.
| Reported measure | V4 Pro-0813 | Current V4 Pro on DeepSeek's website |
|---|---|---|
| Tasks passed in all three attempts | 24 / 50 (48%) | 32 / 50 (64%) |
| Tasks passed at least once in three attempts | 80% | 82% |
That is an increase of 16 percentage points, or eight tasks, on the three-for-three measure. The evaluator also said the current result ranked first among Chinese models in its comparison. That is a claim about this particular private benchmark and its tested field, not a universal ranking of Chinese AI models.
Explore our DeepSeek Coding Plans and current availability.
Why does passing all three attempts matter?
A task passed at least once tells you that a model can sometimes find a workable path. Passing all three attempts asks a tougher question: does it finish repeatedly under the test's conditions? For an agent that must call tools and work through several steps, repeatability can matter as much as a single impressive result.
The reported at-least-once rates were close, while the three-for-three scores were farther apart. That suggests better repeatability on this test. Three runs and 50 tasks cannot eliminate chance, however, and a private test set is harder for outsiders to inspect or reproduce. The result does not by itself establish a general improvement across every coding task or agent setup.
Is the tested model DeepSeek V4.1 Pro?
Its identity has not been established by the benchmark. The evaluator described the current V4 Pro available on DeepSeek's website and suggested a possible V4.1 Pro gray test. A higher score, a familiar display name or a change in response style does not identify the model checkpoint or rule out changes in configuration, routing or the surrounding agent system.
DeepSeek's September V4.1 Flash announcement referred to a future V4.1 Pro launch. Its API changelog subsequently said V4 Pro API service would continue after September 14, so the announcement's original routing plan should not be treated as the current API configuration. The web test should not automatically be assumed to describe what every API customer receives.
Did the outage signal a release?
DeepSeek's status history records partial web and API interruptions on November 3 and November 7. The November 7 incident affected V4 Pro API, V4.1 Flash API and other services and was listed as resolved after 45 minutes. The timing of an interruption and a reported performance change is not evidence that one caused the other.
Some social posts also claim that DeepSeek denied the current gray-test rumor. We have not independently verified the alleged original official statement, so that claim remains unconfirmed as well.
When will DeepSeek V4.1 Pro launch?
There is no confirmed release date in the official materials reviewed for this update. Predictions of a launch next week are speculation. The August 13 release of V4 Pro-0813 provides useful context, but the time since that release does not establish a schedule for its successor.
How can developers check for a meaningful change?
- Choose several representative tasks, including ones that previously failed, and keep the prompts, tools, agent framework and evaluation criteria fixed.
- Run each task multiple times. Record both whether it ever succeeds and how often it succeeds, along with errors and completion time.
- Record the date, service or provider, displayed model name, API model ID if applicable, settings and any version or fingerprint returned. A stable model name alone does not prove that the underlying model is unchanged.
- Compare results across enough tasks and runs to see whether an apparent gain persists. Avoid treating a small sample as proof of a model's identity.
For earlier context, see our October report on the V4.1 Pro rollout speculation and our coverage of the V4 Pro-0813 release.