Z.ai Open-Sources HarnessPro — Agent Self-Evolution That Cuts Costs by Up To 53%
By Cameron Voss
TL;DRZ.ai and Tsinghua's Jie Tang team have open-sourced HarnessPro, a framework that lets a coding agent rewrite its own harness between runs. It keeps a Pareto set of three operating points: Lite for cost, Medium for balance, and Heavy for performance. On SWE-Bench Pro with GPT-5.5 and Codex, Medium reached 79.0% pass rate at $1.26 per task, 53.0% below RHO's $2.68. The trade-off is explicit: Medium gives up 1.0 percentage point against RHO. Heavy beat RHO on all three benchmarks while costing less.
On this page
Recursive self-improvement has a cost problem. An agent that edits its own harness can discover better strategies, but the search often pays for those gains with more tokens, more retries, and longer runs. HarnessPro is an attempt to price that search and stop it from running away.
The harness is the code around the model: instructions, tools, hooks, routing rules, and the checks that decide when a task is finished. Change the harness and you change how the model spends tokens, even when the model itself stays frozen.
What did the Jie Tang team actually release?
The release is HarnessPro, not a new foundation model. The paper describes a multi-agent loop that evolves a harness over several epochs and returns three harnesses instead of one. The code and the evolved configurations are public.
Jie Tang is a professor in Tsinghua's computer science department and chief scientist at Zhipu AI. Hong Zhao of Tsinghua's automation department is the other corresponding author. Zhuoshi Pan and Zhiyuan Feng, both at Tsinghua, are co-first authors. The paper lists Tsinghua, Z.ai, Shanghai Jiao Tong University, and the University of Warwick in its affiliations.
The authors position the work against AFlow, SkillMOO, RHO, and AHE. Their comparison table claims a combination those methods do not offer: full harness scope, label-free feedback, recursive iteration, and a Pareto set as output. Whether that combination is genuinely new is a question for peer review. The measured cost numbers are easier to check.
Why does RSI usually get more expensive?
Most harness evolution methods buy score with search. They explore more configurations, run more verification, and accept longer traces. That works until the bill arrives. In the paper's GPT-5.5 and Codex runs, RHO lifted SWE-Bench Pro pass rate from 59.0% for the unmodified harness to 80.0%, but mean cost per task rose from $1.88 to $2.68, a 42.6% increase.
HarnessPro starts from a different objective. The system does not ask for the highest pass rate. It asks which harnesses sit on the performance-cost frontier, then keeps three representatives: one cheap, one balanced, one strong. The user picks the operating point instead of accepting whatever the search produces.
How does the evolution loop work?
Each epoch runs four agent roles in sequence. The roles are specialized, and each one has a narrow job.
- The trajectory analyzer runs each candidate harness three times on the training set. It reads all three traces, finds why a task failed, and flags operations that burned tokens without helping. One case in the appendix involved a mail server configuration where repeated logging on unrelated files pushed a single task to 2.51 million input tokens.
- The evolution planner reads the diagnosis and the cost metrics, compares candidates with a diff tool, and writes three redesign plans: cost-first, balanced, and performance-first.
- The harness editor applies a plan in an isolated workspace, produces a runnable harness, and commits the change to Git so the team can trace or roll it back.
- The test synthesizer writes 10 fixed executable unit tests from the task description. This replaces the LLM judge used by earlier label-free methods.
The judge replacement matters more than it sounds. The authors measured the LLM judge's recall on failed cases at 41.7%. In other words, it missed most failures and often called broken code a success. The synthesized tests raised that recall to 83.3%, a 41.7 percentage point gain. The tests are not official, and they can still be wrong, but they are executable and repeatable in a way that a model's opinion is not.
After evaluation, the system removes candidates that fall below the starting baseline or sit on the wrong side of the Pareto frontier, meaning they cost more and score lower than another candidate. A normalized weighted squared distance then picks the Lite, Medium, and Heavy representatives for the next epoch.
What did the agents change on their own?
The evolved harnesses did not simply cut context or lower reasoning effort across the board. The paper highlights three mechanisms that showed up in the final code.
Dynamic routing is the most visible change. The harness reads the task description and picks one of five profiles: compact, lean, standard, deep, or research. Each profile sets a reasoning level and a tool-call budget. Simple tasks stay cheap. Hard tasks get more room.
Runtime hooks handle the budget. In Terminal-Bench 2 Medium mode, the harness adds a reminder at 14 tool calls that the run is entering an economy phase and should consolidate checks. At 26 calls it warns that the run is entering a convergence phase and should stop broad exploration. At 44 calls it blocks further tool use and tells the agent to submit with the evidence it has. The reminders are appended to tool results rather than inserted at the start of the prompt, which preserves prefix cache reuse.
Log folding deals with a different kind of waste. When a command repeats the same error or build output, the harness collapses the repetition into a single line such as [previous line repeated N more times] and truncates long output to the head and tail. This is a small change with a large effect on trace length.
None of these ideas is exotic. The interesting part is that the search found them together and kept them only when the combination improved both pass rate and cost. The ablation table supports that reading. Bolting dynamic routing or budget hooks onto RHO saves money, but the pass rate gives ground on SWE-Bench Pro. The gain comes from the full loop, not from copying one trick.
Does the 53% cost cut hold up?
The headline comparison uses GPT-5.5 with Codex on SWE-Bench Pro. RHO reaches 80.0% at $2.68 per task. HarnessPro Medium reaches 79.0% at $1.26. That is 53.0% lower cost for 1.0 percentage point less pass rate.
Scroll horizontally to compare all benchmark results.
| Method | Terminal-Bench 2 pass rate | TB2 cost per task | SWE-Bench Pro pass rate | SWE cost per task | ProgramBench pass rate | PB cost per task |
|---|---|---|---|---|---|---|
| Vanilla harness | 69.5% | $0.89 | 59.0% | $1.88 | 60.3% | $2.65 |
| RHO | 76.3% | $1.09 | 80.0% | $2.68 | 68.8% | $3.59 |
| HarnessPro-Lite | 72.9% | $0.43 | 72.3% | $0.80 | 64.3% | $2.18 |
| HarnessPro-Medium | 76.3% | $0.56 | 79.0% | $1.26 | 67.9% | $2.90 |
| HarnessPro-Heavy | 80.8% | $0.72 | 80.7% | $1.97 | 69.6% | $2.95 |
GPT-5.5 with Codex. Pass rate is the mean; cost is the mean per task. Standard deviations are in the paper.
Heavy is the strongest single result. On Terminal-Bench 2 it reaches 80.8%, 4.5 points above RHO, at $0.72 versus $1.09, a 33.9% cost reduction. On SWE-Bench Pro it edges RHO at 80.7% versus 80.0% and costs $1.97 versus $2.68. On ProgramBench it reaches 69.6% at $2.95 versus RHO's 68.8% at $3.59.
Lite is the more portable claim. It beats the unmodified harness on pass rate across all three benchmarks while costing 17.7% to 57.4% less. That range is wider than the headline 53% because the comparison point changes with each benchmark. On SWE-Bench Pro, Lite costs $0.80 against the vanilla harness's $1.88, but its pass rate is 72.3% against 59.0%, so it is not a like-for-like performance match.
One baseline deserves a mention because it narrows the gap. RHO+Iteration reaches the same pass rates as RHO on the first two benchmarks at lower cost: $0.82 on Terminal-Bench 2 and $1.90 on SWE-Bench Pro. Medium still costs less on both, $0.56 and $1.26, but the 53% figure is specifically against the original RHO protocol. Against RHO+Iteration on SWE-Bench Pro, the saving is about 33.7% at a 1-point lower pass rate.
The GLM-5.2 and Claude Code runs reproduce the pattern. On SWE-Bench Pro, Heavy reaches 77.3% at $1.01 versus RHO's 75.0% at $2.17, and mean execution time falls from 520 seconds to 412 seconds even though latency was not an optimization target.
What should a team take from this?
The practical lesson is narrower. Pass rate alone hides the cost of getting there, and a harness that reports both numbers is easier to operate. A few ideas from the paper are cheap to test without the full framework.
- Add a tool-call budget with staged reminders. A soft warning at 14 calls and a hard stop at 44 is a concrete policy you can implement in a hook today.
- Route by task shape. Even a simple classifier that sends short tasks to a lean profile and long tasks to a deep profile can cut spend without touching the model.
- Fold repeated logs before they enter the context. The difference between a 2,000-line build log and a 40-line summary is paid on every subsequent turn.
If you run coding agents as a team, watch cost per accepted task rather than cost per token. A cheap run that fails twice is more expensive than a slightly pricier run that passes once. HarnessPro's contribution is a method for keeping that ratio visible during the search.
What is still unproven?
The results come from one team and three benchmarks. Independent replication has not happened yet, and the baselines are mostly harness evolution methods from the same research line. The numbers are not automatically wrong. A production team should still treat the 53% as a hypothesis to test on its own traces.
Costs also depend on provider pricing. The paper computes execution cost from token prices, so a provider price change moves the dollar figures even if token usage stays the same. The percentage gap against RHO is more stable than the absolute dollars.
Label-free does not mean the system runs without tests. HarnessPro generates its own unit tests, and those tests can encode a wrong interpretation of the task. The paper's recall improvement is encouraging, but a synthesized test suite is still a proxy for the real acceptance criteria.
The 53% is also a specific pairing: Medium against RHO on SWE-Bench Pro. Heavy gives up some of that saving to beat RHO on pass rate. Lite gives up pass rate to save more. The right choice depends on what a failed task costs you.
FAQ
What is RSI in AI agents?
RSI means recursive self-improvement. An agent studies traces from its own runs, edits its harness or instructions, and runs again so the next attempt can perform better. HarnessPro applies this loop to the harness layer rather than to model weights.
What did Z.ai and Jie Tang open-source?
The team released HarnessPro, a multi-agent framework for harness self-evolution, together with the evolved harness configurations. The paper lists Tsinghua, Z.ai, Shanghai Jiao Tong University, and the University of Warwick in its author affiliations.
How much cheaper is HarnessPro?
The headline result is 53.0% lower mean cost than RHO on SWE-Bench Pro with GPT-5.5 and Codex: $1.26 versus $2.68 per task, at 79.0% versus 80.0% pass rate. The Lite configuration saves 17.7% to 57.4% against the unmodified harness across the three benchmarks while beating it on pass rate.
Does HarnessPro need labeled test cases?
No. It does not use official test cases or reference answers during evolution. A test synthesizer agent writes 10 fixed executable unit tests from each task description. In the paper, that raised recall on failed cases from 41.7% with an LLM judge to 83.3%.
Is HarnessPro open source?
Yes. The paper says the code and evolution configurations are public. The experiments use GPT-5.5 with Codex and GLM-5.2 with Claude Code, so the method operates at the harness layer and is not tied to one model vendor.
What is the catch behind the 53% number?
The 53.0% figure is one comparison: Medium versus RHO on SWE-Bench Pro with GPT-5.5 and Codex, and Medium gives up 1.0 percentage point of pass rate. It is not a universal discount, and the results have not been independently replicated.
Sources
QbitAI, "Z.ai and Jie Tang's team open-source a new RSI result: agents evolve themselves while cost drops 53%," October 23, 2026.