Nvidia Shows the Harness, Not the Model, Is the Real Hero
Nvidia just published some fascinating research. And it turns out, the AI model itself might not be the most important part of an AI agent.
Let me break down what they found.
The TL;DR
Nvidia researchers got Claude Opus 5 to achieve a 100% score on the interactive reasoning benchmark ARC-AGI-3. That’s a set of 2D games with no instructions, where the model has to figure out how to play and win, similar to how a human would.
Without the harness, Opus 5 scored 30%, which was already the top result among all models tested.
The difference? A custom harness tweaked to handle memory well and including a “supervisor” boss-like component.
What’s a Harness?
A harness is the software wrapper around an AI model. It includes the tools, memory management, and rules that turn a raw model into something that can act on its own.
Adel El Hallak, vice president of product in Nvidia’s AI unit, put it this way:
“Generally speaking, the world interprets an agent almost as an API of the model. It is the model. It is the scaffolding around the model, which we call the harness, i.e. the set of tools that it utilizes. It is the runtime and the associated skills and libraries that we give it access to.”
The Supervisor Component
Here’s the key insight.
The harness needs a “supervisor” component that prods the agent in the right direction if it gets stuck.
“The more interesting part was introducing a supervising agent in addition to your main agent that’s doing the work. It almost acts like a CEO to nudge the agent when it goes off direction or starts exploring a path that it might lead to a dead end, or re-explore a path that it had previously trod.”
This isn’t exactly a new concept. But most agent users today are relying on only one layer for their harness, like Claude Code, Codex, or Hermes.
Nvidia researchers created their own souped-up harness called the Agentic Variation Operators (AVO).
Why This Matters
Long-horizon tasks are those that require stringing many decisions together, sometimes over days, to produce completed work. Figuring out how to get an AI to do this without getting distracted is one of the holy grails in agentic research.
Models stringing decisions together on their own have been caught deleting users’ files, deleting entire databases, or turning to criminal behavior like collusion and hacking to achieve their objectives.
So getting the harness right is critical.
The Open AI Connection
The choice by Nvidia researchers to use this interactive reasoning benchmark is particularly meaningful. OpenAI was so flustered by its models’ abysmal scores (less than 10%) on ARC-AGI-3 that it conducted its own research last month.
Like Nvidia, OpenAI discovered that simply by tweaking two settings on the harness, their models tripled their scores.
But none of the models came close to hitting a 100% score like Nvidia’s researchers achieved.
The Cost Factor
The harness also dramatically impacts costs.
Databricks published research in July showing that you can pick the same model but different harnesses and get significantly different costs.
“You can pick the same model but different harnesses, and you get significantly more cost if you use the wrong harness. So you think, oh, this is an expensive model. This is a cheap model. But wait, which harness are you using? That itself can 2x your cost.”
The Larger Point
Nvidia’s research shows that open harnesses, like open models, put users in control far more than they realize.
“We believe, and we’re demonstrating with the ecosystem, how open harnesses allow you to turn a lot more knobs to drive up that accuracy. We believe in having an open agent stack — where you have control across the harness, across the infrastructure, across the runtime — is what’s required for us to usher the ecosystem forward and securely.”
The Bottom Line
Nvidia showed that a custom harness got Claude Opus 5 to 100% on a tough benchmark, compared to 30% without it. The key is a “supervisor” component that acts like a CEO, nudging the agent when it gets stuck.
The harness handles memory, context, and feedback. It turns a raw model into an agent. And it dramatically impacts both performance and cost.
Model choice matters. But the harness might matter more.
