Nvidia's SoL-Pi Cuts Coding Agent Token Usage by Nearly Half — Without Touching the Model

The standard assumption in AI agent optimization is that better performance requires a better model. SoL-Pi challenges that assumption by doing something different: making the harness — the control layer between the model and its environment — significantly more efficient. The result is 44 to 54% less token usage on coding tasks, with no model changes, yielding roughly $4 to $13 in savings per agent-hour depending on configuration.

What the Harness Actually Does

In a coding agent like Claude Code or OpenAI’s Codex, the model itself is only part of the system. The harness manages everything else: which tools get called, how results are fed back to the model, how context accumulates across turns, and what happens when a tool call fails. The harness is invisible in most benchmarks, which measure end-task success, not the efficiency of the control logic getting there.

SoL-Pi targets this overlooked layer. Rather than modifying the model’s reasoning or prompting strategy, it uses a research agent to analyze execution traces, propose specific harness modifications, and test whether those modifications maintain task performance while reducing token consumption. Of 152 optimization directions explored across nearly 3,000 runs, only the changes that passed both criteria — maintain performance, reduce tokens — made it into the final system.

The Four Mechanisms

Action Fusion: When an agent makes consecutive tool calls that could be batched, SoL-Pi merges them. Instead of returning to the model between every small step, the harness handles sequences of operations in a single round trip. Fewer model turns means fewer tokens.

Online Context Compaction: As a coding session progresses, the conversation context grows. SoL-Pi trims accumulated context on the fly — removing information that’s no longer relevant to the current state of the task — rather than waiting until context overflow forces a hard truncation.

ObservationPack: Tool outputs can be verbose. A successful file read returns the entire file; a search result might include surrounding text the model doesn’t need. ObservationPack summarizes archived tool outputs before including them in context, reducing the token cost of carrying prior results forward.

Evidence-Preserving Reducer: Error messages and stack traces are critical information, but they’re also long. This mechanism processes error logs through a smaller, cheaper model to distill the essential signal — the part that actually informs what to do next — while dropping the structural noise.

The Numbers

These reductions translate directly to cost: on AWS p4d.24xlarge infrastructure, the system estimates savings of $4 to $13 per agent-hour at current model pricing, with the range depending on which base model and which task distribution.

What This Changes for Teams Running Coding Agents

The immediate implication is cost. If your team runs coding agents at any significant volume — automated code review, refactoring pipelines, test generation, PR assistance — a 44–54% token reduction changes the unit economics of those workloads materially.

The less obvious implication is what SoL-Pi reveals about where agent efficiency gains actually come from. Most optimization effort goes into prompt engineering, model selection, or fine-tuning. The harness layer — how results flow between the model and the environment, how context is managed, how tool outputs are packaged — is typically treated as infrastructure that runs correctly by default. SoL-Pi demonstrates that this layer can be a significant source of waste, and that the waste can be reduced automatically.

Content created with AI assistance and reviewed for accuracy.