LangSmith Coding Agents: What a Merged PR Really Costs

LangSmith Coding Agents: What a Merged PR Really Costs

Agentic AI

LangChain puts enterprise spend on coding AI at $34 billion in 2026, yet most engineering leaders still can’t say what that money buys. LangSmith’s new coding agent dashboards try to answer it with one number per pull request, which gives engineering managers and finance teams the same unit to argue about.

$34B
enterprise AI spend on coding in 2026, the largest application category, per LangChain

What Happened

LangChain announced LangSmith for Coding Agents on November 18, in public beta on LangSmith Cloud. The core idea is a new concept called outcomes. LangSmith links GitHub pull requests or GitLab merge requests to the agent threads behind them, so each PR carries the cost of the sessions that produced it. The attribution handles both directions: one thread that opens several PRs, and one PR that took several threads to finish.

Coding agents are the first specialized “agent type” on the rebuilt LangSmith Agent Platform. Supported harnesses are Claude Code, OpenAI Codex, Cursor, GitHub Copilot in VS Code and LangChain’s own Deep Agents. Each needs its own plugin, and LangChain’s setup docs cover each one. Traces count toward normal LangSmith tracing usage.

Why It Matters

Usage analytics show tokens consumed, not what those tokens produced. LangChain says it hit the problem itself when spend on its own coding agent, Robin, kept climbing. The new overview page breaks spend down by PR state (open, closed or merged), so a team can see how much went into work that was closed without merging.

The design avoids one obvious trap. Review, investigation and codebase exploration often don’t end in a PR, so LangSmith reports that spend separately as unattributed rather than counting it as waste. A team with a high cost per merged PR may still be working sensibly, and engineering can open those threads to show finance what the money went toward. A work-type view labels each thread as a feature, bug fix, investigation or code review, and each organization can flag requests it considers off topic.

What This Means for Builders

The most useful views point at specific habits. Sorting PRs by attributed cost surfaces the expensive ones and the threads behind them, such as a thread that kept growing or a frontier model used on a small fix. When one engineer gets the same kind of work merged for much less, the team can find out why.

Thread length is the clearest lever. Spend concentrates in very long threads, LangChain explains, because every new turn carries the whole conversation’s context. Starting a fresh thread between tasks avoids paying for that context again, and LangChain points to Anthropic’s Claude Code cost guide as making the same recommendation. The cost-by-thread-length view then shows whether the habit actually changed.

Model and skill choices become testable too. Cost per merged PR before and after a model switch shows whether the new model justifies its price, and moving subagents to a smaller model should lower cost, with outcomes showing whether work still gets merged. Skill calls are traced, so a skill that keeps appearing in long, expensive threads may need its instructions rewritten. One that shows up in short threads ending in merged PRs is worth rolling out more widely. It’s the same lesson as the harness study finding that harnesses buy tokens, not wins: measure the outcome, not the activity.

The So What

Engineering leads running more than one coding agent should install the plugin on one agent and one repo first, as LangChain suggests, then check two numbers before changing any policy. Cost per merged PR is the unit to report. The share of spend sitting in long threads is the quickest to cut: if it’s high, tell the team to start a fresh thread per task, and check the same view a few weeks later.

Source: LangChain Blog

Content created with AI assistance and reviewed for accuracy.

💬

Join the conversation

Stack Insiders is our free community for readers who want to go deeper — share resources, ask questions, and connect with others across every vertical we cover.

Join Stack Insiders →

Newsletter coming soon.

Curated digests across AI, biohacking, photography, travel, and more. Be the first to know when we launch.