AWS Hydro: Compiler Guardrails for Coding Agents' Mistakes

AWS’s Hydro team recently landed what its research lead called “like a 5000 line PR”, written mostly by agents with humans driving, for a framework that other developers rely on to catch bugs. Shadaj Laddad’s argument is that code review can’t keep pace with agent output, and that tests written by the same agent can’t be fully trusted either. For platform teams adopting coding agents on distributed systems, his proposed fix moves the safety net into the compiler.

How It Works

The New Stack spoke to Laddad, research lead for the Hydro team at AWS, at AGNTCon + MCPCon in San Jose. Hydro is an open-source Rust framework for distributed systems, built jointly by AWS and UC Berkeley. Agents let the team aim higher, he said, with pull requests arriving faster and running longer. The risk is in combination: “You can have the feature work well in isolation, then you mix it with something else, and suddenly the guarantees break down.”

In most agent workflows the agent writes both the code and its unit tests. “Agents are great, but they’re unpredictable,” Laddad said, which makes it hard for a human to confirm every test checks what the team intended. His answer is correctness checks outside the agent’s “zone of trust”, enforced by tools like a compiler. Rust’s borrow checker is the model: it catches race conditions before code runs. Hydro does the same for some distributed systems bugs, catching several types at compile time so an agent gets instant feedback.

The team now looks for recurring patterns in past projects that could become shared, domain-specific guardrails. Idempotency, making sure a retried request doesn’t run twice, comes up repeatedly in design reviews, so it’s a candidate. Laddad frames it as a ladder: C spared programmers from managing function calls in assembly, Java took over memory allocation, and agents make the next layer more urgent.

The same team built Infinity, an Apache 2.0 agent runtime announced October 23. Most harnesses block while a tool runs, holding a process and memory. Infinity treats an agent like a stackless coroutine: when it calls a tool, the runtime saves the conversation, dispatches the call and suspends the agent. Each step is a bounded slice that loads history from a store, runs one model completion, persists it and dispatches at most one tool call. Tool results arrive as messages on a queue that resumes the agent. Because a suspended agent occupies no memory, a serverless function can end while a tool call runs for days.

What This Means for You

AWS says Infinity fits 75,000 concurrent agents in the 8 GB of a Raspberry Pi 5. The New Stack notes that figure is extrapolated from a server test in which each idle agent used about 100 KB. Both projects are early: AWS calls Infinity a research release, and Laddad warns Hydro users to expect rough edges and a learning curve, aimed at teams running high-throughput, low-latency services.

Compilers have limits too. The New Stack reported in August that AI-generated Rust can compile perfectly and still be wrong. A guardrail only covers the class of bug it was designed for.

Stack Archive covered how durable Lambda functions record whole executions; Infinity takes a similar suspend-and-resume approach for the agent loop itself.

Source: The New Stack

Content created with AI assistance and reviewed for accuracy.