Zero-Trust AI Agent Access in Practice: IBM watsonx and HashiCorp Boundary
Enterprise Developer PlatformsThe access model problem for AI agents is well-understood in principle: an agent that can autonomously access infrastructure needs credentials, and credentials that an agent can access are credentials that a compromised agent can misuse. The principle-of-least-privilege answer — give the agent only what it needs for this specific task, for this specific duration — is straightforward in theory and complex to implement in practice.
HashiCorp’s proof-of-concept with IBM watsonx.ai shows one concrete implementation: an agent that performs infrastructure diagnostics without ever seeing the credentials it uses to do so.
The Architecture
The PoC uses four components:
- IBM watsonx.ai — agent reasoning and task planning
- HashiCorp Boundary — access control proxy and session management
- HashiCorp Vault — credential injection and secrets management
- IBM Verify — identity provider (OIDC)
The workflow: a platform engineer submits a natural-language request (“investigate the latency spike on the database cluster”). The watsonx.ai agent develops an investigation plan — what systems to query, in what order, with what commands. It then initiates Boundary sessions to execute each step.
Boundary as the access layer: The agent doesn’t connect to infrastructure directly. Every connection routes through Boundary workers, which verify the agent’s identity (tied to the requesting engineer’s OIDC identity via IBM Verify), confirm the target is in the agent’s authorized scope, and proxy the session. The agent sees query results; it doesn’t see the underlying credentials or network paths.
Vault for credential injection: When the agent initiates a Boundary session that requires credentials, Vault generates them dynamically for that session — short-lived, scoped to the specific target, revoked on session end. The agent doesn’t receive these credentials; they’re injected at the proxy layer. If the agent session is compromised, the leaked credentials are already expired.
Session recording: Every session the agent opens generates an audit record: which target was accessed, which engineer authorized the session, what commands were executed, connection metadata. This is the audit trail that enterprise governance requires for AI-assisted infrastructure operations.
What This Solves Operationally
The credential injection pattern addresses the specific failure mode that makes organizations hesitant about AI agents in production environments: secret sprawl. Traditional agent credential management — API keys in environment variables, service account tokens stored in agent memory, static credentials passed at initialization — creates secrets that outlive any individual session and accumulate in ways that are hard to audit or rotate.
The Boundary + Vault pattern eliminates stored agent credentials. There’s no credential to leak from the agent process because the agent never holds one. The security boundary is the Boundary worker layer, not the agent’s runtime security.
How It Differs from the Architectural Overview
HashiCorp published a broader architectural overview of JIT credentials for AI agents in June 2026 — the framework and the five requirements for production agentic AI access. This PoC is the operational implementation of that framework in a specific technology stack, with IBM watsonx.ai as the agent reasoning layer.
The distinction matters for organizations evaluating whether to adopt the pattern: the June article argued the case; this PoC shows the mechanics. The IBM watsonx integration demonstrates that the pattern isn’t model-specific — it applies wherever you can route agent-initiated connections through Boundary.
The Practical Limitations
This is a proof-of-concept, not a production template. The PoC-specific elements:
- IBM Verify as the IdP — organizations using Okta, Azure AD, or other OIDC providers would substitute their IdP at this layer
- IBM watsonx.ai as the reasoning model — any agent framework that can initiate authenticated Boundary sessions works in this role
- The investigation scenario (database latency diagnostics) is representative but not exhaustive; agents with broader tool access would require additional scope definitions
The core pattern — Boundary as access proxy, Vault for credential injection, OIDC identity as the authorization anchor — is portable across IdPs and agent frameworks. The IBM watsonx stack in the PoC is illustrative, not prescriptive.
The So What
Zero-trust AI agent access is the answer to the question “how do we let agents do useful infrastructure work without creating a security liability.” The Boundary + Vault pattern provides the practical architecture: agents operate through a proxy that injects short-lived credentials, records every session, and enforces authorization based on the human user’s identity rather than the agent’s.
For organizations running HashiCorp infrastructure who are also evaluating AI-assisted operations: this PoC is the reference implementation. For organizations using other security platforms: the same pattern applies — the implementation details change, the access-proxy + credential-injection + session-recording architecture doesn’t.
Source: HashiCorp Blog
Content created with AI assistance and reviewed for accuracy.
Join the conversation
Stack Insiders is our free community for readers who want to go deeper — share resources, ask questions, and connect with others across every vertical we cover.
Join Stack Insiders →