AWS Agentic AI Lens: Smarsh Turned 150 Checks Into a Skill

AWS’s Agentic AI Lens has 150 best practices across 41 questions and six pillars, and no way to run it in the Well-Architected Tool console yet. Smarsh, which archives regulated communications for more than 7,000 organizations, needed a review before its first multi-agent workload reached customers. Its answer turns a document into a tool, and the review found the gateway’s ingress authorizer running with authentication disabled.

How It Works

The AWS Architecture Blog explains that the Agentic AI Lens is published as prescriptive guidance but isn’t selectable as a lens in the tool. The workload under review was a compliance alert review system built on Amazon Bedrock AgentCore’s runtime, gateway, registry and identity capabilities. The options were a multi-day manual review, with every reviewer structuring it differently, or waiting for console support.

Smarsh built a Claude skill with three parts:

1. lens_data.json: all 150 practices extracted as structured data, with ID, pillar, question, risk level (90 High, 59 Medium, 1 Low) and desired outcome. Risk levels were cross-checked against the lens appendix, and the skill is told never to invent or renumber items.
2. assessment-guide.md: a five-value status rubric (Compliant, Partially Compliant, Non-Compliant, Not Applicable, Not Reviewed), an evidence checklist and honesty rules. Every status must cite evidence from the repository, and items with no signal are marked Not Reviewed rather than guessed.
3. build_report.py: a deterministic script that renders the workbook and aborts on unknown IDs or invalid statuses, so a hallucinated item can’t ship.

A review is one request: point Claude at the repository and ask for it. Claude reads the README, architecture docs, infrastructure as code, CI/CD workflows and tests read-only, assesses all 150 items, builds the workbook and summarizes the top high-risk gaps. Humans then validate.

Smarsh tested the skill before trusting it, on two synthetic repositories, one of them a rough proof of concept with hardcoded credentials and a runaway-cost incident. Graded on eight expectations per repository, runs with the skill passed 16 of 16. Runs without it passed 8 of 16 and produced ad hoc checklists of 18 to 39 items with no link back to the lens.

What This Means for You

The real review took hours. Strengths included per-agent IAM roles scoped to named runtime ARNs and SigV4-signed MCP calls through AgentCore Gateway. The high-risk gaps were specific: the ingress authorizer with authentication disabled under a documented risk acceptance, no prompt-injection screening on message content that bad actors can influence, rate limiting still a TODO, and no automated cost controls.

The human step changed the result. Engineers merged several items into one anomaly-detection epic and upgraded one finding after pointing to dependency scanning the repository review had missed. Questions no repository could answer, such as InvokeAgentRuntime retry semantics and break-glass patterns for halting agents, became an itemized agenda for AWS Solutions Architects.

This is an AWS-published customer story, and the outcomes are Smarsh’s own early figures. The pattern of treating guidance as data, enforcing honesty rules and validating the output in code fits a theme Stack Archive has followed in AWS’s Well-Architected agent preview, where AWS applies AI to its own framework.

Source: AWS Architecture Blog

Content created with AI assistance and reviewed for accuracy.