
AWS DevOps Agent Custom Skills: Aurora Triage in 14 Minutes
Cloud Platform EngineeringAn Aurora PostgreSQL incident that took an on-call engineer 52 minutes to resolve took 14 once AWS DevOps Agent followed a custom Skill written from three senior database engineers’ triage habits. AWS measured that 73 percent cut on just three simulated incidents, so SRE teams should treat it as a pattern to copy, not a benchmark to quote.
The Framework
AWS’s walkthrough on reducing database incident MTTR with a custom DevOps Agent Skill starts from a familiar problem. Troubleshooting a degraded Aurora PostgreSQL cluster means correlating connection pools, replication, queries, storage and parameters, which takes experienced engineers 45 to 60 minutes. The knowledge of which branch to check first “lives in the heads of senior database reliability engineers.”
Skills are how that knowledge reaches the agent. Each is a directory of Markdown files: a required SKILL.md with frontmatter and investigation procedures, plus optional references such as threshold tables and query templates. The agent reads each Skill’s description to decide relevance, and Skills follow the open Agent Skills specification.
Step 1: Interview the Experts Before Writing Markdown
The team interviewed three senior DBREs and documented a shared mental model: assess impact scope (writer, readers or cluster-wide), correlate the timeline with deployments, scaling events or traffic changes, branch into a specific failure mode, then output root cause with immediate, short-term and long-term actions.
Step 2: Build One Decision Tree, Not Five Checklists
Instead of five separate Skills, AWS built one procedural Skill covering five failure modes: connection pool exhaustion, replication lag, slow query regression, storage pressure and parameter drift. The logic branches (“if metric X exceeds threshold Y, check Z before proceeding to W”) rather than running a linear checklist.
Step 3: Encode Priority and Thresholds Explicitly
Order matters because symptoms overlap. Connection exhaustion is checked first because it “masks every other symptom”: the agent compares DatabaseConnections with max_connections and confirms exhaustion above 85 percent, then separates RDS Proxy pinned sessions from the application pool. Replication lag uses stated thresholds, under 20 ms normal and over 100 ms critical, before checking writer pressure and reader CPU. Parameter drift gets special attention because its symptoms mimic other failures, and without guidance the agent “chases symptoms rather than tracing back to the configuration change.”
Step 4: Map Findings to Causes
Each branch ends by interpreting patterns, not just reporting metrics. Connections rising steadily point to a leak in code; spikes that follow traffic mean an undersized pool; spikes right after a deployment mean a new code path isn’t releasing connections.
Where It Doesn’t Apply
The 73 percent figure comes from three simulated incidents in a controlled internal test, and the Skill is specific to Aurora PostgreSQL metrics such as AuroraReplicaLag and Performance Insights data. It needs an Agent Space with access to the cluster’s account and Performance Insights enabled. AWS also positions it against its own workshop, whose two example Skills are a single-file checklist and an architecture-context Skill, neither of which “proves measurable MTTR reduction.”
Stack Archive has tracked the agent’s growing reach, from autonomous incident response to third-party integrations. Custom Skills are the piece that lets teams add their own runbooks.
The So What
SRE teams already running AWS DevOps Agent should pick the one incident type that most often needs a senior engineer, interview those engineers, and encode their triage as a single prioritized decision tree with explicit thresholds. Measure MTTR on that incident class before and after, rather than relying on AWS’s three-incident result.
Source: AWS DevOps & Developer Productivity Blog
Content created with AI assistance and reviewed for accuracy.
Join the conversation
Stack Insiders is our free community for readers who want to go deeper — share resources, ask questions, and connect with others across every vertical we cover.
Join Stack Insiders →