Lambda Durable Functions Telemetry: One Record per Execution

A Lambda durable function that suspends four times writes to four separate log streams, mixed in with every other execution running at the same time. That is why platform teams running durable workflows have struggled to answer a simple question like “which step failed most often this week”. AWS’s new Durable Execution Telemetry Exporter answers it by changing the unit of observability from the invocation to the whole execution.

How It Works

Durable functions, built with the Durable Execution SDK, can run for minutes, days or up to a year, across many invocations with waits and retries in between. The AWS Compute Blog explains why existing tools miss that shape. CloudWatch metrics give the function’s error rate, not the failing step. X-Ray and OpenTelemetry show one execution’s timing, but aren’t built for questions across thousands of runs.

The exporter is an open-source plugin for the SDK in TypeScript, Python, Java and Go. It runs in the same process as the handler, outside the request path. Because the SDK replays the handler from the start on every resume and passes checkpointed state to the plugin, the plugin can rebuild the whole execution so far on each invocation. It emits a cumulative snapshot, a DurableExecutionTelemetryRecord, with every step, wait, callback and invoke, plus timing, retries, status and errors. One record is one execution, so “slowest step” becomes an ordinary query.

Setup is a one-line change. By default, records go as JSON to the function’s own CloudWatch log group, with no new infrastructure or IAM permissions, and can be queried with Logs Insights. Thirteen exporters cover other destinations, including S3 for Athena, DynamoDB for live lookups, Aurora for joining with business data, EventBridge for alerting, and any OTLP or HTTPS endpoint. Records go to all configured exporters in parallel.

Three settings control cost and exposure:

• emitMode: on-complete by default, on-failure for the least overhead, or on-change to watch progress in near real time.
• samplingRate: a hash of the execution ARN decides, so a sampled execution sends all its records and an unsampled one sends none.
• content: input and output are included as-is by default. AWS calls that convenient in development and often unacceptable in production, and recommends a redacting transform.

A companion MCP server lets agents such as Kiro, Claude Code and Cursor query the records through five tools, including describe_schema and list_executions. On SQL destinations it runs only SELECT queries, or WITH statements ending in a SELECT, and checks each query before any AWS call.

What This Means for You

The exporter has a blind spot AWS states plainly. It runs inside the function, so it can’t see terminal states the Lambda service sets without an invocation: STOPPED after a manual stop, and TIMED_OUT when the execution timeout passes. Full coverage needs a small handler on Lambda’s durable execution lifecycle events in EventBridge.

Two more details matter in production. Operation names are used verbatim as keys, and a dot is a field separator in Logs Insights and OpenSearch, so stick to letters, digits, hyphens and underscores. And keeping the full tree of child operations across resumes forces an extra replay pass, so enable only the depth you need.

Stack Archive covered Datadog’s automatic tracing for durable functions, which builds a per-execution flame graph. AWS’s exporter is the first-party option for teams that want the records in their own stores.

Source: AWS Compute Blog

Content created with AI assistance and reviewed for accuracy.