AWS Lambda GPU Functions: Serverless Inference on EC2 G-Series

AWS Lambda GPU Functions: Serverless Inference on EC2 G-Series

Cloud Platform Engineering

AWS Lambda can now run functions on EC2 G-series GPUs, and AWS’s own demo went from a Hugging Face model ID to a live, OpenAI-compatible endpoint in about 10 minutes with a one-line Dockerfile. For platform teams who’ve been running small GPU fleets for custom inference, the question is now whether the convenience is worth a 9.75 percent fee on top of the instance price.

9.75%
management fee Lambda adds to the EC2 price of GPU instances on Managed Instances

What Happened

AWS announced GPU support for Event Functions on Lambda Managed Instances, generally available now on G4dn, G5, G6, G6f, Gr6, Gr6f and G6e instances, with more families to follow. Functions ship as OCI container images. Lambda handles drivers, CUDA versions, provisioning, routing and scaling, so there are no AMIs to reconcile and no cluster or device plugins to run.

Pricing is the underlying EC2 instance fee, which Savings Plans and Reserved Instances cover, plus the 9.75 percent management fee and the standard $0.20 per million requests, per AWS Lambda pricing. AWS curates base images for CUDA, CuPy, PyTorch, and the vLLM and SGLang serving engines. Limits include container images up to 32 GB, 32 GB of ephemeral storage, 3 to 48 GB of accelerator memory and up to 48 concurrent requests per execution environment.

Why It Matters

AWS names the problem it’s targeting: GPUs are expensive and usually underused, because a single small inference task rarely saturates one. Lambda’s answer is packing. Each execution environment can serve several concurrent invocations on the same GPU, controlled by a per-environment concurrency setting. In AWS’s demo, four concurrent requests to a vLLM function finished in about the same time as one, sharing one copy of the model in GPU memory.

The scope is deliberately narrow. AWS says the feature is optimized for inference, not training, and for custom models rather than large foundation models already on Bedrock. Its examples include computer vision, speech, recommendation, document analysis, small-to-mid-size language models, media transcoding and rendering. GPU functions get the extended Managed Instances timeout of up to 90 minutes for asynchronous and event-source invocations, enough for batch transcoding or multi-step inference.

What This Means for Builders

The main gain is integration. A GPU function is triggered by the same 200-plus event sources as any Lambda function, including S3, SQS, API Gateway and EventBridge, with the same retries and error handling. AWS pitches it as one step in a larger workflow: Lambda durable functions orchestrate around it, and Lambda MicroVMs can run an agent’s tool-use code on CPU before calling the GPU function for inference.

Cold starts move to deploy time. Managed Instances functions run from a published version, and publishing provisions the environments, downloads the weights and warms the engine before the version goes live. Swapping models means changing the MODEL_ID environment variable and publishing a new version, with no image rebuild. For production, AWS recommends copying weights to S3 in the same Region and streaming them with VLLM_LOAD_FORMAT=runai_streamer, which removes the Hugging Face Hub dependency.

There are practical catches. New accounts may start with a G-series quota of 0, so request an increase first. Listing several instance types in the capacity provider lets Lambda fall back across generations when capacity is short. For guaranteed GPUs, an On-Demand Capacity Reservation matching the configuration is used automatically. GPU utilization, memory and power metrics show up in CloudWatch with no extra setup.

The So What

Platform teams running custom inference on a handful of self-managed G-series instances should price one workload on Lambda GPU functions. Compare the instance price plus the 9.75 percent fee against the driver, scaling and event-integration work you’d stop doing, and against the utilization gain from packing concurrent requests. Keep training and large-model serving where they are; this is built for event-driven inference steps.

Source: AWS News Blog

Content created with AI assistance and reviewed for accuracy.

💬

Join the conversation

Stack Insiders is our free community for readers who want to go deeper — share resources, ask questions, and connect with others across every vertical we cover.

Join Stack Insiders →

Newsletter coming soon.

Curated digests across AI, biohacking, photography, travel, and more. Be the first to know when we launch.