EKS Fargate: How Serverless Kubernetes Runs One Pod per Node

Serverless Kubernetes on EKS Fargate doesn’t remove the node. It builds a new one for every pod and tears it down when the pod exits. A walkthrough of the internals in The New Stack shows how EKS does that without breaking Kubernetes’ expectations, and why the same design produces the limits platform teams run into: one network interface per pod, no DaemonSets, and pods that EKS will evict on its own schedule to patch them.

How It Works

The New Stack’s walkthrough, written from the EKS side, follows a pod through four stages.

1. Routing before scheduling. A cluster can mix EC2 nodes and Fargate, so placement has to be decided early. A Fargate profile selects pods by namespace and optional labels, and sets the private subnet and security group for each pod’s network interface. An AWS-managed mutating webhook in the control plane labels matching pods and sets their scheduler name to the Fargate scheduler. This happens before the pod is written to etcd, so the default scheduler never claims it.

2. A scheduler that creates capacity. The Fargate scheduler runs on the control plane and requests new capacity sized to the pod, instead of picking from existing nodes. It tentatively ties the pod to a nominated node so a retry can’t provision a second environment. The node registers when ready, the scheduler binds the pod, and when the pod terminates the node is removed.

3. Three network namespaces. Each pod gets its own Elastic Network Interface in the customer VPC, so security groups and flow logs apply per pod. Inside the environment, a platform namespace carries Fargate’s telemetry, credential vending and provisioning, a node namespace holds the kubelet and the ENI, and a workload namespace runs the pod. The task metadata endpoint is reachable from the node namespace but blocked from the workload, and EC2 instance metadata isn’t available, so applications should use IAM roles for service accounts.

4. A node in miniature. The runtime image matches the control-plane version and supplies the kubelet. A custom PodAdmitHandler admits only pods meant for the Fargate path, a second check after API-server admission. EKS also turns off the kubelet’s logs, profiling and debug-flag handlers, controls the authors say they contributed upstream.

What This Means for You

The same design sets the limits. Workloads that need node-level access fall outside the Fargate contract: DaemonSets, host networking, host ports, privileged operations and GPUs. Storage follows the same rule. HostPath and host-attached block storage need a persistent node, EBS can’t be mounted, and EFS is the supported option for shared durable storage, per the EKS Fargate documentation the article points to.

Networking needs planning. A design that fits on EC2, where many pods share a node’s interface, has to account for one ENI per Fargate pod, which means subnet capacity. Pods get no public IPs, so outbound access goes through NAT gateways or VPC endpoints. Because the ENI carries both Kubernetes management traffic and pod traffic, its security group has to allow what kubectl logs and exec need.

Patching is the operational surprise. EKS periodically replaces Fargate pods to apply security patches, using the Eviction API and respecting PodDisruptionBudgets. If a budget leaves no room and eviction can’t succeed by the scheduled termination time, EKS can eventually delete the pod anyway. Replicas and readiness probes decide whether the app keeps serving.

For teams weighing options, Stack Archive covered EKS Auto Mode, which manages EC2 nodes for you instead of removing them.

Source: The New Stack

Content created with AI assistance and reviewed for accuracy.