Spot Instance Termination with Karpenter

AWS spot instance termination notice is a critical aspect to consider when integrating Amazon EC2 spot instances with Kubernetes clusters. Karpenter helps us dynamically manage the lifecycle of nodes in a Kubernetes cluster.

Unlike the traditional Cluster Autoscaler, which depends on predefined node pools, Karpenter selects the most appropriate instance type based on workload requirements and cloud resource availability. It scales up new nodes and de-provisions unused ones, optimizing your cluster's performance automatically.Karpenter can integrate with Amazon EC2 Spot Instances, which offer AWS customers access to unused compute capacity at reduced rates (up to 90% off the On-Demand price).

The challenge, however, is that these Spot Instances can be terminated when AWS reclaims the capacity, posing a risk to workload availability.

Let's take a look at the best practices for managing Karpenter's spot instance termination notifications and strategies for effective Karpenter consolidations.

Understanding Spot Instance Terminations

Amazon provides a two-minute Spot Instance Termination Notice before the instance is stopped or reclaimed. Managing these interruptions effectively is important, especially for production workloads where downtime can be risky. Without effective strategies, sudden termination of Spot Instances can lead to pod evictions, service latency, and application downtime. This is where Karpenter does its job by dynamically responding to these notifications, orchestrating replacement Spot Instances or consolidating your cluster to minimize the impact of termination. Let’s first understand how Karpenter handles Spot instance terminations.

Managing AWS Spot Instance Termination Notifications with Karpenter

Karpenter can handle Spot interruptions, but it requires additional setup to enable this capability. You need to create an SQS queue that Karpenter monitors for interruption events. AWS EventBridge forwards these events to the SQS queue. Then, configure Karpenter using the --interruption-queue-name CLI argument with the name of the provisioned queue. This setup allows Karpenter to dynamically manage Spot interruptions through node drainage and replacement, as explained below:

1. Spot Termination Notification Reception: Karpenter listens for the EC2 Spot interruption notice either through the instance metadata service or the SQS queue you configure. Upon receiving a two-minute termination notice, Karpenter triggers parallel processes to mitigate/reduce the interruption.

2. Graceful Node Drainage: Karpenter initiates node drainage to evict pods gracefully. At the same time, it provisions a replacement node. The parallel execution ensures that workloads are rescheduled without downtime.

3. Node Replacement and Scheduling: Karpenter dynamically provisions a new Spot or On-Demand instance and works with the Kubernetes scheduler to reschedule evicted pods onto the new node or other available capacity. The synchronized process prevents service disruptions.

Best Practices for AWS Spot Instance Termination Handling

1. Use Pod Disruption Budgets (PDBs): Define PDBs for critical workloads to limit concurrent evictions and maintain availability during node drain operations. This makes sure some replicas of your application remain available during interruptions.

apiVersion: policy/v1
kind: PodDisruptionBudget
metadata:
  name: my-app-pdb
spec:
  minAvailable: 2
  selector:
    matchLabels:
      app: my-app

2. Optimize Grace Periods: Configure appropriate termination grace periods for your pods to ensure Kubernetes waits long enough for graceful shutdowns but avoids indefinite hanging during node drains. Set a reasonable terminationGracePeriodSeconds value based on your application’s shutdown behavior to avoid pod termination without proper cleanup.

3. Monitor Metrics: Use monitoring tools like Prometheus to track interruptions caused by Spot interruptions and Grafana for visualization of these metrics. These metrics ensure you are not having an impact on your service’s availability and provide insights into how frequently your cluster experiences Spot interruptions and how effectively Karpenter manages them. You can also check Grafana dashboard configuration here

>> Take a look at Nodepool Selection Strategies: Performance vs. Cost

Strategies for Effective Consolidations Using Karpenter

Karpenter’s consolidation capability optimizes node usage by identifying underutilized nodes and decommissioning them when appropriate. The introduction of Spot-to-Spot consolidation enhances Karpenter’s ability to migrate workloads between Spot Instances without reverting to On-Demand capacity.

1. Spot-to-Spot Consolidation: Improving Cost Efficiency

With Karpenter v0.34.0 and later, Spot-to-Spot consolidation allows for the dynamic replacement of Spot Instances with other Spot Instances, rather than falling back to On-Demand nodes. This feature allows your cluster to continue to leverage the cost savings of Spot Instances while reducing the probability of interruptions.
For example, if Karpenter detects that a Spot Instance is being charged at a higher price point or is underutilized, it can proactively decommission the node and replace it with another cheaper Spot Instance that offers the same or better capacity. This results in cost optimization without compromising workload performance.

To enable Spot-to-Spot consolidation, configure the NodePool with the consolidation:

apiVersion: karpenter.sh/v1
kind: NodePool
metadata:
  name: app-nodepool
spec:
  template:
    metadata:
      labels:
        workload-type: apps
    spec:
      nodeClassRef:
        name: spot-instance-class
      requirements:
        - key: karpenter.sh/capacity-type
          operator: In
          values: ["spot"]
        - key: karpenter.k8s.aws/instance-family
          operator: In
          values: ["c", "m", "r"]
        - key: karpenter.k8s.aws/instance-size
          operator: NotIn
          values: ["nano", "micro", "small", "medium"]
        - key: karpenter.k8s.aws/instance-hypervisor
          operator: In
          values: ["nitro"]
  limits:
    cpu: 120
    memory: 120Gi
  disruption:
    consolidationPolicy: WhenEmpty

2.  Right-Sizing Instances with Karpenter

Karpenter right-sizes instances based on workload resource needs, ensuring resources are neither over-allocated nor under-allocated. This right-sizing is valuable during consolidations, where Karpenter can dynamically provision smaller or cheaper instances to replace over-provisioned nodes. This process happens transparently to the user, optimizing cost and resource utilization in real time.

PerfectScale can enhance this process by providing deeper insights and optimization recommendations. PerfectScale analyzes historical workload patterns, resource utilization trends, and cost metrics to suggest optimal instance types.

>> Take a look at Getting the most out of Karpenter with PerfectScale

‍
3. Utilizing Flexible Instance Types

Define various instance types that can be provisioned based on your workload’s needs. By specifying multiple instance families (e.g., c5, m5, r5), you give Karpenter the flexibility to select the most appropriate and available Spot Instances, reducing the risk of capacity shortages.

spec:
  requirements:
    - key: "instance-type"
      operator: In
      values: ["c5.large", "m5.large", "r5.large", "t3.medium"]

>> Take a look at Karpenter: The Ultimate Guide

4. Node Expiration for Idle Nodes

Karpenter enables the disruption of idle or underutilized nodes using expiration policies. This is configured in the NodePool spec by defining an expiration duration. For example:

apiVersion: karpenter.sh/v1
kind: NodePool
metadata:
  name: app-nodepool
spec:
  disruption:
    expireAfter: 1h

This configuration will ensure nodes are terminated after one hour of existence, helping optimize cluster costs and maintain a clean resource environment.

In summary, Karpenter is a powerful tool for managing Spot interruptions and optimizing cluster consolidations. With proper setup, including SQS queues for Spot instance termination handling and thoughtful configuration of NodePools, you can minimize downtime and maximize cost efficiency. By implementing the best practices shared in this article, you can maintain a resilient, high-performing Kubernetes cluster that effectively uses spot instances.