Kubernetes v1.31: What's New and Improved?

Kubernetes 1.31, codenamed "Elli," marks the first release after Kubernetes' 10th anniversary. This version introduces 45 enhancements, including 11 features graduating to stable status, several beta improvements, and exciting new alpha features. The release name "Elli" symbolizes the joyful and dedicated spirit of the Kubernetes community, represented by a cute dog wearing a sailor's cap.

Let's explore the key enhancements in Kubernetes v1.31, with a focus on reliability and resource management:

Stable Features

1. Improved Ingress Connectivity Reliability for kube-proxy

Feature group: sig-network #3836

Kubernetes v1.31 enhances the reliability of ingress connectivity through improvements in kube-proxy. This feature addresses one of the most common challenges in Kubernetes load balancer implementations: synchronization between various components to prevent traffic drops during node termination events.

The core of this enhancement lies in implementing a connection-draining mechanism within kube-proxy. This mechanism is designed to terminate nodes that are exposed by LoadBalancer services with an externalTrafficPolicy set to Cluster. By introducing this feature, Kubernetes establishes best practices for cloud providers and load balancer implementations, reducing the risk of service disruptions during cluster scaling or node maintenance activities.

To use this feature in your Kubernetes cluster:

a. Make sure kube-proxy is running as the default service proxy.

b. Verify that your load balancer supports connection draining.

apiVersion: v1
kind: Service
metadata:
  name: my-service
spec:
  type: LoadBalancer
  externalTrafficPolicy: Cluster
  ports:
    - port: 80
      targetPort: 8080
  selector:
   app: my-app
  

It enhances reliability by reducing service disruptions during cluster scaling or node maintenance.

Note: No specific changes are required to enable this feature. It has been automatically enabled in kube-proxy since v1.30 and has now been promoted to stable status in v1.31. 

2. AppArmor Support in Kubernetes

Feature-group: sig-node #24

Kubernetes v1.31 improves container security with the graduation of AppArmor support to stable status. AppArmor, a Linux kernel security module, provides an additional layer of protection by allowing administrators to define fine-grained, mandatory access control profiles for applications. This feature enhances the overall security posture of Kubernetes clusters.

Prior to v1.30, AppArmor profiles were applied using annotations, which could be cumbersome. With the stable release, Kubernetes now introduces a more streamlined approach by utilizing the appArmorProfile.type field within the container's securityContext. This change not only simplifies the configuration process but also aligns with Kubernetes' broader move towards using fields instead of annotations for core functionality.

To implement AppArmor profiles in your Kubernetes deployments, you can now specify the desired profile directly in the container specification. For example:

spec:
  containers:
  - name: myapp
    securityContext:
      appArmorProfile:
        type: LocalhostProfile
        localhostProfile: runtime/default

This configuration applies the runtime/default AppArmor profile to the myapp container, providing a baseline level of protection.

3. Persistent Volume Last Phase Transition Time

Feature-group: sig-storage #3762

Kubernetes PersistentVolumes (PVs) are a critical component in managing storage resources within a cluster. They represent a piece of storage in the cluster that has been provisioned by an administrator or dynamically provisioned using Storage Classes. Understanding the lifecycle and transitions of PersistentVolumes is crucial for effective storage management and troubleshooting.

In Kubernetes v1.31, the Persistent Volume last phase transition time feature has graduated to General Availability (GA). This enhancement adds a new field to the PersistentVolumeStatus, providing valuable timing information about volume phase transitions. The feature allows cluster administrators and developers to track when a PersistentVolume last changed its phase, and provides insights into the storage provisioning and binding process.

With this feature, every PersistentVolume object now includes a .status.lastTransitionTime field. This timestamp indicates when the volume last transitioned between its various phases (Pending, Bound, or Released). It's important to note that this change isn't retroactive; the new field will be populated when a PersistentVolume is updated and transitions between phases after upgrading to Kubernetes v1.31.

Beta Features

4. Multiple Service CIDRs: Flexible IP Management for Services

Feature group: sig-network #1880

Kubernetes v1.31 introduces an enhancement to service IP management with the beta release of Multiple Service CIDRs disabled by default. This feature addresses long-standing challenges related to IP exhaustion in large or long-lived clusters, offering a more dynamic approach to managing service IP ranges.

Generally, service IP ranges were defined during cluster creation as a hardcoded flag in the kube-apiserver. This static allocation often led to IP exhaustion issues, requiring complex and risky maintenance operations to expand, shrink, or replace the assigned Service CIDR range. With this new feature, cluster administrators can dynamically modify Service CIDR ranges without causing downtime.

To use this feature in your Kubernetes v1.31 cluster:

1. Enable the MultiCIDRServiceAllocator feature gate

2. Enable the networking.k8s.io/v1alpha1 API group

Once enabled, the control plane replaces the existing etcd allocator with a new implementation using IPAddress and ServiceCIDR objects.

This is how you can interact with the new IP address objects:

kubectl get services
NAME         TYPE        CLUSTER-IP        EXTERNAL-IP   PORT(S)   AGE
kubernetes   ClusterIP   2001:db8:1:2::1           443/TCP   3d1h

kubectl get ipaddresses
NAME              PARENTREF
2001:db8:1:2::1   services/default/kubernetes
2001:db8:1:2::a   services/kube-system/kube-dns

5. Traffic Distribution for Services: Enhanced Routing Control

Feature-group: sig-network #4444

Kubernetes Services has long been a fundamental component for exposing applications within a cluster. However, their traffic distribution mechanism was previously limited, often leading to suboptimal routing in geographically distributed environments. In Kubernetes v1.31, the Traffic Distribution for Services feature moves to beta and is enabled by default, marking an advancement in service networking capabilities.

This feature introduces the trafficDistribution field in the Service specification, providing administrators with more granular control over traffic routing. It's beneficial for multi-zone cluster setups, where network topology plays a crucial role in performance and cost optimization.

apiVersion: v1
kind: Service
metadata:
  name: my-service
spec:
  selector:
    App: main-app
  ports:
    - protocol: TCP
      port: 80
      targetPort: 8988
  trafficDistribution:
    PreferClose: true

In this configuration, the Service attempts to route traffic to endpoints in the same zone as the client. If no endpoints are available in the client's zone, traffic will be routed cluster-wide.

To know more about this feature, refer to our previous article Traffic Distribution for Services in Kubernetes v1.30

6. VolumeAttributesClass: Dynamic Volume Modifications

Feature-group: sig-storage #3751

Kubernetes persistent volumes have traditionally been static in nature, with their attributes set at the time of provisioning. This limitation has made it challenging for applications with varying storage requirements to utilize resources and manage costs. The VolumeAttributesClass feature, now moving to beta in Kubernetes v1.31, addresses this issue by providing a mechanism for dynamically modifying volume attributes post-provisioning.

VolumeAttributesClass introduces a new API that allows administrators to define classes of storage with mutable attributes. This feature enables workloads to vertically scale their volumes on-line, balancing cost and performance based on current needs. It's useful for applications that experience fluctuating I/O demands or need to adjust storage performance characteristics without recreating volumes.

To use VolumeAttributesClass in v1.31, you need to enable the VolumeAttributesClass feature gate and the storage.k8s.io/v1beta1 API group. This can be done by adding the following flags to your kube-apiserver, kube-controller-manager, and kube-scheduler:

--feature-gates="...,VolumeAttributesClass=true"
--runtime-config=storage.k8s.io/v1beta1=true

Note: This feature requires CSI driver support through the ModifyVolume API. Not all storage providers may support dynamic modifications, so check your CSI driver's capabilities before implementing this feature.

Alpha Features

7. Enhanced Dynamic Resource Allocation (DRA) APIs

Feature-group: sig-node #3063

Kubernetes has long faced challenges in managing specialized hardware resources like GPUs, FPGAs, and other accelerators. The Dynamic Resource Allocation (DRA) feature aims to address these challenges by providing a flexible way to allocate and manage such resources.

The new Dynamic Resource Allocation (DRA) APIs in Kubernetes v1.31 bring several improvements. Structured parameters enhance transparency in resource management, enabling advanced features like cluster autoscaling. The updated kubelet support allows version skew between the kubelet and control plane, offering greater flexibility in cluster upgrades. Scheduler-based ResourceClaim allocation leverages these structured parameters for more efficient and intelligent resource distribution. To maintain backward compatibility, the "classic DRA" is still supported through a separate feature gate named  DRAControlPlaneController, ensuring continued support for existing allocation policies.

To enable the new DRA features, you need to ensure the appropriate feature gates are enabled. 

apiVersion: resource.k8s.io/v1alpha1
kind: ResourceClaim
metadata:
  name: heavy-workload
spec:
  resourceClassName: nvidia-gpu
  parameters:
    gpuType: "A100"
    memory: "20Gi"
    compute: "4"
 

This example creates a ResourceClaim for requesting a GPU resource. It specifies a resource of type "nvidia-gpu" and requests an NVIDIA A100 GPU with 20GB of memory and 4 compute units.

8. Support for Image Volumes

Feature-group: sig-node, sig-storage #4639

Kubernetes v1.31 introduces an alpha feature that allows using Open Container Initiative (OCI) compatible images as native volume sources in pods. This new capability, called ImageVolume, is designed to meet the growing demands of Artificial Intelligence (AI) and Machine Learning (ML) workloads, which require efficient handling of large datasets and model artifacts.

The ImageVolume feature enables users to specify an OCI image reference as a volume in a pod definition, which can then be mounted within containers. This approach leverages existing OCI standards for storing and distributing content.

To use this feature, you need to enable the ImageVolume feature gate. Defining an image volume in a pod specification:

apiVersion: v1
kind: Pod
metadata:
  name: image-volume-example
spec:
  volumes:
  - name: model-data
    image:
      reference: docker.io/myorg/ml-model:v1.0
      pullPolicy: IfNotPresent
  containers:
  - name: ml-inference
    image: docker.io/myorg/inference-engine:latest
    volumeMounts:
    - name: model-data
      mountPath: /models

In this example, we define a volume named model-data using an OCI image as the source. The container ml-inference then mounts this volume at the /models path, allowing the inference engine to access the model data directly.

Note: As with any alpha feature, it's recommended to test thoroughly in non-production environments before considering it for critical workloads.

9. Exposing Device Health Information Through Pod Status

Feature-group: sig-node #4680

In version 1.31, Kubernetes introduces an alpha feature that enhances visibility into device health within Pods, providing crucial information for troubleshooting and maintaining the reliability of applications that depend on specific hardware resources.

Prior to Kubernetes v1.31, determining whether a Pod was associated with a failed device required querying the PodResources API, which wasn't always efficient. The new feature, disabled by default in its alpha state, adds a field called allocatedResourcesStatus to each container's status within the Pod's .status. This field provides health information for each device assigned to the container, offering a more direct way to monitor device health.

To enable this feature, you need to activate the ResourceHealthStatus feature gate. Once enabled, you can inspect the health status of allocated devices directly through the Pod's status.

10. Fine-Grained Authorization Based on Selectors

Feature-group: sig-auth #4601

Kubernetes v1.31 introduces a powerful new feature that significantly enhances the platform's authorization capabilities, and allows for fine-grained authorization based on selectors. The feature enables webhook authorizers and future in-tree authorizers to permit list and watch requests with specific conditions based on label and field selectors.

This new functionality addresses a long-standing limitation in Kubernetes' Role-Based Access Control (RBAC) system, which lacked the granularity needed for complex authorization scenarios. With this feature, it's now possible to create highly specific access policies. For example, an authorizer can allow a user to list only the pods running on a particular node, or permit a developer to watch all Secrets in a namespace except those labeled as confidential.

The benefits of this enhancement are manifold. It allows for better implementation of the principle of least privilege, improving overall cluster security. By reducing the scope of accessible resources, it can also help improve cluster performance and scalability. 

Kubernetes 1.31 brings a total of 45 Kubernetes Enhancement Proposals (KEPs) implemented. These enhancements include Kubernetes functionalities, storage improvements, networking advancements, security measures, and more. Beyond the major changes we've discussed, there are other features added by the k8s team. We encourage you to have a look at the Kubernetes v1.31 release notes and check this for more details.