Red Hat build of Kueue 1.3: Enhanced batch workload management on Kubernetes

Red Hat build of Kueue 1.3 is now available. This Kubernetes-native job queuing controller manages how batch workloads share resources in a cluster. The 1.3 release adds support for specialized workload controllers and updates the API for future development.

Key highlights of Kueue 1.3

The 1.3 release helps Kueue schedule batch jobs more efficiently in Kubernetes environments. Key changes include support for JobSet and LeaderWorkerSet workloads and the introduction of v1beta2 APIs for long-term stability.

Managing batch jobs with JobSet

Kueue 1.3 now integrates with JobSet. JobSet is a Kubernetes controller that manages batches of similar jobs. These jobs often require coordinated scaling, failure handling, and lifecycle management for high-performance computing and AI workloads.

What this means for you:

• Coordinated resource allocation: Define JobSet workloads and manage resource use through Kueue queuing and resource policies.
• Improved reliability: Use JobSet features for stable batch execution with Kueue fair-sharing and preemption logic.
• Simplified operations: Define complex, multi-job dependencies within a single JobSet for Kueue to schedule.

Coordinating distributed workloads with LeaderWorkerSet

Following the theme of deeper workload integration, Kueue 1.3 also includes support for LeaderWorkerSet. This pattern helps machine learning (ML) and distributed computing workloads where one leader process coordinates multiple worker processes.

Key benefits of LeaderWorkerSet support:

• Optimized for distributed ML: Schedule and manage distributed serving workloads to ensure all processes receive resources at once.
• Better resource use: Kueue admits the LeaderWorkerSet only when resources are available, which prevents partial job startups and wasted compute cycles.
• Fair sharing: Manage these specialized workloads alongside standard jobs, using Kueue quota management.

Introduction of v1beta2 APIs

This release introduces the v1beta2 API versions for key resources. The v1beta1 APIs are now deprecated and will be removed in a future release.

We recommend moving to the new APIs, though v1beta1 remains supported for now.

The shift to v1beta2 includes:

• Updated structure: Refinements to the API schema make resource definitions clearer and more intuitive.
• Scalability and support: This update lays the groundwork for future features and stricter validation.

Get started with Red Hat build of Kueue 1.3

Current users can now explore these integrations and migrate to the v1beta2 APIs.

After installing the operator, apply the following YAML to enable LeaderWorkerSet and JobSet:

apiVersion: kueue.openshift.io/v1
kind: Kueue
metadata:
  labels:
    app.kubernetes.io/name: kueue-operator
    app.kubernetes.io/managed-by: kustomize
  name: cluster
  namespace: openshift-kueue-operator
spec:
  managementState: Managed
  config:
    integrations:
      frameworks:
      - BatchJob
      - Pod
      - Deployment
      - StatefulSet
      - JobSet
      - LeaderWorkerSet

Find installation instructions and release notes in the Red Hat documentation for Kueue.

To learn more about AI workloads in Red Hat OpenShift, see the official documentation.