ReadingResearchRadarInvestment frameworkAI Company Primary Sources
Sign in / Sign up
← Back to reading
NewsMaterial4 min readDisconfirmAI Editorial

NVIDIA Workload Map: Migrating AI Workload Structural Knowledge from Adapter Code to Declarative Definitions

An NVIDIA technical blog post explains how Workload Map migrates the structural knowledge of AI workloads on Kubernetes from compiled adapter code to auditable YAML data definitions, enabling schedulers, controllers, and platforms to perform workload inspection, gang scheduling, and unified management through a single description.

2026-11-03 00:00 GMT+8

When running distributed AI training or inference tasks on Kubernetes, a real-world workload is rarely a single Pod. It is a small resource tree: several groups of Pods, each with a role, replica count, Pod template, and status dependent on the workload type. Frameworks such as JobSet, LeaderWorkerSet, PyTorchJob, and RayCluster encode this information in different ways. Teams building schedulers, controllers, dashboards, or cost tools therefore face a combinatorial explosion problem: for every new framework CRD integrated, they must write a set of adapters to answer the same three questions—what is the structure of the workload? How is the state interpreted into operational phases? And how can Pod specifications be modified without breaking the workload?

The NVIDIA Run:ai team encountered this bottleneck while building their platform. In an NVIDIA technical blog post published in November 2026, Alon Lavian (Product Manager at NVIDIA Run:ai, responsible for positioning and promoting Workload Map), Yuval Grauer (Engineering Lead at NVIDIA and maintainer of Workload Map), and Ron Lev (Senior Software Engineer at NVIDIA and technical lead for Workload Map) described the current situation: every new customer framework requires another adapter; thus, adapter development becomes a roadmap item scheduled per framework rather than a one-time cost. This is why support for new workload types often has to wait for a version release.

Workload Map (formerly Karta; the CRD kind remains Karta) proposes moving workload structural knowledge from compiled code to declarative data. It consists of a Kubernetes CRD and a Go interpretation library. Each definition uses YAML to describe the Pod template path, replica count calculation method, and status field location for that workload type. The article uses JobSet as an example to show key parts: the root component translates the JobSet-specific replicatedJobsStatus field into unified Running, Completed, and Failed phases using jq expressions; child components use path expressions to locate the Pod template for each replicated Job and calculate the actual number of Pods by multiplying replicas by parallelism.

Sign in / Sign up
ReadingResearchRadarInvestment frameworkAI Company Primary Sources

This definition provides more than just status translation. statusMappings maps the distinct native states of various frameworks into a unified vocabulary of Running, Completed, and Failed—since JobSet does not directly report a Running condition, the definition uses a jq expression to check if any Job is ready and active with no failures; when RayJob or LeaderWorkerSet are described, their respective different status fields are mapped to the same shared categories. Meanwhile, path expressions in the definition allow consumers to extract Pod templates, replica counts, metadata, and understand the workload hierarchy, as well as update Pod specs, labels, and annotations across all instances. Consumer tools therefore do not need to write dedicated parsing logic for each framework but instead complete inspection, scheduling, and management through the same structural description.

The article demonstrates scenarios where the same JobSet definition is used by three consumers. The Workload Map CLI loads the definition and parses live objects in the cluster, outputting a workload tree containing leader and worker roles with a total request of 12 GPUs. KAI Scheduler uses the gangScheduling instruction in the definition for Pod grouping and gang admission; for workload types without native plugins, Workload Map serves as a fallback pod grouper providing structural information, while existing native plugins retain priority. NVIDIA Run:ai registers catalog definitions via a single API call, managing JobSet as a single workload. Additionally, the multi-cluster scheduling tool Elotl Nova uses Workload Map to interpret workload structure and resource requirements during cluster selection.

The project ships with a pre-built catalog covering JobSet, PyTorchJob, RayCluster, RayJob, RayService, KServe, Knative, MPIJob, NVIDIA NIM, LeaderWorkerSet, Milvus, Grove PodCliqueSet, and NVIDIA Dynamo. New workload types are added by writing YAML definitions, requiring no modification to consumer code. Workload Map is open-sourced under the Apache 2.0 license and is already in use in NVIDIA Run:ai production environments.

It should be noted that this approach shifts the cost from writing adapter code to writing and reviewing definition data. jq expressions are sensitive to framework API versions—if the structure of JobSet's replicatedJobsStatus changes in subsequent versions, the definition must be updated synchronously. The article does not report data on time savings for cross-framework adapter development, nor does it provide independent third-party verification of the accuracy of catalog definitions. For schedulers that have already written native plugins for specific frameworks, Workload Map serves only as a fallback and does not replace existing logic.

Sources

  • developer.nvidia.comNVIDIA Technical Blog↗
Continue researching →