NVIDIA Releases GPU Fractioning Operator: Integrating CUDA 13.4 VRAM Isolation Capabilities into Kubernetes
CUDA 13.4 introduces application-transparent VRAM boundary enforcement for the first time, but Kubernetes clusters cannot directly utilize this capability. NVIDIA's open-source GPU Fractioning Operator, working in tandem with the NvFractions mode of KAI Scheduler v0.18.0, fills this orchestration gap: the scheduler writes per-container annotations, and the operator configures MPS daemons and enforces limits on nodes. The article explains the design trade-offs between two compute modes: time-slicing (serial execution with error isolation) and sm-sharing (parallel execution with a shared error domain).
Many AI inference and fine-tuning tasks require only a small fraction of a GPU's memory, yet allocating an entire card to each workload leaves significant capacity idle. In a technical blog post dated October 27, 2026, NVIDIA introduced a new underlying capability added in CUDA 13.4: memory boundary enforcement that is transparent to applications. When a workload declares a specific memory size, CUDA restricts it to that range; allocations exceeding the limit fail, as if running on a smaller GPU. This offers a guaranteed minimum value plus an expandable upper limit when memory is available. While CUDA provides this capability, Kubernetes clusters cannot invoke it directly—nodes must run MPS daemons, write limits and compute modes before container startup, and report usage by fraction rather than by whole GPU.
The GPU Fractioning Operator, paired with the NvFractions mode of KAI Scheduler v0.18.0, bridges this gap: the scheduler selects GPUs and writes per-container annotations, while the operator executes configurations on the node. There are two compute modes: time-slicing allows workloads to take turns using the entire GPU, each maintaining its own independent error domain; sm-sharing allocates streaming multiprocessors (SMs) simultaneously to multiple workloads, sharing a single error domain. A single GPU runs only one compute mode at any given time, and KAI Scheduler tracks this constraint per GPU rather than per node.