From 8d6b6d8313d71440df93a55021b9b3ece809d071 Mon Sep 17 00:00:00 2001 From: Andrei Kvapil Date: Mon, 7 Sep 2026 18:32:40 +0200 Subject: [PATCH 1/2] docs(gpu): add a GPU overview page that maps the five paths GPU documentation is spread over five pages in four sections, and nothing tells a reader which one applies to them. The variant choice is made once per node and is not reversible by a values edit, so a reader who lands on the wrong page first pays for it on the host. Adds one orientation page under Learn Cozystack: a table from consumer to path, the reason the three operator variants are mutually exclusive, how tenant clusters stack on passthrough, and the limits worth knowing before planning around them (MIG, vGPU licensing, HAMi glibc, HAMi over the container variant). No procedure is restated, so the page does not go stale when the underlying guides change. Assisted-by: LLM Signed-off-by: Andrei Kvapil --- content/en/docs/next/guides/gpu.md | 56 ++++++++++++++++++++++++++++++ content/en/docs/v1.6/guides/gpu.md | 56 ++++++++++++++++++++++++++++++ 2 files changed, 112 insertions(+) create mode 100644 content/en/docs/next/guides/gpu.md create mode 100644 content/en/docs/v1.6/guides/gpu.md diff --git a/content/en/docs/next/guides/gpu.md b/content/en/docs/next/guides/gpu.md new file mode 100644 index 00000000..ce446428 --- /dev/null +++ b/content/en/docs/next/guides/gpu.md @@ -0,0 +1,56 @@ +--- +title: "GPU in Cozystack" +linkTitle: "GPU" +description: "How Cozystack serves GPUs to pods, virtual machines and tenant Kubernetes clusters: which path to pick, what each one requires, and where the boundaries are." +weight: 35 +--- + +Cozystack has no single "GPU feature". It has several paths that serve a GPU to a different kind of consumer, and they are not interchangeable — the choice is made once per node, at the level of who owns the host driver, and it decides what that node can serve. Each path is documented in full on its own page. This one is the map: how to pick a path, what it costs, and what it rules out. + +## Pick a path + +| The GPU is consumed by | Runs on | Sharing granularity | Selected through | Guide | +| --- | --- | --- | --- | --- | +| Containers — CUDA pods, training, inference | Management cluster | Whole GPU | `cozystack.gpu-operator`, `container` variant | [Containerized GPU Workloads](/docs/next/operations/gpu-container-workloads/) | +| One virtual machine | Management cluster | Whole GPU | `bundles.iaas.gpuOperatorVariant: default` | [GPU Passthrough](/docs/next/virtualization/gpu/) | +| Several virtual machines sharing one card | Management cluster | vGPU profile per VF | `bundles.iaas.gpuOperatorVariant: vgpu` | [NVIDIA vGPU](/docs/next/virtualization/vgpu/) | +| Containers inside a tenant Kubernetes cluster | Tenant cluster | Whole GPU | `addons.gpuOperator.enabled` on the tenant cluster, plus GPUs declared on its worker node pool | [Managed Kubernetes](/docs/next/kubernetes/) | +| Containers inside a tenant Kubernetes cluster, several per card | Tenant cluster | Memory and compute quota per pod | The above, plus `addons.hami.enabled` | [GPU Sharing with HAMi](/docs/next/kubernetes/gpu-sharing/) | + +Everything on this page is NVIDIA. The `cozystack.gpu-operator` package wraps NVIDIA's GPU Operator, and the KubeVirt host-device defaults the platform ships cover PCI vendor `10DE`. + +## A management cluster node serves one variant, and only one + +The `cozystack.gpu-operator` package ships three variants, and they differ in exactly one thing that matters: who is allowed to own the GPU on the host. + +- **`default`** — the operator unbinds whatever holds the card and binds `vfio-pci`, so the GPU can be handed to a virtual machine. The host must not carry an NVIDIA driver of its own. +- **`vgpu`** — the proprietary NVIDIA vGPU host driver owns the card and slices it into SR-IOV virtual functions, each of which becomes a GPU to one VM. +- **`container`** — the host owns the card through its own distro-installed driver, and the operator only advertises it to kubelet. The operator's driver, container-toolkit and VFIO components are pinned off precisely so that it does not fight the host install. + +These are not degrees of the same setting. Two of them on one node is a broken node: the passthrough variant refuses to bind `vfio-pci` when it detects a pre-installed host driver, and the container variant has nothing to advertise when the driver has been unbound. That is why the variant is the first decision, before any workload exists — and why moving a node from one to the other is a host-level migration, not a values edit. If a node has ended up in the crossed state, [GPU Operator: host driver](/docs/next/operations/troubleshooting/gpu-operator-host-driver/) describes the symptom and the way out. + +For the two VM-facing variants the platform also mirrors the choice into the KubeVirt custom resource — the `HostDevices` feature gate and a starter `permittedHostDevices` table — so there is no `kubectl edit kubevirt` step, and a hand edit to the live resource is reverted on the next reconcile. Extend that table through platform values instead; see [GPU Passthrough](/docs/next/virtualization/gpu/). The `container` variant serves pods and gets none of this wiring, by design: the host driver stays bound and no GPU reaches a VM. + +## Tenant clusters stack on top of passthrough + +A tenant Kubernetes cluster runs its own GPU Operator instance, enabled as a cluster addon and configured independently of the management cluster. Its worker nodes, however, are virtual machines — so the GPU has to reach them as a KubeVirt host device first. In practice this means the management cluster must be on a VM-facing variant for a tenant node to see a GPU at all, and the resource name a tenant node pool asks for is the one the management cluster's sandbox device plugin advertises, such as `nvidia.com/GA102GL_A10`. Confirm the exact string with `kubectl describe node | grep nvidia.com/` rather than guessing it from the model name; the slug is derived mechanically from the PCI IDs database and carries every token that string holds. + +Inside the tenant cluster the GPU is then served to pods — whole by default, or split by memory and compute quota with [HAMi](/docs/next/kubernetes/gpu-sharing/). + +## What the paths do not cover + +**MIG is a container mechanism only.** MIG partitions a card into isolated instances, but they are logical divisions inside one PCIe device and VFIO cannot pass them to a virtual machine; reaching MIG from a VM requires vGPU layered on top, which is licensed. None of the three variants configures MIG geometry. + +**vGPU is not free and not open.** It needs an NVIDIA vGPU Software or AI Enterprise subscription and a reachable Delegated License Service endpoint. The driver is not redistributable, which is also why Talos Linux is not a recommended host for it — Sidero cannot ship the guest driver in a system extension under NVIDIA's current terms. + +**HAMi's isolation depends on the workload image, not on the cluster.** HAMi-core loads through `LD_PRELOAD` into the workload container and relies on a glibc symbol removed in 2.34, so on a modern base image compute isolation degrades or disappears — silently, with the workload still running and no limit enforced. The compatibility table is on the [HAMi page](/docs/next/kubernetes/gpu-sharing/); read it before sizing a shared node. + +**HAMi does not stack on the `container` variant on the management cluster.** That variant pins the NVIDIA device plugin on and HAMi ships its own, so both would register `nvidia.com/gpu`. Fractional sharing is currently a tenant-cluster capability, where the tenant chart disables the operator's plugin for you. + +## Where each page lives + +- [Containerized GPU Workloads](/docs/next/operations/gpu-container-workloads/) — CUDA pods on management cluster nodes, and the host prerequisites the `container` variant assumes. +- [GPU Passthrough](/docs/next/virtualization/gpu/) — a whole GPU inside a VM, the automatic KubeVirt wiring, and how to extend the device table for a card the defaults do not list. +- [NVIDIA vGPU](/docs/next/virtualization/vgpu/) — slicing one card across several VMs, SR-IOV profile assignment and DLS licensing. +- [GPU Sharing with HAMi](/docs/next/kubernetes/gpu-sharing/) — fractional GPU for pods in a tenant cluster, the resource names, and the isolation limits. +- [GPU Operator: host driver](/docs/next/operations/troubleshooting/gpu-operator-host-driver/) — recovering a node where the host driver and the passthrough variant are both trying to own the card. diff --git a/content/en/docs/v1.6/guides/gpu.md b/content/en/docs/v1.6/guides/gpu.md new file mode 100644 index 00000000..f5514ce5 --- /dev/null +++ b/content/en/docs/v1.6/guides/gpu.md @@ -0,0 +1,56 @@ +--- +title: "GPU in Cozystack" +linkTitle: "GPU" +description: "How Cozystack serves GPUs to pods, virtual machines and tenant Kubernetes clusters: which path to pick, what each one requires, and where the boundaries are." +weight: 35 +--- + +Cozystack has no single "GPU feature". It has several paths that serve a GPU to a different kind of consumer, and they are not interchangeable — the choice is made once per node, at the level of who owns the host driver, and it decides what that node can serve. Each path is documented in full on its own page. This one is the map: how to pick a path, what it costs, and what it rules out. + +## Pick a path + +| The GPU is consumed by | Runs on | Sharing granularity | Selected through | Guide | +| --- | --- | --- | --- | --- | +| Containers — CUDA pods, training, inference | Management cluster | Whole GPU | `cozystack.gpu-operator`, `container` variant | [Containerized GPU Workloads](/docs/v1.6/operations/gpu-container-workloads/) | +| One virtual machine | Management cluster | Whole GPU | `bundles.iaas.gpuOperatorVariant: default` | [GPU Passthrough](/docs/v1.6/virtualization/gpu/) | +| Several virtual machines sharing one card | Management cluster | vGPU profile per VF | `bundles.iaas.gpuOperatorVariant: vgpu` | [NVIDIA vGPU](/docs/v1.6/virtualization/vgpu/) | +| Containers inside a tenant Kubernetes cluster | Tenant cluster | Whole GPU | `addons.gpuOperator.enabled` on the tenant cluster, plus GPUs declared on its worker node pool | [Managed Kubernetes](/docs/v1.6/kubernetes/) | +| Containers inside a tenant Kubernetes cluster, several per card | Tenant cluster | Memory and compute quota per pod | The above, plus `addons.hami.enabled` | [GPU Sharing with HAMi](/docs/v1.6/kubernetes/gpu-sharing/) | + +Everything on this page is NVIDIA. The `cozystack.gpu-operator` package wraps NVIDIA's GPU Operator, and the KubeVirt host-device defaults the platform ships cover PCI vendor `10DE`. + +## A management cluster node serves one variant, and only one + +The `cozystack.gpu-operator` package ships three variants, and they differ in exactly one thing that matters: who is allowed to own the GPU on the host. + +- **`default`** — the operator unbinds whatever holds the card and binds `vfio-pci`, so the GPU can be handed to a virtual machine. The host must not carry an NVIDIA driver of its own. +- **`vgpu`** — the proprietary NVIDIA vGPU host driver owns the card and slices it into SR-IOV virtual functions, each of which becomes a GPU to one VM. +- **`container`** — the host owns the card through its own distro-installed driver, and the operator only advertises it to kubelet. The operator's driver, container-toolkit and VFIO components are pinned off precisely so that it does not fight the host install. + +These are not degrees of the same setting. Two of them on one node is a broken node: the passthrough variant refuses to bind `vfio-pci` when it detects a pre-installed host driver, and the container variant has nothing to advertise when the driver has been unbound. That is why the variant is the first decision, before any workload exists — and why moving a node from one to the other is a host-level migration, not a values edit. If a node has ended up in the crossed state, [GPU Operator: host driver](/docs/v1.6/operations/troubleshooting/gpu-operator-host-driver/) describes the symptom and the way out. + +For the two VM-facing variants the platform also mirrors the choice into the KubeVirt custom resource — the `HostDevices` feature gate and a starter `permittedHostDevices` table — so there is no `kubectl edit kubevirt` step, and a hand edit to the live resource is reverted on the next reconcile. Extend that table through platform values instead; see [GPU Passthrough](/docs/v1.6/virtualization/gpu/). The `container` variant serves pods and gets none of this wiring, by design: the host driver stays bound and no GPU reaches a VM. + +## Tenant clusters stack on top of passthrough + +A tenant Kubernetes cluster runs its own GPU Operator instance, enabled as a cluster addon and configured independently of the management cluster. Its worker nodes, however, are virtual machines — so the GPU has to reach them as a KubeVirt host device first. In practice this means the management cluster must be on a VM-facing variant for a tenant node to see a GPU at all, and the resource name a tenant node pool asks for is the one the management cluster's sandbox device plugin advertises, such as `nvidia.com/GA102GL_A10`. Confirm the exact string with `kubectl describe node | grep nvidia.com/` rather than guessing it from the model name; the slug is derived mechanically from the PCI IDs database and carries every token that string holds. + +Inside the tenant cluster the GPU is then served to pods — whole by default, or split by memory and compute quota with [HAMi](/docs/v1.6/kubernetes/gpu-sharing/). + +## What the paths do not cover + +**MIG is a container mechanism only.** MIG partitions a card into isolated instances, but they are logical divisions inside one PCIe device and VFIO cannot pass them to a virtual machine; reaching MIG from a VM requires vGPU layered on top, which is licensed. None of the three variants configures MIG geometry. + +**vGPU is not free and not open.** It needs an NVIDIA vGPU Software or AI Enterprise subscription and a reachable Delegated License Service endpoint. The driver is not redistributable, which is also why Talos Linux is not a recommended host for it — Sidero cannot ship the guest driver in a system extension under NVIDIA's current terms. + +**HAMi's isolation depends on the workload image, not on the cluster.** HAMi-core loads through `LD_PRELOAD` into the workload container and relies on a glibc symbol removed in 2.34, so on a modern base image compute isolation degrades or disappears — silently, with the workload still running and no limit enforced. The compatibility table is on the [HAMi page](/docs/v1.6/kubernetes/gpu-sharing/); read it before sizing a shared node. + +**HAMi does not stack on the `container` variant on the management cluster.** That variant pins the NVIDIA device plugin on and HAMi ships its own, so both would register `nvidia.com/gpu`. Fractional sharing is currently a tenant-cluster capability, where the tenant chart disables the operator's plugin for you. + +## Where each page lives + +- [Containerized GPU Workloads](/docs/v1.6/operations/gpu-container-workloads/) — CUDA pods on management cluster nodes, and the host prerequisites the `container` variant assumes. +- [GPU Passthrough](/docs/v1.6/virtualization/gpu/) — a whole GPU inside a VM, the automatic KubeVirt wiring, and how to extend the device table for a card the defaults do not list. +- [NVIDIA vGPU](/docs/v1.6/virtualization/vgpu/) — slicing one card across several VMs, SR-IOV profile assignment and DLS licensing. +- [GPU Sharing with HAMi](/docs/v1.6/kubernetes/gpu-sharing/) — fractional GPU for pods in a tenant cluster, the resource names, and the isolation limits. +- [GPU Operator: host driver](/docs/v1.6/operations/troubleshooting/gpu-operator-host-driver/) — recovering a node where the host driver and the passthrough variant are both trying to own the card. From 300c8984faa2e690547288c44ddac132ce98e465 Mon Sep 17 00:00:00 2001 From: Andrei Kvapil Date: Mon, 21 Sep 2026 16:49:04 +0100 Subject: [PATCH 2/2] docs(gpu): align the overview with current main and AI Conformance The KubeVirt paragraph no longer promises a starter device table for every VM-facing variant: on main the vgpu variant gets the feature gate and no table at stock values, so the page now says the platform owns the list and points at the guide for what it carries. The MIG item now says only what holds everywhere, that a MIG instance cannot be passed to a VM, so it does not read as contradicting the AI Conformance page on MIG inside tenant clusters. That page is now linked from the tenant section, since it is where DRA, Kueue and DCGM on tenant clusters are recorded. Assisted-by: LLM Signed-off-by: Andrei Kvapil --- content/en/docs/next/guides/gpu.md | 43 +++++++++++++++--------------- content/en/docs/v1.6/guides/gpu.md | 43 +++++++++++++++--------------- 2 files changed, 44 insertions(+), 42 deletions(-) diff --git a/content/en/docs/next/guides/gpu.md b/content/en/docs/next/guides/gpu.md index ce446428..f32237c5 100644 --- a/content/en/docs/next/guides/gpu.md +++ b/content/en/docs/next/guides/gpu.md @@ -5,52 +5,53 @@ description: "How Cozystack serves GPUs to pods, virtual machines and tenant Kub weight: 35 --- -Cozystack has no single "GPU feature". It has several paths that serve a GPU to a different kind of consumer, and they are not interchangeable — the choice is made once per node, at the level of who owns the host driver, and it decides what that node can serve. Each path is documented in full on its own page. This one is the map: how to pick a path, what it costs, and what it rules out. +Cozystack can serve a GPU to a container, to a virtual machine, or to a workload inside a tenant Kubernetes cluster. Each of these is a separate path with its own guide, and they are not interchangeable: the choice is made once per node, by deciding who owns the card on the host, and it fixes what that node can serve. This page is the map: which path fits, what it costs, and what it rules out. ## Pick a path | The GPU is consumed by | Runs on | Sharing granularity | Selected through | Guide | | --- | --- | --- | --- | --- | -| Containers — CUDA pods, training, inference | Management cluster | Whole GPU | `cozystack.gpu-operator`, `container` variant | [Containerized GPU Workloads](/docs/next/operations/gpu-container-workloads/) | +| Containers: CUDA pods, training, inference | Management cluster | Whole GPU | `cozystack.gpu-operator`, `container` variant | [Containerized GPU Workloads](/docs/next/operations/gpu-container-workloads/) | | One virtual machine | Management cluster | Whole GPU | `bundles.iaas.gpuOperatorVariant: default` | [GPU Passthrough](/docs/next/virtualization/gpu/) | -| Several virtual machines sharing one card | Management cluster | vGPU profile per VF | `bundles.iaas.gpuOperatorVariant: vgpu` | [NVIDIA vGPU](/docs/next/virtualization/vgpu/) | +| Several virtual machines sharing one card | Management cluster | vGPU profile | `bundles.iaas.gpuOperatorVariant: vgpu` | [NVIDIA vGPU](/docs/next/virtualization/vgpu/) | | Containers inside a tenant Kubernetes cluster | Tenant cluster | Whole GPU | `addons.gpuOperator.enabled` on the tenant cluster, plus GPUs declared on its worker node pool | [Managed Kubernetes](/docs/next/kubernetes/) | | Containers inside a tenant Kubernetes cluster, several per card | Tenant cluster | Memory and compute quota per pod | The above, plus `addons.hami.enabled` | [GPU Sharing with HAMi](/docs/next/kubernetes/gpu-sharing/) | Everything on this page is NVIDIA. The `cozystack.gpu-operator` package wraps NVIDIA's GPU Operator, and the KubeVirt host-device defaults the platform ships cover PCI vendor `10DE`. -## A management cluster node serves one variant, and only one +## A management cluster node serves one variant -The `cozystack.gpu-operator` package ships three variants, and they differ in exactly one thing that matters: who is allowed to own the GPU on the host. +The `cozystack.gpu-operator` package ships three variants. What separates them is who owns the GPU on the host. -- **`default`** — the operator unbinds whatever holds the card and binds `vfio-pci`, so the GPU can be handed to a virtual machine. The host must not carry an NVIDIA driver of its own. -- **`vgpu`** — the proprietary NVIDIA vGPU host driver owns the card and slices it into SR-IOV virtual functions, each of which becomes a GPU to one VM. -- **`container`** — the host owns the card through its own distro-installed driver, and the operator only advertises it to kubelet. The operator's driver, container-toolkit and VFIO components are pinned off precisely so that it does not fight the host install. +- **`default`**: the operator unbinds whatever holds the card and binds `vfio-pci`, so the GPU can be handed to a virtual machine. The host must not carry an NVIDIA driver of its own. +- **`vgpu`**: the proprietary NVIDIA vGPU host driver owns the card and slices it, and each slice becomes a GPU for one VM. +- **`container`**: the host owns the card through its own distro-installed driver, and the operator only advertises it to kubelet. The operator's driver, container toolkit and VFIO components are pinned off so that it does not fight the host install. -These are not degrees of the same setting. Two of them on one node is a broken node: the passthrough variant refuses to bind `vfio-pci` when it detects a pre-installed host driver, and the container variant has nothing to advertise when the driver has been unbound. That is why the variant is the first decision, before any workload exists — and why moving a node from one to the other is a host-level migration, not a values edit. If a node has ended up in the crossed state, [GPU Operator: host driver](/docs/next/operations/troubleshooting/gpu-operator-host-driver/) describes the symptom and the way out. +Two of them on one node make a broken node. The passthrough variant refuses to bind `vfio-pci` when it finds a pre-installed host driver, and the container variant has nothing to advertise once the driver is unbound. So pick the variant before any workload exists, and treat moving a node from one to another as a host migration rather than a values edit. If a node already ended up in that state, [GPU Operator: host driver](/docs/next/operations/troubleshooting/gpu-operator-host-driver/) shows the symptom and the way back. -For the two VM-facing variants the platform also mirrors the choice into the KubeVirt custom resource — the `HostDevices` feature gate and a starter `permittedHostDevices` table — so there is no `kubectl edit kubevirt` step, and a hand edit to the live resource is reverted on the next reconcile. Extend that table through platform values instead; see [GPU Passthrough](/docs/next/virtualization/gpu/). The `container` variant serves pods and gets none of this wiring, by design: the host driver stays bound and no GPU reaches a VM. +For the two VM-facing variants the platform sets the `HostDevices` feature gate on the KubeVirt custom resource and owns its `permittedHostDevices` list. There is no `kubectl edit kubevirt` step, and a hand edit to the live resource is reverted on the next reconcile. Which devices the list carries by default depends on the variant; [GPU Passthrough](/docs/next/virtualization/gpu/) shows how to add a card through platform values. The `container` variant gets none of this: the host driver stays bound and no GPU reaches a VM. ## Tenant clusters stack on top of passthrough -A tenant Kubernetes cluster runs its own GPU Operator instance, enabled as a cluster addon and configured independently of the management cluster. Its worker nodes, however, are virtual machines — so the GPU has to reach them as a KubeVirt host device first. In practice this means the management cluster must be on a VM-facing variant for a tenant node to see a GPU at all, and the resource name a tenant node pool asks for is the one the management cluster's sandbox device plugin advertises, such as `nvidia.com/GA102GL_A10`. Confirm the exact string with `kubectl describe node | grep nvidia.com/` rather than guessing it from the model name; the slug is derived mechanically from the PCI IDs database and carries every token that string holds. +A tenant Kubernetes cluster runs its own GPU Operator, enabled as a cluster addon and configured separately from the management cluster. Its worker nodes are virtual machines, though, so the GPU has to reach them as a KubeVirt host device first. The management cluster therefore has to run a VM-facing variant before a tenant node can see a GPU at all, and a tenant node pool asks for the resource name that the management cluster advertises, such as `nvidia.com/GA102GL_A10`. Take the exact string from `kubectl describe node | grep nvidia.com/` rather than guessing it from the model name: it is derived from the PCI IDs database and keeps every token that entry holds. -Inside the tenant cluster the GPU is then served to pods — whole by default, or split by memory and compute quota with [HAMi](/docs/next/kubernetes/gpu-sharing/). +Inside the tenant cluster the GPU is then served to pods, whole by default, or split by memory and compute quota with [HAMi](/docs/next/kubernetes/gpu-sharing/). What else a tenant cluster offers AI workloads (Dynamic Resource Allocation, Kueue, DCGM metrics) is recorded on the [AI Conformance](/compliance/ai-conformance/) page, together with the command that checks each item. ## What the paths do not cover -**MIG is a container mechanism only.** MIG partitions a card into isolated instances, but they are logical divisions inside one PCIe device and VFIO cannot pass them to a virtual machine; reaching MIG from a VM requires vGPU layered on top, which is licensed. None of the three variants configures MIG geometry. +**A MIG instance cannot be handed to a virtual machine.** MIG partitions a card into isolated instances inside one PCIe device, and VFIO passes whole devices. A VM gets either the whole card or, with vGPU, a licensed slice of it. -**vGPU is not free and not open.** It needs an NVIDIA vGPU Software or AI Enterprise subscription and a reachable Delegated License Service endpoint. The driver is not redistributable, which is also why Talos Linux is not a recommended host for it — Sidero cannot ship the guest driver in a system extension under NVIDIA's current terms. +**vGPU needs an NVIDIA license.** It requires an NVIDIA vGPU Software or AI Enterprise subscription and a reachable Delegated License Service endpoint. The driver is not redistributable, which is also why Talos Linux is not a recommended host for it: Sidero cannot ship it in a system extension under NVIDIA's current terms. -**HAMi's isolation depends on the workload image, not on the cluster.** HAMi-core loads through `LD_PRELOAD` into the workload container and relies on a glibc symbol removed in 2.34, so on a modern base image compute isolation degrades or disappears — silently, with the workload still running and no limit enforced. The compatibility table is on the [HAMi page](/docs/next/kubernetes/gpu-sharing/); read it before sizing a shared node. +**HAMi isolation depends on the workload image.** HAMi-core is loaded into the workload container through `LD_PRELOAD` and relies on a glibc symbol removed in 2.34. On a recent base image compute isolation degrades or disappears, silently: the workload keeps running and no limit is enforced. Check the compatibility table on the [HAMi page](/docs/next/kubernetes/gpu-sharing/) before sizing a shared node. -**HAMi does not stack on the `container` variant on the management cluster.** That variant pins the NVIDIA device plugin on and HAMi ships its own, so both would register `nvidia.com/gpu`. Fractional sharing is currently a tenant-cluster capability, where the tenant chart disables the operator's plugin for you. +**HAMi does not stack on the `container` variant on the management cluster.** That variant keeps the NVIDIA device plugin on and HAMi ships its own, so both would register `nvidia.com/gpu`. Fractional sharing is a tenant cluster capability for now, where the tenant chart turns the operator's plugin off for you. ## Where each page lives -- [Containerized GPU Workloads](/docs/next/operations/gpu-container-workloads/) — CUDA pods on management cluster nodes, and the host prerequisites the `container` variant assumes. -- [GPU Passthrough](/docs/next/virtualization/gpu/) — a whole GPU inside a VM, the automatic KubeVirt wiring, and how to extend the device table for a card the defaults do not list. -- [NVIDIA vGPU](/docs/next/virtualization/vgpu/) — slicing one card across several VMs, SR-IOV profile assignment and DLS licensing. -- [GPU Sharing with HAMi](/docs/next/kubernetes/gpu-sharing/) — fractional GPU for pods in a tenant cluster, the resource names, and the isolation limits. -- [GPU Operator: host driver](/docs/next/operations/troubleshooting/gpu-operator-host-driver/) — recovering a node where the host driver and the passthrough variant are both trying to own the card. +- [Containerized GPU Workloads](/docs/next/operations/gpu-container-workloads/): CUDA pods on management cluster nodes, and the host prerequisites the `container` variant assumes. +- [GPU Passthrough](/docs/next/virtualization/gpu/): a whole GPU inside a VM, the KubeVirt wiring, and adding a card the defaults do not list. +- [NVIDIA vGPU](/docs/next/virtualization/vgpu/): one card sliced across several VMs, profile assignment and DLS licensing. +- [GPU Sharing with HAMi](/docs/next/kubernetes/gpu-sharing/): fractional GPU for pods in a tenant cluster, the resource names, and the isolation limits. +- [GPU Operator: host driver](/docs/next/operations/troubleshooting/gpu-operator-host-driver/): recovering a node where the host driver and the passthrough variant both claim the card. +- [AI Conformance](/compliance/ai-conformance/): what a tenant cluster certifies for AI workloads, and how to verify it. diff --git a/content/en/docs/v1.6/guides/gpu.md b/content/en/docs/v1.6/guides/gpu.md index f5514ce5..597664c4 100644 --- a/content/en/docs/v1.6/guides/gpu.md +++ b/content/en/docs/v1.6/guides/gpu.md @@ -5,52 +5,53 @@ description: "How Cozystack serves GPUs to pods, virtual machines and tenant Kub weight: 35 --- -Cozystack has no single "GPU feature". It has several paths that serve a GPU to a different kind of consumer, and they are not interchangeable — the choice is made once per node, at the level of who owns the host driver, and it decides what that node can serve. Each path is documented in full on its own page. This one is the map: how to pick a path, what it costs, and what it rules out. +Cozystack can serve a GPU to a container, to a virtual machine, or to a workload inside a tenant Kubernetes cluster. Each of these is a separate path with its own guide, and they are not interchangeable: the choice is made once per node, by deciding who owns the card on the host, and it fixes what that node can serve. This page is the map: which path fits, what it costs, and what it rules out. ## Pick a path | The GPU is consumed by | Runs on | Sharing granularity | Selected through | Guide | | --- | --- | --- | --- | --- | -| Containers — CUDA pods, training, inference | Management cluster | Whole GPU | `cozystack.gpu-operator`, `container` variant | [Containerized GPU Workloads](/docs/v1.6/operations/gpu-container-workloads/) | +| Containers: CUDA pods, training, inference | Management cluster | Whole GPU | `cozystack.gpu-operator`, `container` variant | [Containerized GPU Workloads](/docs/v1.6/operations/gpu-container-workloads/) | | One virtual machine | Management cluster | Whole GPU | `bundles.iaas.gpuOperatorVariant: default` | [GPU Passthrough](/docs/v1.6/virtualization/gpu/) | -| Several virtual machines sharing one card | Management cluster | vGPU profile per VF | `bundles.iaas.gpuOperatorVariant: vgpu` | [NVIDIA vGPU](/docs/v1.6/virtualization/vgpu/) | +| Several virtual machines sharing one card | Management cluster | vGPU profile | `bundles.iaas.gpuOperatorVariant: vgpu` | [NVIDIA vGPU](/docs/v1.6/virtualization/vgpu/) | | Containers inside a tenant Kubernetes cluster | Tenant cluster | Whole GPU | `addons.gpuOperator.enabled` on the tenant cluster, plus GPUs declared on its worker node pool | [Managed Kubernetes](/docs/v1.6/kubernetes/) | | Containers inside a tenant Kubernetes cluster, several per card | Tenant cluster | Memory and compute quota per pod | The above, plus `addons.hami.enabled` | [GPU Sharing with HAMi](/docs/v1.6/kubernetes/gpu-sharing/) | Everything on this page is NVIDIA. The `cozystack.gpu-operator` package wraps NVIDIA's GPU Operator, and the KubeVirt host-device defaults the platform ships cover PCI vendor `10DE`. -## A management cluster node serves one variant, and only one +## A management cluster node serves one variant -The `cozystack.gpu-operator` package ships three variants, and they differ in exactly one thing that matters: who is allowed to own the GPU on the host. +The `cozystack.gpu-operator` package ships three variants. What separates them is who owns the GPU on the host. -- **`default`** — the operator unbinds whatever holds the card and binds `vfio-pci`, so the GPU can be handed to a virtual machine. The host must not carry an NVIDIA driver of its own. -- **`vgpu`** — the proprietary NVIDIA vGPU host driver owns the card and slices it into SR-IOV virtual functions, each of which becomes a GPU to one VM. -- **`container`** — the host owns the card through its own distro-installed driver, and the operator only advertises it to kubelet. The operator's driver, container-toolkit and VFIO components are pinned off precisely so that it does not fight the host install. +- **`default`**: the operator unbinds whatever holds the card and binds `vfio-pci`, so the GPU can be handed to a virtual machine. The host must not carry an NVIDIA driver of its own. +- **`vgpu`**: the proprietary NVIDIA vGPU host driver owns the card and slices it, and each slice becomes a GPU for one VM. +- **`container`**: the host owns the card through its own distro-installed driver, and the operator only advertises it to kubelet. The operator's driver, container toolkit and VFIO components are pinned off so that it does not fight the host install. -These are not degrees of the same setting. Two of them on one node is a broken node: the passthrough variant refuses to bind `vfio-pci` when it detects a pre-installed host driver, and the container variant has nothing to advertise when the driver has been unbound. That is why the variant is the first decision, before any workload exists — and why moving a node from one to the other is a host-level migration, not a values edit. If a node has ended up in the crossed state, [GPU Operator: host driver](/docs/v1.6/operations/troubleshooting/gpu-operator-host-driver/) describes the symptom and the way out. +Two of them on one node make a broken node. The passthrough variant refuses to bind `vfio-pci` when it finds a pre-installed host driver, and the container variant has nothing to advertise once the driver is unbound. So pick the variant before any workload exists, and treat moving a node from one to another as a host migration rather than a values edit. If a node already ended up in that state, [GPU Operator: host driver](/docs/v1.6/operations/troubleshooting/gpu-operator-host-driver/) shows the symptom and the way back. -For the two VM-facing variants the platform also mirrors the choice into the KubeVirt custom resource — the `HostDevices` feature gate and a starter `permittedHostDevices` table — so there is no `kubectl edit kubevirt` step, and a hand edit to the live resource is reverted on the next reconcile. Extend that table through platform values instead; see [GPU Passthrough](/docs/v1.6/virtualization/gpu/). The `container` variant serves pods and gets none of this wiring, by design: the host driver stays bound and no GPU reaches a VM. +For the two VM-facing variants the platform sets the `HostDevices` feature gate on the KubeVirt custom resource and owns its `permittedHostDevices` list. There is no `kubectl edit kubevirt` step, and a hand edit to the live resource is reverted on the next reconcile. Which devices the list carries by default depends on the variant; [GPU Passthrough](/docs/v1.6/virtualization/gpu/) shows how to add a card through platform values. The `container` variant gets none of this: the host driver stays bound and no GPU reaches a VM. ## Tenant clusters stack on top of passthrough -A tenant Kubernetes cluster runs its own GPU Operator instance, enabled as a cluster addon and configured independently of the management cluster. Its worker nodes, however, are virtual machines — so the GPU has to reach them as a KubeVirt host device first. In practice this means the management cluster must be on a VM-facing variant for a tenant node to see a GPU at all, and the resource name a tenant node pool asks for is the one the management cluster's sandbox device plugin advertises, such as `nvidia.com/GA102GL_A10`. Confirm the exact string with `kubectl describe node | grep nvidia.com/` rather than guessing it from the model name; the slug is derived mechanically from the PCI IDs database and carries every token that string holds. +A tenant Kubernetes cluster runs its own GPU Operator, enabled as a cluster addon and configured separately from the management cluster. Its worker nodes are virtual machines, though, so the GPU has to reach them as a KubeVirt host device first. The management cluster therefore has to run a VM-facing variant before a tenant node can see a GPU at all, and a tenant node pool asks for the resource name that the management cluster advertises, such as `nvidia.com/GA102GL_A10`. Take the exact string from `kubectl describe node | grep nvidia.com/` rather than guessing it from the model name: it is derived from the PCI IDs database and keeps every token that entry holds. -Inside the tenant cluster the GPU is then served to pods — whole by default, or split by memory and compute quota with [HAMi](/docs/v1.6/kubernetes/gpu-sharing/). +Inside the tenant cluster the GPU is then served to pods, whole by default, or split by memory and compute quota with [HAMi](/docs/v1.6/kubernetes/gpu-sharing/). What else a tenant cluster offers AI workloads (Dynamic Resource Allocation, Kueue, DCGM metrics) is recorded on the [AI Conformance](/compliance/ai-conformance/) page, together with the command that checks each item. ## What the paths do not cover -**MIG is a container mechanism only.** MIG partitions a card into isolated instances, but they are logical divisions inside one PCIe device and VFIO cannot pass them to a virtual machine; reaching MIG from a VM requires vGPU layered on top, which is licensed. None of the three variants configures MIG geometry. +**A MIG instance cannot be handed to a virtual machine.** MIG partitions a card into isolated instances inside one PCIe device, and VFIO passes whole devices. A VM gets either the whole card or, with vGPU, a licensed slice of it. -**vGPU is not free and not open.** It needs an NVIDIA vGPU Software or AI Enterprise subscription and a reachable Delegated License Service endpoint. The driver is not redistributable, which is also why Talos Linux is not a recommended host for it — Sidero cannot ship the guest driver in a system extension under NVIDIA's current terms. +**vGPU needs an NVIDIA license.** It requires an NVIDIA vGPU Software or AI Enterprise subscription and a reachable Delegated License Service endpoint. The driver is not redistributable, which is also why Talos Linux is not a recommended host for it: Sidero cannot ship it in a system extension under NVIDIA's current terms. -**HAMi's isolation depends on the workload image, not on the cluster.** HAMi-core loads through `LD_PRELOAD` into the workload container and relies on a glibc symbol removed in 2.34, so on a modern base image compute isolation degrades or disappears — silently, with the workload still running and no limit enforced. The compatibility table is on the [HAMi page](/docs/v1.6/kubernetes/gpu-sharing/); read it before sizing a shared node. +**HAMi isolation depends on the workload image.** HAMi-core is loaded into the workload container through `LD_PRELOAD` and relies on a glibc symbol removed in 2.34. On a recent base image compute isolation degrades or disappears, silently: the workload keeps running and no limit is enforced. Check the compatibility table on the [HAMi page](/docs/v1.6/kubernetes/gpu-sharing/) before sizing a shared node. -**HAMi does not stack on the `container` variant on the management cluster.** That variant pins the NVIDIA device plugin on and HAMi ships its own, so both would register `nvidia.com/gpu`. Fractional sharing is currently a tenant-cluster capability, where the tenant chart disables the operator's plugin for you. +**HAMi does not stack on the `container` variant on the management cluster.** That variant keeps the NVIDIA device plugin on and HAMi ships its own, so both would register `nvidia.com/gpu`. Fractional sharing is a tenant cluster capability for now, where the tenant chart turns the operator's plugin off for you. ## Where each page lives -- [Containerized GPU Workloads](/docs/v1.6/operations/gpu-container-workloads/) — CUDA pods on management cluster nodes, and the host prerequisites the `container` variant assumes. -- [GPU Passthrough](/docs/v1.6/virtualization/gpu/) — a whole GPU inside a VM, the automatic KubeVirt wiring, and how to extend the device table for a card the defaults do not list. -- [NVIDIA vGPU](/docs/v1.6/virtualization/vgpu/) — slicing one card across several VMs, SR-IOV profile assignment and DLS licensing. -- [GPU Sharing with HAMi](/docs/v1.6/kubernetes/gpu-sharing/) — fractional GPU for pods in a tenant cluster, the resource names, and the isolation limits. -- [GPU Operator: host driver](/docs/v1.6/operations/troubleshooting/gpu-operator-host-driver/) — recovering a node where the host driver and the passthrough variant are both trying to own the card. +- [Containerized GPU Workloads](/docs/v1.6/operations/gpu-container-workloads/): CUDA pods on management cluster nodes, and the host prerequisites the `container` variant assumes. +- [GPU Passthrough](/docs/v1.6/virtualization/gpu/): a whole GPU inside a VM, the KubeVirt wiring, and adding a card the defaults do not list. +- [NVIDIA vGPU](/docs/v1.6/virtualization/vgpu/): one card sliced across several VMs, profile assignment and DLS licensing. +- [GPU Sharing with HAMi](/docs/v1.6/kubernetes/gpu-sharing/): fractional GPU for pods in a tenant cluster, the resource names, and the isolation limits. +- [GPU Operator: host driver](/docs/v1.6/operations/troubleshooting/gpu-operator-host-driver/): recovering a node where the host driver and the passthrough variant both claim the card. +- [AI Conformance](/compliance/ai-conformance/): what a tenant cluster certifies for AI workloads, and how to verify it.