Skip to main content
Version: v1.2.x

Multi-Instance GPU

Multi-Instance GPU (MIG) is an NVIDIA feature that partitions a supported GPU into as many as seven isolated GPU instances. Each instance has dedicated compute, memory, and memory bandwidth, so several workloads can share one physical GPU without competing for the same resources. MIG helps you increase GPU utilization when individual workloads do not need a full GPU.

In PaletteAI, MIG is configured per Compute Pool and governed at the System, Tenant, and Project level. The NVIDIA GPU Operator performs the actual partitioning on each workload cluster, and PaletteAI drives the operator by labeling the GPU nodes. This page explains the MIG concepts that PaletteAI exposes and how MIG fits into the Compute Pool lifecycle.

info

MIG requires the NVIDIA GPU Operator on the workload cluster. Deploy it through a GPU-optimized Profile Bundle. Only newer NVIDIA data center GPUs support MIG.

Hardware Support for MIG​

Only newer NVIDIA data center GPUs support MIG. Whether a given GPU is MIG-capable, and which partitions it accepts, depends on the GPU model and its driver version. PaletteAI determines MIG capability from a hardware lookup table that records, for each supported GPU, whether MIG is available and which MIG profiles and configurations the GPU accepts.

PaletteAI uses this table to keep the user interface honest about what the fleet can do.

  • When a Compute Pool's GPUs support MIG, PaletteAI presents the MIG configurations that are valid for that GPU model.
  • When the GPUs do not support MIG, PaletteAI blocks MIG enablement for that pool and states that MIG is unsupported.
  • When a fleet mixes MIG-capable and non-capable GPUs, PaletteAI restricts the available options to the configurations that the underlying hardware accepts.

MIG Profiles​

A MIG profile describes the shape of a single GPU instance. Profiles use the following notation.

<gpu-slices>g.<memory-size>gb

For example, the 1g.10gb profile represents one GPU compute slice with approximately 10 GB of GPU memory. The 2g.20gb profile represents a larger isolated partition with more compute slices, more memory, and additional bandwidth and cache resources. A MIG-capable GPU exposes a fixed number of compute slices, and each profile consumes a portion of them.

Each GPU model supports a different set of profiles. PaletteAI sources the valid profiles for each GPU from the hardware lookup table.

MIG Configurations​

A MIG configuration, also called a MIG config, is a named preset that describes how the GPU operator partitions every GPU on a node. PaletteAI applies a configuration to a Compute Pool's worker pool, and the GPU operator partitions the matching GPUs to match.

The following are examples of MIG configuration names.

ConfigurationResult
all-disabledMIG is disabled on every GPU on the node. This is the default state.
all-1g.10gbEvery GPU is partitioned into seven 1g.10gb instances.
all-2g.20gbEvery GPU is partitioned into 2g.20gb instances.

The configurations that PaletteAI offers for a worker pool are filtered by the GPU model in that pool, so you can only select a configuration that the hardware accepts.

MIG Strategies​

The MIG strategy defines how the GPU operator advertises MIG instances to Kubernetes.

  • single - Every MIG instance on a node uses the same profile, and the device plugin advertises the instances as generic nvidia.com/gpu resources. Kubernetes sees uniform resources, which keeps scheduling straightforward. PaletteAI configures MIG using the single strategy.
  • mixed - Different profiles coexist on the same GPU, and the device plugin advertises them as typed resources, such as nvidia.com/mig-1g.10gb. Pods request a specific instance type. The mixed strategy supports heterogeneous partitioning, such as small instances for inference alongside larger instances for training on the same GPU.

PaletteAI reports the strategy that each workload cluster runs so that you can confirm how its GPUs are advertised.

MIG Reconfiguration​

When PaletteAI changes the MIG configuration on a node, the GPU operator reconfigures the GPUs to match. Reconfiguration is disruptive: the operator stops the GPU pods on the node, applies the new partitions, and, on some hardware, reboots the node. Because the amount of schedulable GPU capacity changes during this process, PaletteAI pauses autoscaling for the affected Compute Pool until the rollout settles.

A rollout settles when every targeted node reaches a terminal state, whether the reconfiguration succeeds or fails. PaletteAI then resumes autoscaling. A node that fails to reconfigure does not hold the pause open indefinitely; instead, PaletteAI surfaces the failure so that you can investigate it. Refer to Autoscaling for how scaling behaves the rest of the time.

PaletteAI reports reconfiguration progress per worker pool. Each node moves through the pending, rebooting, success, and failed states, and PaletteAI aggregates these counts into the Compute Pool's hardware capacity so that you can track the rollout. Refer to MIG Hardware Capacity for the status.hardwareCapacity fields that report MIG state and layout.

Day 1 and Day 2 MIG​

PaletteAI supports MIG both when you create a Compute Pool and after the pool is running.

  • Day 1 - You set a MIG configuration on a worker pool while you create the Compute Pool. PaletteAI provisions the worker nodes with the configuration already applied, so the GPUs are partitioned as the nodes join.
  • Day 2 - You change the MIG configuration on a running Compute Pool. PaletteAI updates the affected worker pools, pauses autoscaling, and lets the GPU operator reconfigure the nodes in place.

The first Day 2 MIG change on a Compute Pool moves that pool from Day 1 management to Day 2 management. After this transition, PaletteAI manages MIG entirely through the running pool's per-worker-pool configuration, and the transition is permanent for the life of the pool. Refer to Configure Multi-Instance GPU for both workflows.

System, Tenant, and Project Governance​

MIG availability in the user interface is governed from the top down so that platform and Tenant administrators control where MIG can be used. Each scope gates MIG through its resource's MIG UI settings.

  • A System administrator enables or disables MIG configuration across all Tenants and Projects from the System scope, through the System resource's MIG UI settings.
  • A Tenant administrator enables MIG for the Tenant and chooses which Projects may configure MIG. By default, every Project in the Tenant may configure MIG.
  • A Project administrator configures MIG within their Project when the higher scopes allow it.
  • Higher scopes are authoritative. A Project cannot override the Tenant or System MIG governance, and a Tenant cannot override the System MIG governance.

These settings control only where MIG configuration appears in the user interface. They do not change how the GPU operator applies MIG. To manage MIG governance, refer to Configure System Settings for the System scope, Configure Tenant Settings for the Tenant scope, and Configure Project Settings for the Project scope.

Resources​

Refer to the following articles to learn more about MIG in PaletteAI.