Skip to main content
Version: v1.4.x

Compute Pools

A Compute Pool is a group of shared Compute resources that PaletteAI turns into one or more Kubernetes clusters where your AI/ML applications and models run. In the hub-spoke architecture, each Compute Pool becomes a spoke cluster on which applications and models are deployed.

You can create and manage Compute Pools from the Compute Pools page in the PaletteAI console, available at both the tenant and project scope. Each Compute Pool detail page displays the pool's status, hardware capacity, clusters, and associated Profile Bundle. Advanced and GitOps users can also manage Compute Pools declaratively through the ComputePool custom resource. For the complete manifest schema, refer to the ComputePool Configuration reference.

A Compute Pool can be defined at System, Tenant, or Project scope. A System- or Tenant-scoped Compute Pool can be shared with lower scopes through its sharedWith field. For details, refer to Sharing Resources.

Types of Compute Pools​

There are two types of Compute Pools, allowing you to deploy your AI/ML applications and models on the infrastructure that best suits your needs.

ModeDescriptionUse Case
DedicatedA single cluster whose resources are not shared with other deployments or tenants- Isolated workloads
- Specific hardware needs
- Stringent compliance or security requirements
SharedOne or more clusters whose resources are shared with other deployments or tenants- Resource efficiency for workloads with similar needs
- Maximize hardware utilization to reduce costs
- Development or staging environments

Dedicated and Shared describe the pool's cluster topology and tenant isolation, not the number of workloads the pool runs. How many workloads run on a Dedicated pool depends on what the pool runs:

  • Apps and Models — one App or Model Deployment per Dedicated pool. A Shared pool hosts multiple App Deployments across its clusters.
  • AI Virtual Machines — many AI Virtual Machines on a single-cluster pool, whether Dedicated or Shared. The pool's remaining capacity bounds how many virtual machines fit, while the vmLimits of each machine pool cap what a single virtual machine may request. Refer to AI Virtual Machines.

What Compute Pools Run​

Every Compute Pool is created to run one kind of workload, and you choose this when you create the Compute Pool.

RunsDescription
Apps and ModelsClusters for App Deployments and Model Deployments. Workloads run as containers and share the pool's GPUs and CPUs.
AI Virtual MachinesClusters with VM orchestration, where users launch GPU virtual machines that they sign in to. These pools resolve to exactly one cluster, can be Dedicated or Shared, are built from a VMO Profile Bundle, and host multiple AI Virtual Machines.

The pool reports its kind in status.infraKind, which is container or vm. A Profile Bundle that does not carry the palette.ai/infra-kind label resolves to container.

A pool that runs AI Virtual Machines caps what a single virtual machine may request with per-VM limits on each machine pool, and its overview links to the VMO console for the cluster. The pool mode describes its topology and isolation only: a single-cluster pool hosts as many AI Virtual Machines as its remaining capacity allows. Refer to AI Virtual Machines.

Compute Pool Provisioning​

You can create shared or dedicated Compute Pools before deploying AI/ML applications. Provisioning Kubernetes clusters in advance reduces the number of steps involved when deploying applications and models by allowing data scientists to select an existing Compute Pool for their workload rather than create a new one.

You can create a Compute Pool from the Compute Pools page, or directly within the App Deployment and Model Deployment workflows. Operators can turn off specific combinations through feature flags in the Mural Helm chart. Refer to the Helm Configuration Reference for details.

warning

When deploying models that are mapped to application-type Profile Bundles, you cannot create dedicated or shared Compute Pools during the deployment workflow. You must deploy models to preexisting Compute Pools. To enable Compute Pool creation during model deployment, use fullstack-type Profile Bundles instead.

When you create a dedicated or shared Compute Pool, PaletteAI provisions the underlying Kubernetes infrastructure:

  1. PaletteAI validates the configuration against available Compute resources.
  2. PaletteAI requests cluster provisioning from Palette using the Profile Bundle's Cluster Profile.
  3. Palette provisions the Kubernetes cluster.
  4. PaletteAI retrieves the cluster's kubeconfig from Palette.
  5. PaletteAI registers the cluster as a spoke with the hub.
  6. PaletteAI installs the required controllers on the spoke.

You can follow the progress on the Compute Pools page. The pool's status badge changes from Provisioning to Running when the pool is ready to receive applications.

To learn how applications are deployed to Compute Pools, refer to our App Deployments guide. For an in-depth look at the complete deployment flow, from Compute Pools to workload provisioning, refer to our Hub and Spoke Model guide.

Single-Node Compute Pools​

You can deploy a Compute Pool as a single-node cluster where one Edge host serves as both the control plane and worker node. This is useful for edge locations or lab environments with a single machine.

warning

A single-node Compute Pool is not considered a High Availability (HA) setup. If the Edge host fails, the cluster and all workloads running on it become unavailable. Use single-node Compute Pools for development, experimentation, or non-critical deployments only.

For instructions, refer to Deploy a Single-Node Compute Pool. For the required configuration fields and behavioral differences, refer to Single-Node Clusters in the configuration reference.

Worker Pools​

Each Compute Pool contains one or more worker pools, which group machines with similar hardware requirements. For example, you can define a CPU-only worker pool for general workloads and a GPU worker pool for training or inference. You define worker pools in the Node config step of the Compute Pool creation workflow.

Worker pools also support advanced options, such as explicit pool names and Kubernetes taints that reserve nodes for specific workloads. Refer to Worker Pool Configuration for details.

Hardware Capacity and Allocation​

PaletteAI continuously tracks each Compute Pool's total hardware capacity and how much of it is allocated to running deployments. This information determines whether a Compute Pool can accept additional workloads and is used for GPU quota enforcement at the Project level.

The pool's Overview tab displays capacity and allocation, including GPU counts by variant. The Compute Pool Resources section of the tab presents those values in tables, scoped by the Clusters and Node Pools selectors:

  • The resource totals table reports the number of clusters or edge hosts in scope, followed by Total CPU, Total Memory, Total GPU, and Total GPU Memory. The count column is labeled Clusters when all clusters are in scope, and Edge Hosts when a single cluster is selected.
  • The node pool breakdown table lists each control plane and worker pool with its node count and its own CPU, Memory, GPU, and GPU Memory totals. The Cluster column appears only while all clusters are in scope.

A Compute Pool that reports a single cluster opens with that cluster already selected, so the tables are scoped to it, the Node Pools selector is available, and the Cluster column is hidden. For the complete list of status fields, including Multi-Instance GPU (MIG) capacity details, refer to the Status Reference.

AI Virtual Machine Resources​

For a Compute Pool that runs AI Virtual Machines, the Compute Pool Resources section replaces the container totals and node pool breakdown with an AI VM resource view. It reports the virtual machine count and the CPU, GPU, and memory capacity in use and free:

  • The resource totals table reports the cluster count, the virtual machine count, Total CPU, Total Memory, Total GPU, and Total GPU Memory.
  • The resource allocation table lists Available CPU, Available Memory, Available GPU, and Available GPU Memory as rows, each split into Total, In use, and Free. The in-use and free figures include each value's share of the total as a percentage, and a row reads - when the controller has not published capacity for that resource. Only the values a virtual machine declares for itself are counted, so a virtual machine that inherits CPU, memory, or GPU from its template is not counted toward In use. In use can therefore understate, and Free overstate, what the pool's virtual machines request.

The AI VMs Resource View offers a Chart or Table view:

  • The Chart view shows histograms of CPU cores, memory, GPUs, and GPU memory per virtual machine. Select a bucket to filter the other charts and the table to the virtual machines in that range. Select outside the charts, or press Escape, to clear the selection.
  • The Table view lists each virtual machine with its share of the pool's published capacity and its state, and supports filtering by state.

When the pool is shared or belongs to the System or Tenant scope, the counts are scoped to the resources available at your current scope. If PaletteAI cannot read the virtual machine list for the scope, the section reports that the list may be incomplete instead of showing a count of zero.

For the virtual machine resource and the states it reports, refer to the AIVirtualMachine reference.

Autoscaling​

Compute Pools support automatic scaling based on CPU and GPU utilization metrics. Autoscaling adjusts the number of machines in response to workload demand, minimizing idle resources and handling demand spikes. The scaling rules come from a Scaling Policy, a reusable resource you attach to one or more Compute Pools.

To enable autoscaling, turn on Auto-Scaling Policy in the Node config step of the Compute Pool creation workflow and select a Scaling Policy, or reference a ScalingPolicy resource in the manifest. Refer to Create and Manage Scaling Policies for step-by-step instructions and to Autoscaling Configuration for manifest details.

Resource Groups​

Resource groups let you restrict which machines a Compute Pool can use. This is useful when you need workloads to run on specific hardware, such as machines in a particular network zone or with high-performance storage.

Resource groups are defined by tags on your machines. An administrator assigns labels that begin with palette.ai.rg/ to Edge hosts in Palette, for example palette.ai.rg/network-pool: '1'. These tags cannot be assigned from the PaletteAI console; PaletteAI discovers them through the Compute inventory.

Once discovered, the resource groups appear in the Resource groups step of the Compute Pool creation workflow, where you choose which groups the pool draws machines from. Only machines whose tags match the selected resource groups are eligible to join the pool. For manifest details, refer to Resource Groups in the configuration reference.

Network Isolation​

A dedicated Compute Pool can separate its network traffic from other tenants' traffic on shared AI edge infrastructure. When you enable network isolation on a pool, PaletteAI applies per-tenant GPU and ingress isolation through an external network fabric provider, and each plane is isolated only when the fabric advertises the matching capability — otherwise the affected plane falls back to the shared path. Set the provider and isolation unit through the pool's networkIsolation field, or require isolation for a scope from the Network Isolation page. For the planes, guarantees, and requirements, refer to Network Isolation.

Permissions​

Compute Pool operations are controlled by Role-Based Access Control (RBAC) permissions. Depending on your role, a read-only view may be displayed or certain actions may be disabled.

  • The spectrocloud.com/computepools:update permission controls your ability to edit Compute Pool settings and modify the associated Profile Bundle.
  • The spectrocloud.com/computepools:delete permission controls your ability to delete Compute Pools.

If an action or button described in the documentation is not displayed, your role does not include the required permission. Contact your administrator to request access.

Deletion Policy​

The spec.deletionPolicy field controls what happens to the underlying Palette cluster and its associated resources when you delete a Compute Pool. Refer to the Deletion Policy section of the ComputePool Configuration reference for details.

Next Steps​