Skip to main content
Version: v1.2.x

Compute Pools

A Compute Pool is a group of shared Compute resources that PaletteAI turns into one or more Kubernetes clusters where your AI/ML applications and models run. In the hub-spoke architecture, each Compute Pool becomes a spoke cluster on which applications and models are deployed.

You can create and manage Compute Pools from the Compute Pools page in the PaletteAI console, available at both the tenant and project scope. Each Compute Pool detail page displays the pool's status, hardware capacity, clusters, and associated Profile Bundle. Advanced and GitOps users can also manage Compute Pools declaratively through the ComputePool custom resource. For the complete manifest schema, refer to the ComputePool Configuration reference.

A Compute Pool can be defined at System, Tenant, or Project scope. A System- or Tenant-scoped Compute Pool can be shared with lower scopes through its sharedWith field. For details, refer to Sharing Resources.

Types of Compute Pools​

There are two types of Compute Pools, allowing you to deploy your AI/ML applications and models on the infrastructure that best suits your needs.

ModeDescriptionUse Case
DedicatedA single cluster used to host a single App Deployment- Isolated workloads
- Specific hardware needs
- Stringent compliance or security requirements
SharedOne or multiple clusters that share resources and are used to host multiple App Deployments- Resource efficiency for workloads with similar needs
- Maximize hardware utilization to reduce costs
- Development or staging environments

Compute Pool Provisioning​

You can create shared or dedicated Compute Pools before deploying AI/ML applications. Provisioning Kubernetes clusters in advance reduces the number of steps involved when deploying applications and models by allowing data scientists to select an existing Compute Pool for their workload rather than create a new one.

You can create a Compute Pool from the Compute Pools page, or directly within the App Deployment and Model Deployment workflows. Operators can turn off specific combinations through feature flags in the Mural Helm chart. Refer to the Helm Configuration Reference for details.

warning

When deploying models that are mapped to application-type Profile Bundles, you cannot create dedicated or shared Compute Pools during the deployment workflow. You must deploy models to preexisting Compute Pools. To enable Compute Pool creation during model deployment, use fullstack-type Profile Bundles instead.

When you create a dedicated or shared Compute Pool, PaletteAI provisions the underlying Kubernetes infrastructure:

  1. PaletteAI validates the configuration against available Compute resources.
  2. PaletteAI requests cluster provisioning from Palette using the Profile Bundle's Cluster Profile.
  3. Palette provisions the Kubernetes cluster.
  4. PaletteAI retrieves the cluster's kubeconfig from Palette.
  5. PaletteAI registers the cluster as a spoke with the hub.
  6. PaletteAI installs the required controllers on the spoke.

You can follow the progress on the Compute Pools page. The pool's status badge changes from Provisioning to Running when the pool is ready to receive applications.

To learn how applications are deployed to Compute Pools, refer to our App Deployments guide. For an in-depth look at the complete deployment flow, from Compute Pools to workload provisioning, refer to our Hub and Spoke Model guide.

Single-Node Compute Pools​

You can deploy a Compute Pool as a single-node cluster where one Edge host serves as both the control plane and worker node. This is useful for edge locations or lab environments with a single machine.

warning

A single-node Compute Pool is not considered a High Availability (HA) setup. If the Edge host fails, the cluster and all workloads running on it become unavailable. Use single-node Compute Pools for development, experimentation, or non-critical deployments only.

For instructions, refer to Deploy a Single-Node Compute Pool. For the required configuration fields and behavioral differences, refer to Single-Node Clusters in the configuration reference.

Worker Pools​

Each Compute Pool contains one or more worker pools, which group machines with similar hardware requirements. For example, you can define a CPU-only worker pool for general workloads and a GPU worker pool for training or inference. You define worker pools in the Node config step of the Compute Pool creation workflow.

Worker pools also support advanced options, such as explicit pool names and Kubernetes taints that reserve nodes for specific workloads. Refer to Worker Pool Configuration for details.

Hardware Capacity and Allocation​

PaletteAI continuously tracks each Compute Pool's total hardware capacity and how much of it is allocated to running deployments. This information determines whether a Compute Pool can accept additional workloads and is used for GPU quota enforcement at the Project level.

The pool's Overview tab displays capacity and allocation, including GPU counts by variant. For the complete list of status fields, including Multi-Instance GPU (MIG) capacity details, refer to the Status Reference.

Autoscaling​

Compute Pools support automatic scaling based on CPU and GPU utilization metrics. Autoscaling adjusts the number of machines in response to workload demand, minimizing idle resources and handling demand spikes. The scaling rules come from a Scaling Policy, a reusable resource you attach to one or more Compute Pools.

To enable autoscaling, turn on Auto-Scaling Policy in the Node config step of the Compute Pool creation workflow and select a Scaling Policy, or reference a ScalingPolicy resource in the manifest. Refer to Create and Manage Scaling Policies for step-by-step instructions and to Autoscaling Configuration for manifest details.

Resource Groups​

Resource groups let you restrict which machines a Compute Pool can use. This is useful when you need workloads to run on specific hardware, such as machines in a particular network zone or with high-performance storage.

Resource groups are defined by tags on your machines. An administrator assigns labels that begin with palette.ai.rg/ to Edge hosts in Palette, for example palette.ai.rg/network-pool: '1'. These tags cannot be assigned from the PaletteAI console; PaletteAI discovers them through the Compute inventory.

Once discovered, the resource groups appear in the Resource groups step of the Compute Pool creation workflow, where you choose which groups the pool draws machines from. Only machines whose tags match the selected resource groups are eligible to join the pool. For manifest details, refer to Resource Groups in the configuration reference.

Permissions​

Compute Pool operations are controlled by Role-Based Access Control (RBAC) permissions. Depending on your role, a read-only view may be displayed or certain actions may be disabled.

  • The spectrocloud.com/computepools:update permission controls your ability to edit Compute Pool settings and modify the associated Profile Bundle.
  • The spectrocloud.com/computepools:delete permission controls your ability to delete Compute Pools.

If an action or button described in the documentation is not displayed, your role does not include the required permission. Contact your administrator to request access.

Deletion Policy​

The spec.deletionPolicy field controls what happens to the underlying Palette cluster and its associated resources when you delete a Compute Pool. Refer to the Deletion Policy section of the ComputePool Configuration reference for details.

Next Steps​