ComputePool Configuration Reference

This page provides technical reference information for configuring ComputePools.

Complete ComputePool Example

The following example shows a complete, production-ready ComputePool manifest with all major fields. Use this as a template and refer to the detailed sections below for field-specific guidance.

apiVersion: spectrocloud.com/v1alpha1
kind: ComputePool
metadata:
  name: ml-cluster
  namespace: my-project
  labels:
    environment: production
    team: ml-platform
  annotations:
    description: 'Production ML compute cluster'
    owner: 'ml-platform-team'
spec:
  # Deletion policy - what happens when ComputePool is deleted
  deletionPolicy: delete

  # ProfileBundle reference - defines infrastructure and application stacks
  profileBundleRef:
    name: edge-profilebundle
    namespace: my-project
    cloudType: edge-native

  # Cluster variant configuration
  clusterVariant:
    # Optional: Resource groups filter which hosts are eligible for this cluster
    controlPlaneResourceGroups:
      'palette.ai': 'true'
    workerResourceGroups:
      'palette.ai': 'true'
      'gpu-enabled': 'true'

    # Dedicated cluster configuration
    dedicated:
      paletteClusterDeploymentConfig:
        # Cloud type for deployment
        cloudType: edge-native

        # Node pool requirements
        nodePoolRequirements:
          # Control plane configuration
          controlPlanePool:
            nodeCount: 3
            workerNodeEligible: false
            architecture: AMD64
            cpu:
              cpuCount: 4
            memory:
              memory: '16 GB'
            labels:
              node-role: control-plane
            annotations:
              description: 'Control plane node'

          # Worker pools
          workerPools:
            - name: cpu-pool
              architecture: AMD64
              cpu:
                cpuCount: 8
              memory:
                memory: '32 GB'
              labels:
                workload-type: cpu-intensive
              annotations:
                description: 'CPU worker pool'
            - name: gpu-pool
              architecture: AMD64
              cpu:
                cpuCount: 16
              memory:
                memory: '64 GB'
              gpu:
                variant: 'NVIDIA A100-SXM4-80GB | 80 GB'
                gpuCount: 2
                gpuMemory: '160 GB'
              labels:
                workload-type: gpu-training
              annotations:
                description: 'GPU worker pool for ML training'

        # Optional: SSH keys for cluster node access
        sshKeys:
          - 'ssh-rsa AAAAB3Nza... your-ssh-key'

        # Edge configuration (overrides ComputeConfig defaults)
        edge:
          # Required: Virtual IP for cluster control plane
          vip: '10.10.162.130'
          # Optional: NTP servers for time synchronization
          ntpServers:
            - time.google.com
            - time.cloudflare.com
          # Optional: Network overlay configuration
          networkOverlayConfig:
            enabled: false
            staticIp: false
            cidr: '192.168.1.0/24'
          # Optional: Two-node deployment mode
          isTwoNode: false

YAML Structure

All configuration shown in the sections below fits into the structure shown in this complete example. Pay special attention to the nesting hierarchy to avoid placement errors.

Key nesting paths:

• Control plane config: spec.clusterVariant.dedicated.paletteClusterDeploymentConfig.nodePoolRequirements.controlPlanePool
• Worker pools config: spec.clusterVariant.dedicated.paletteClusterDeploymentConfig.nodePoolRequirements.workerPools
• Edge config: spec.clusterVariant.dedicated.paletteClusterDeploymentConfig.edge
• Deletion policy: spec.deletionPolicy

Control Plane Configuration

Use nodePoolRequirements.controlPlanePool to configure the control plane node pool.

• Node count: 1, 3, or 5.

• Single-node clusters: To deploy a single-node cluster, set singleNodeCluster: true. When singleNodeCluster is true, you must also set nodeCount: 1 and workerNodeEligible: true, and include at least one worker pool in the configuration (the worker pool is used for edge host selection). Without these settings, single-node clusters cannot schedule workloads and will not become Ready.

• Worker eligibility: For multi-node clusters (3 or 5 nodes), you have two options:

  1. Set to false to keep the control plane dedicated and define one or more worker pools
  2. Set to true with an empty workerPools array to schedule workloads directly on control plane nodes (HA control-plane-only topology)

• Architecture: AMD64 or ARM64. Defaults to AMD64.

• Resource requirements: Set cpu.cpuCount and optional memory.memory (a string with units, for example '16 GB').

Example for a single-node cluster:

controlPlanePool:
  nodeCount: 1
  singleNodeCluster: true
  workerNodeEligible: true
  architecture: AMD64
  cpu:
    cpuCount: 4
  memory:
    memory: '16 GB'

workerPools:
  - name: default-pool
    architecture: AMD64
    cpu:
      cpuCount: 4
    memory:
      memory: '16 GB'

Example for a multi-node cluster:

controlPlanePool:
  nodeCount: 3
  workerNodeEligible: false
  architecture: AMD64
  cpu:
    cpuCount: 4
  memory:
    memory: '16 GB'

Single-Node Clusters

Single-node Compute Pools use a Compute Config with singleNodeCluster set to true, workerNodeEligible set to true, and nodeCount set to 1 in the control plane configuration. During initial provisioning, PaletteAI uses the worker pool requirements to select the Edge host and then promotes it to serve both the control plane and worker roles.

Single-node clusters have two behavioral differences from multi-node clusters:

• After initial provisioning, PaletteAI only manages the control plane machine pool. Worker pool requirements are used during initial cluster creation to select the Edge host, but day-2 machine pool operations (scale, replace, delete) are not performed.

• Worker pool host selection uses controlPlaneResourceGroups instead of workerResourceGroups. This ensures that the single node selected for both roles matches the control plane resource group constraints.

For step-by-step instructions, refer to Deploy a Single-Node Compute Pool.

Worker Pool Configuration

Use nodePoolRequirements.workerPools to define worker pools.

• Create multiple worker pools to separate workload types (for example, CPU-only and GPU-enabled).

• Each worker pool can have different resource requirements.

• Set gpu values for GPU-enabled worker pools.

• Each worker pool must have a unique name within the ComputePool.

Empty Worker Pools

The workerPools array can be empty if and only if:

• The control plane has more than one node (nodeCount is 3 or 5)
• controlPlanePool.workerNodeEligible is set to true

For single-node clusters (singleNodeCluster: true), at least one worker pool is required even when workerNodeEligible is true; the worker pool is used for edge host selection. For dedicated (non-worker-eligible) control planes, at least one worker pool is required because workloads cannot schedule on the control plane.

workerPools:
  - name: cpu-pool
    architecture: AMD64
    cpu:
      cpuCount: 4
  - name: gpu-pool
    architecture: AMD64
    cpu:
      cpuCount: 8
    gpu:
      variant: 'NVIDIA A100-SXM4-80GB | 80 GB'
      gpuCount: 2
      gpuMemory: '160 GB'

Worker Pool Fields

Required fields:

• name. The unique name of the worker pool. Must start with a letter and must not contain any of the following characters: /&#+%?,$. Must not exceed 62 characters. These naming rules apply when a worker pool is first named; names that already exist on a Compute Pool remain valid on update.

Optional fields:

• architecture. CPU architecture (AMD64 or ARM64). Defaults to AMD64.

• cpu. CPU requirements for the worker pool.

  • cpuCount. Number of CPU cores per node.

• memory. Memory requirements for the worker pool.

  • memory. Memory per node as a string with units (for example, '16 GB' or '8192 MiB').

• gpu. GPU requirements for GPU-enabled worker pools.

  • variant. GPU variant name (for example, NVIDIA A100-SXM4-80GB | 80 GB). The deprecated family field is retained for backward compatibility and is not read by controllers.

  • gpuCount. Total number of GPUs across all selected edge hosts in the pool.

  • gpuMemory. Total GPU memory across all selected edge hosts, as a string with units (for example, 160 GB).

• labels. Key-value pairs to apply to nodes in this worker pool.

• annotations. Key-value pairs to attach metadata to nodes in this worker pool.

Worker Pool Names

Each worker pool in nodePoolRequirements.workerPools must include a name field. Worker pool names must not exceed 62 characters.

Worker pool names must start with a letter and must not contain any of the following characters: /&#+%?,$. These naming rules apply when a worker pool is first named: when you create a Compute Pool, or when you add or rename a worker pool on an update. Worker pool names that already exist on a Compute Pool remain valid on update.

Worker pool names provide a stable reference for matching allocated machine pools back to their requirements. This enables accurate pool evaluation when requirements change, such as during scaling operations.

Example worker pool with explicit name

workerPools:
  - name: gpu-workers
    gpu:
      variant: 'NVIDIA A100-SXM4-80GB | 80 GB'
      gpuCount: 8
    minWorkerNodes: 2

Worker Pool Taints

You may apply Kubernetes taints to worker pools. Taints control workload scheduling by preventing workloads from being scheduled on nodes unless the workloads have matching tolerations. This is useful when you want to reserve specific node pools for particular types of workloads, such as GPU-intensive applications or workloads with special hardware requirements.

You can configure taints in the nodePoolRequirements.workerPools section of a ComputePool. Each taint consists of a key, value, and effect:

• key - The taint key to apply to nodes in the worker pool.
• value - The taint value associated with the key.
• effect - The effect of the taint. Possible values include NoSchedule, PreferNoSchedule, and NoExecute.

Workloads must include matching tolerations in their pod specifications to be scheduled on nodes with taints. Refer to the Kubernetes taints and tolerations documentation for more information.

Example worker pool with taints

workerPools:
  - name: gpu-workers
    gpu:
      variant: 'NVIDIA A100-SXM4-80GB | 80 GB'
      gpuCount: 8
    minWorkerNodes: 2
    taints:
      - key: workload-type
        value: gpu-intensive
        effect: NoSchedule
      - key: hardware
        value: high-performance
        effect: PreferNoSchedule

Edge Configuration

Use edge to configure edge clusters.

VIP Requirements

The Virtual IP (VIP) is the control plane endpoint for the Kubernetes cluster. VIP is required when network overlay is disabled. When network overlay is enabled, the Kubernetes API server endpoint is derived from the overlay CIDR, so VIP may be omitted. Proper VIP planning is critical for successful cluster deployment when VIP is required.

VIP Planning Checklist:

VIPs are required only when network overlay is not enabled. If you enable network overlay configuration (networkOverlayConfig.enabled: true), the VIP is omitted and these steps do not apply.

Before provisioning a Compute Pool without network overlay, complete the following VIP requirements:

1. Reserve the VIP from your network team or IPAM system. The VIP must be allocated and documented before cluster creation.

2. Verify network placement:

   • The VIP must be in the same Layer 2 network segment as the control plane nodes.
   • The VIP must be reachable from all edge hosts that will join the cluster.
   • The VIP must not be in a CIDR range routed through a proxy.

3. Confirm VIP availability:

   • The VIP must not be assigned to any other device or cluster.
   • Verify no IP conflicts exist using Address Resolution Protocol (ARP) or ping tests.

4. VIP format:

   • The VIP can be an IPv4 address (for example, 10.10.162.130) or a Fully Qualified Domain Name (FQDN) that resolves to an IP address.
   • Ensure DNS resolution is configured if using an FQDN.

5. VIP advertisement mechanism:

   • PaletteAI edge clusters use kube-vip to advertise the VIP using Layer 2 ARP or BGP.
   • Layer 2 (ARP) mode is the default and most common configuration:
     • The VIP is advertised via ARP (Address Resolution Protocol)
     • Requires that control plane nodes are on the same L2 network segment
     • Confirm that your network allows ARP traffic between control plane nodes
     • No additional configuration needed in the ComputePool manifest
   • BGP mode (advanced) requires additional Palette cluster profile configuration:
     • The VIP is advertised via BGP peering with network routers
     • Requires BGP configuration in the Palette cluster profile (not in the ComputePool manifest)
     • Use when control plane nodes span multiple L2 segments
     • Requires coordination with your network team for BGP peering setup
   • How to determine which mode is used:
     • By default, kube-vip operates in Layer 2 (ARP) mode
     • BGP mode must be explicitly configured in the Palette infrastructure profile
     • Check your ProfileBundle's referenced Palette cluster profile for kube-vip BGP settings
     • If no BGP configuration exists, Layer 2 mode is active

Pre-flight VIP validation:

Before applying the Compute Pool manifest, validate the VIP from each edge host:

# Verify VIP is not in use (should timeout or fail)
ping -c 3 <vip-address>

# Check ARP table for conflicts (should return no results)
arp -a | grep <vip-address>

# Verify VIP is in same subnet as control plane nodes
ip route get <vip-address>

# If using FQDN, verify DNS resolution
nslookup <vip-fqdn>

VIP constraints:

• The VIP is immutable after cluster creation. You cannot change the VIP without deleting and recreating the cluster.

• The VIP must be unique across all Compute Pools in your environment.

• The VIP must not be used by any other device, service, or load balancer on your network.

Example edge configuration with VIP:

# This configuration goes under:
# spec.clusterVariant.dedicated.paletteClusterDeploymentConfig.edge
edge:
  vip: '10.10.162.130'
  ntpServers:
    - time.google.com
    - time.cloudflare.com
  networkOverlayConfig:
    enabled: true
    cidr: '192.168.1.0/24'

Optional Edge Settings

YAML path: spec.clusterVariant.dedicated.paletteClusterDeploymentConfig.edge

Additional edge configuration options (all are optional except VIP, which is required only when network overlay is disabled):

• NTP servers (ntpServers): Configure time synchronization servers. Use this when edge hosts do not have default NTP configuration or when specific time sources are required.

  edge:
    ntpServers:
      - time.google.com
      - time.cloudflare.com

• Network overlay (networkOverlayConfig): Configure VXLAN overlay network for pod-to-pod communication across nodes. Use this for advanced networking scenarios or when the underlying network does not support native pod routing.

  edge:
    networkOverlayConfig:
      enabled: true
      staticIp: false
      cidr: '192.168.1.0/24'

  Fields:

  • enabled (boolean) - Enable or disable overlay network
  • staticIp (boolean) - Use static IP assignment for overlay
  • cidr (string) - CIDR range for overlay network

  The schema also accepts an overlayNetworkType field, but it is not read when deploying the cluster. VXLAN is the only supported overlay network type.

• SSH keys (sshKeys): Configure SSH access to cluster nodes. Use this for operational access and troubleshooting. This is under paletteClusterDeploymentConfig (sibling to edge), not inside edge.

  paletteClusterDeploymentConfig:
    sshKeys:
      - 'ssh-rsa AAAAB3Nza... your-key-1'
      - 'ssh-rsa AAAAB3Nza... your-key-2'

• Two-node deployment (isTwoNode): Enable two-node cluster mode. Special configuration for edge deployments with exactly two nodes.

  edge:
    isTwoNode: true

  warning

  Two-node deployment is only supported with Palette Optimized K3s or Palette Optimized Canonical Kubernetes. Other Kubernetes distributions (such as Palette Optimized RKE2 or PXK-E) do not support two-node deployment.

ProfileBundle Reference

profileBundleRef defines the infrastructure stack and, optionally, application stacks for deployment. This is a top-level field under spec.

YAML path: spec.profileBundleRef

Required fields:

• name - Name of the ProfileBundle resource

• namespace - Namespace of the ProfileBundle (must match the Project namespace)

Optional fields:

• cloudType - Cloud provider type (edge-native or maas)

ProfileBundle types:

• infrastructure - Contains only infrastructure profiles (Kubernetes, networking, storage)

• fullstack - Contains both infrastructure and application profiles

• application - Contains only application profiles (for Imported clusters only)

Example:

spec:
  profileBundleRef:
    name: edge-profilebundle
    namespace: my-project
    cloudType: edge-native

System-Level Sharing

The sharedWith field controls which Tenants and Projects can view and clone system-scoped ComputePools that exist in the pai-system namespace. This field is ignored for ComputePools in non-system namespaces.

YAML path: spec.sharedWith

The field is a map where keys are Tenant name selectors and values are lists of Project name selectors. Both keys and values support globs (*) and negation (!), as described in Sharing Resources.

Semantics:

• Field not set - Defaulted to {"*": ["*"]}: all Tenants and all of their Projects have access to the ComputePool.

• Empty map ({}) - No Tenants have access to the ComputePool.

• One entry per Tenant selector - Matching Tenants have access to the ComputePool. The value controls Project access within those Tenants:

  • null or empty list ([]) - The Tenant is visible, but none of its Projects have access.
  • ['*'] - All Projects in that Tenant have access.
  • Non-empty list - Only the Projects matching the listed selectors have access.

For the full policy language, including negation and how deny entries resolve, refer to Sharing Resources.

Example:

apiVersion: spectrocloud.com/v1alpha1
kind: ComputePool
metadata:
  name: shared-gpu-pool
  namespace: pai-system
spec:
  sharedWith:
    tenant-a: ['*'] # All Projects in tenant-a can access
    tenant-b: [] # tenant-b is visible, but none of its Projects can access
    tenant-c: # Only specific Projects in tenant-c can access
      - project-1
      - project-2
  profileBundleRef:
    name: gpu-profilebundle
    namespace: pai-system

Resource Groups

Resource groups restrict which machines a Compute Pool can use. Tag machines in Palette with labels that begin with palette.ai.rg/, then reference the same key-value pairs in the ComputePool manifest.

YAML path: spec.clusterVariant.controlPlaneResourceGroups and spec.clusterVariant.workerResourceGroups

The key-value pair in palette.ai.rg/<key>: "<value>" assigned to the machine must match the key-value pair defined in the Compute Pool for the machine to be added to the Compute Pool. For example, a machine tagged palette.ai.rg/network-pool: '1' matches a Compute Pool that defines network-pool: '1'.

Example resource group configuration

spec:
  clusterVariant:
    controlPlaneResourceGroups:
      network-pool: '1'
    workerResourceGroups:
      storage-tier: 'high-performance'

For single-node clusters, worker pool host selection uses controlPlaneResourceGroups instead of workerResourceGroups. Refer to Single-Node Clusters for details.

Autoscaling Configuration

To enable autoscaling, reference a ScalingPolicy resource in the Compute Pool's clusterVariant configuration. Both dedicated and shared Compute Pools support autoscaling.

Example dedicated ComputePool with autoscaling

spec:
  clusterVariant:
    dedicated:
      scalingPolicyRef:
        name: my-scaling-policy
        namespace: default

Example shared ComputePool with autoscaling

spec:
  clusterVariant:
    shared:
      scalingPolicyRef:
        name: my-scaling-policy
        namespace: default

Scaling policies define utilization thresholds, scaling durations, resource bounds, and cooldown periods. Scaling behavior details:

• Scaling is only triggered when utilization strictly crosses the configured thresholds, not when equal to the threshold. This prevents unnecessary scaling operations when utilization is at the boundary.

• PaletteAI tracks the status of each scaling action in the ComputePoolEvaluation resource. Host status transitions through Provisioning, Healthy, Unhealthy, Failed, or Deleting states during scaling operations. For an example-driven guide to inspecting this resource and interpreting its scaling decisions, refer to Debug Scaling Decisions with ComputePoolEvaluation.

• If a scale-up operation does not complete within the configured abort duration, PaletteAI aborts the operation by removing pending nodes that have not reached Healthy status. Successfully provisioned nodes are retained, and the pool transitions to a cooldown period. Scale-down operations are not aborted and continue until all node removals complete.

For step-by-step instructions on creating and managing Scaling Policies, refer to Create and Manage Scaling Policies.

Status Reference

PaletteAI tracks hardware resources and machine pool allocations in the ComputePool status field. This information determines whether a Compute Pool can accept additional workloads and is used for GPU quota enforcement at the Project level.

View the status of a Compute Pool with the following command.

kubectl get computepool <pool-name> --namespace <project-namespace> --output yaml

Status Fields

• status.hardwareCapacity - Total resources available across all control plane and worker nodes in the Compute Pool's clusters. This includes CPU count, architecture, memory, and GPU variant and count. For MIG-capable GPUs, each GPU entry includes an optional mig field that provides detailed MIG configuration and layout information.
• status.hardwareAllocation - Resources currently allocated to App Deployments running on the Compute Pool. Allocation is summed across all applications deployed to the pool.
• status.totalHardwareAllocated - Sum of physical compute resources from every allocated machine pool across all Palette clusters in the Compute Pool. Reflects the actual hardware currently backing the Compute Pool, including control plane and worker pools. Entries are grouped by architecture, with one entry per architecture.
• status.hardwareAllocatedByNodePool - Physical compute resources allocated to each machine pool, kept distinct per Palette cluster so that pools in multiple clusters of a Shared Compute Pool are not aggregated together. Each entry contains clusterName (the Palette cluster the machine pool belongs to), poolName (the allocated machine pool name, such as control-plane-pool), and hardware (the summed physical compute of the pool's hosts).
• status.status - Overall health of the Compute Pool. Possible values include Running, Failed, Provisioning, Unknown, Unhealthy, Updating, and Deleting. The Unknown status indicates that the underlying cluster's health cannot be determined, which may occur when spoke connectivity or managed cluster availability cannot be verified. Compute Pools with Failed or Deleting status cannot be selected for App Deployments or other workloads.
• status.paletteClusterStatuses - Contains status information for each Palette cluster in the Compute Pool. Each entry includes:
  • health: The overall health of the Palette cluster:
    • Healthy: The cluster is operating normally.
    • UnHealthy: The cluster has health issues.
    • Unknown: The cluster health status is not available.
    • Maintenance Mode: The cluster is in maintenance mode.
  • cloudConfigUUID: The UUID of the cloud configuration used to provision the Palette cluster. This is used to update the machine pools of the cluster.
  • allocatedMachinePools: Tracks which Edge Hosts are allocated to each machine pool within the cluster. Each entry includes:
    • name: The machine pool name (for example, control-plane-pool or worker-pool-nvidia-amd64-0).
    • nodeType: The type of nodes in this machine pool. Possible values:
      • ControlPlaneOnly: Only control plane nodes.
      • WorkerOnly: Only worker nodes.
      • ControlPlaneAndWorker: Nodes serve as both control plane and worker.
    • hosts: Map of Edge Host UIDs to their details (architecture, CPU count, memory, GPU count and memory by variant, status, and status timestamp). Each host includes:
      • status: Provisioning and health status of the host. Possible values: Initial, Provisioning, Healthy, Unhealthy, Failed, Deleting, Unknown.
      • statusUpdatedAt: Timestamp of the last status change for the host.
    • labels: Kubernetes labels applied to nodes in this pool (for example, ["control-plane"] or ["worker", "gpu-model-nvidia-a100-sxm4-80gb"]).
    • workerPoolRequirementsName: (Worker pools only) The name of the WorkerPool requirement that this machine pool was created from. Used to match allocated pools back to their requirements for scaling operations.

MIG Hardware Capacity

For MIG-capable GPUs, the mig field appears within GPU entries in status.hardwareCapacity and provides detailed NVIDIA Multi-Instance GPU (MIG) configuration and layout information. MIG fields are only populated when the Compute Pool's SpokeNodes report MIG configuration state. For MIG concepts, strategies, and the Day 1 and Day 2 configuration workflows, refer to Multi-Instance GPU. This field includes:

• config - The MIG configuration preset name (for example, all-1g.10gb).
• strategy - The MIG partitioning strategy. Possible values include single (all GPUs use the same MIG profile) or mixed (GPUs can use different MIG profiles).
• configurationState - Counters for the current MIG configuration state across nodes, with fields for pending, rebooting, success, and failed nodes.
• layouts - Available MIG device layouts, each showing:
  • layoutGpuCount - Total count of GPUs using this layout.
  • devices - Array of MIG device profiles available in this layout, each containing:
    • profile - The MIG profile name (for example, 1g.10gb).
    • deviceCountPerGPU - Number of MIG devices per GPU.
    • gpuSlicesPerDevice - Number of GPU slices allocated to each MIG device.
    • gpuMemoryPerDevice - Amount of GPU memory allocated to each MIG device (for example, 10GB).

Example Status

Example Compute Pool status

status:
  status: Running
  hardwareCapacity:
    - architecture: AMD64
      totalCPU: 32
      totalMemory: '128Gi'
      gpu:
        - variant: 'NVIDIA A100-SXM4-80GB | 80 GB'
          gpuCount: 8
          mig:
            config: all-1g.10gb
            strategy: single
            configurationState:
              success: 8
            layouts:
              - layoutGpuCount: 8
                devices:
                  - profile: 1g.10gb
                    deviceCountPerGPU: 7
                    gpuSlicesPerDevice: 1
                    gpuMemoryPerDevice: 10GB
  hardwareAllocation:
    - architecture: AMD64
      gpu:
        - variant: 'NVIDIA A100-SXM4-80GB | 80 GB'
          gpuCount: 4
  paletteClusterStatuses:
    - health: Healthy
      cloudConfigUUID: cloud-config-uuid-1
      allocatedMachinePools:
        - name: control-plane-pool
          nodeType: ControlPlaneAndWorker
          hosts:
            host-uid-1:
              architecture: AMD64
              cpuCount: 16
              memoryGB: 64
            host-uid-2:
              architecture: AMD64
              cpuCount: 16
              memoryGB: 64
          labels:
            - control-plane
        - name: worker-pool-nvidia-amd64-0
          nodeType: WorkerOnly
          workerPoolRequirementsName: gpu-workers
          hosts:
            host-uid-3:
              architecture: AMD64
              cpuCount: 32
              memoryGB: 128
              gpuCountByVariant:
                'NVIDIA A100-SXM4-80GB | 80 GB': 4
              gpuMemoryGBByVariant:
                'NVIDIA A100-SXM4-80GB | 80 GB': 160
              status: Healthy
              statusUpdatedAt: '2024-01-15T10:30:00Z'
            host-uid-4:
              architecture: AMD64
              cpuCount: 32
              memoryGB: 128
              gpuCountByVariant:
                'NVIDIA A100-SXM4-80GB | 80 GB': 4
              gpuMemoryGBByVariant:
                'NVIDIA A100-SXM4-80GB | 80 GB': 160
              status: Healthy
              statusUpdatedAt: '2024-01-15T10:32:00Z'
          labels:
            - worker
            - gpu-model-nvidia-a100-sxm4-80gb
  aiWorkloadRefs:
    - name: training-job-1
      namespace: project-a

Machine Pool Lifecycle

PaletteAI automatically reconciles machine pools to match the Compute Pool's requirements. When you update pool requirements, PaletteAI classifies the changes and performs the appropriate operations:

• Create - Adds a machine pool for a requirement that does not have an allocated pool.
• Delete - Removes a machine pool that no longer matches any requirement. PaletteAI deletes individual machines first, marking their hosts as Deleting. Once all machines are confirmed removed, the pool itself is deleted.
• Scale - Adds or removes hosts from a pool to match updated requirements. When removing hosts, PaletteAI waits for at least one replacement host to reach Healthy status before removing the old hosts, preventing downtime.
• Replace - Rebuilds a pool when all hosts are invalid (for example, due to hardware requirement changes). PaletteAI adds replacement hosts while keeping one "bridge" host from the old set. Once a replacement host reaches Healthy status, the bridge host is removed.

For single-node clusters, day-2 machine pool operations are not performed. Refer to Single-Node Clusters for details.

Deletion Policy

deletionPolicy controls what happens to the Palette cluster when you delete the Compute Pool resource from PaletteAI.

YAML path: spec.deletionPolicy

info

The nested spec.clusterVariant.dedicated.paletteClusterDeploymentConfig.deletionPolicy field is deprecated. Use the top-level spec.deletionPolicy field instead.

• delete (default). Deletes the Palette cluster when the Compute Pool is deleted. Use this for ephemeral or development clusters where you want complete cleanup.

• orphan. Keeps the Palette cluster running when the Compute Pool is deleted. The cluster becomes unmanaged by PaletteAI. Use this when you want to preserve the cluster infrastructure but stop managing it through PaletteAI.

Affected Resources

When a Compute Pool is deleted, PaletteAI cleans up associated resources. Which resources are removed depends on the deletion policy.

Always deleted (regardless of policy):

• AIWorkloads — all AIWorkloads referencing the Compute Pool are force-deleted before the Compute Pool is removed. Orphaning AIWorkloads is not supported.
• WorkloadDeployments — all WorkloadDeployments associated with the Compute Pool itself are deleted. Such WorkloadDeployments only exist when the active infrastructure variant from the Compute Pool's Profile Bundle includes WorkloadProfiles of type infrastructure. The controller waits for all WorkloadDeployments to be fully removed before proceeding with cluster cleanup.

Deleted when deletionPolicy: delete:

• Palette Cluster — the underlying Kubernetes cluster is deleted from Palette.
• Environment — the Environment resource created for the Compute Pool is deleted. For Shared Compute Pools with multiple clusters, the Environment is only deleted if all clusters have deletionPolicy: delete. If any cluster uses orphan, the Environment is preserved.

Preserved when deletionPolicy: orphan:

• Palette Cluster — the cluster continues running in Palette, unmanaged by PaletteAI.
• Environment — the Environment resource is retained.

Example:

spec:
  deletionPolicy: delete

Orphan mode cleanup:

When using deletionPolicy: orphan, you must manually clean up resources:

1. The Kubernetes cluster remains running in Palette and continues to consume infrastructure resources.

2. Any workloads deployed by PaletteAI remain running but are no longer managed.

3. You must manually delete the cluster from Palette if you no longer need it.

4. Edge hosts remain allocated to the cluster until you delete the cluster from Palette.

UI Field to YAML Mapping

The following table maps User Interface wizard fields to their corresponding YAML paths in the ComputePool manifest. Use this when translating UI configurations created in Canvas to Kubernetes manifests for kubectl or GitOps workflows.

UI Wizard Step	UI Field Name	YAML Path	Notes
General	Compute pool name	metadata.name	Must be unique in namespace
General	Description	metadata.annotations['spectrocloud.com/description']	The canonical annotation key is spectrocloud.com/description. Resources created before this key was standardized may use the legacy key palette-ai/description, which is read as a fallback.
General	Labels / Annotations	metadata.labels / metadata.annotations	Key-value pairs. Refer to Operational Labels.
Mode	Dedicated resources	spec.clusterVariant.dedicated	Mutually exclusive with shared
Profile Bundle	Profile Bundle	spec.profileBundleRef.name	Must exist in same namespace
Profile Bundle	Cloud Type	spec.profileBundleRef.cloudType	Typically edge-native
Profile Bundle	Version	spec.profileBundleRef.name	To select a specific revision, use the format <profile-bundle-name>@v<revision> (for example, edge-profilebundle@v3).
Node Config	Node Count (control plane)	spec.clusterVariant.dedicated .paletteClusterDeploymentConfig .nodePoolRequirements .controlPlanePool.nodeCount	Values: 1, 3, or 5
Node Config	Workload Eligible	spec.clusterVariant.dedicated .paletteClusterDeploymentConfig .nodePoolRequirements .controlPlanePool.workerNodeEligible	Boolean: true for single-node
Node Config	Architecture	spec.clusterVariant.dedicated .paletteClusterDeploymentConfig .nodePoolRequirements .controlPlanePool.architecture	Values: AMD64, ARM64
Node Config	CPU Count	spec.clusterVariant.dedicated .paletteClusterDeploymentConfig .nodePoolRequirements .controlPlanePool.cpu.cpuCount	Integer
Node Config	Memory	spec.clusterVariant.dedicated .paletteClusterDeploymentConfig .nodePoolRequirements .controlPlanePool.memory.memory	String with units (for example, 16 GB or 16384 MiB).
Node Config	Worker Pool Name	spec.clusterVariant.dedicated .paletteClusterDeploymentConfig .nodePoolRequirements .workerPools[].name	Required. Must be unique within the ComputePool.
Node Config	Worker Pool Architecture	spec.clusterVariant.dedicated .paletteClusterDeploymentConfig .nodePoolRequirements .workerPools[].architecture	Array element
Node Config	Worker Pool CPU Count	spec.clusterVariant.dedicated .paletteClusterDeploymentConfig .nodePoolRequirements .workerPools[].cpu.cpuCount	Integer
Node Config	Worker Pool Memory	spec.clusterVariant.dedicated .paletteClusterDeploymentConfig .nodePoolRequirements .workerPools[].memory.memory	String with units (for example, 16 GB)
Node Config	Min Worker Nodes	spec.clusterVariant.dedicated .paletteClusterDeploymentConfig .nodePoolRequirements .workerPools[].minWorkerNodes	Minimum nodes to provision
Node Config	GPU Variant	spec.clusterVariant.dedicated .paletteClusterDeploymentConfig .nodePoolRequirements .workerPools[].gpu.variant	e.g., NVIDIA A100-SXM4-80GB | 80 GB
Node Config	GPU Count	spec.clusterVariant.dedicated .paletteClusterDeploymentConfig .nodePoolRequirements .workerPools[].gpu.gpuCount	Integer. Total across all selected edge hosts.
Node Config	GPU Memory	spec.clusterVariant.dedicated .paletteClusterDeploymentConfig .nodePoolRequirements .workerPools[].gpu.gpuMemory	String with units (for example, 160 GB). Total across all selected edge hosts.
Deployment	Deletion Policy	spec.deletionPolicy	Values: delete, orphan
Deployment	Settings Ref	spec.clusterVariant.dedicated .paletteClusterDeploymentConfig .settingsRef.name	Settings resource name used for Palette integration.
Deployment	SSH Keys	spec.clusterVariant.dedicated .paletteClusterDeploymentConfig .sshKeys	Array of SSH public keys
Deployment	VIP	spec.clusterVariant.dedicated .paletteClusterDeploymentConfig .edge.vip	Required when network overlay is disabled. Immutable. Omitted when networkOverlayConfig.enabled: true.
Deployment	Network Overlay (enabled)	spec.clusterVariant.dedicated .paletteClusterDeploymentConfig .edge.networkOverlayConfig .enabled	Boolean
Deployment	Network Overlay CIDR	spec.clusterVariant.dedicated .paletteClusterDeploymentConfig .edge.networkOverlayConfig .cidr	CIDR notation

Operational Labels

Operational labels control ComputePool behavior.

Pause Label

Use the controller.spectrocloud.com/pause label to pause ComputePool reconciliation. When set to "true" (as a string), the controller skips reconciliation for the ComputePool.

YAML path: metadata.labels

Example:

apiVersion: spectrocloud.com/v1alpha1
kind: ComputePool
metadata:
  name: ml-cluster
  namespace: my-project
  labels:
    controller.spectrocloud.com/pause: 'true'
spec:
  # ... rest of configuration

Use this label to temporarily halt operations on a ComputePool without deleting it. This is useful during maintenance windows or troubleshooting. To resume reconciliation, remove the label or set it to any value other than "true".

info

The pause label only stops normal ComputePool reconciliation. It does not prevent deletion. When a ComputePool has metadata.deletionTimestamp set, the controller's delete and finalizer logic still executes even when the pause label is present. Therefore, the pause label cannot be used to block teardown or deletion of a ComputePool.

Status

Each Compute Pool's status includes the following fields.

• status.status - Overall health of the Compute Pool. Possible values include Running, Failed, Provisioning, Unknown, Unhealthy, Updating, and Deleting. The Unknown status indicates that the underlying cluster's health cannot be determined, which may occur when spoke connectivity or managed cluster availability cannot be verified. Compute Pools with Failed or Deleting status cannot be selected for App Deployments or other workloads.

• status.conditions - Array of conditions representing the latest available observations of the ComputePool's ComputePool-level state. Tracks conditions such as Validated, RBACConfigured, and GarbageCollected. Each condition follows standard Kubernetes condition semantics and includes:

  • type: The condition type in camelCase (for example, Validated).
  • status: The condition status (True, False, or Unknown).
  • reason: A programmatic identifier indicating the reason for the condition's last transition.
  • message: A human-readable message indicating details about the transition.
  • lastTransitionTime: The last time the condition transitioned from one status to another.
  • observedGeneration: The .metadata.generation that the condition was set based upon.

  These conditions are distinct from per-Palette-cluster conditions, which live on each paletteClusterStatuses[].conditions entry.

• status.workloadDeployments - Array of WorkloadDeployment resource names that have been reconciled by the compute pool controller. This field is populated during workload reconciliation and tracks the WorkloadDeployment resources associated with the Compute Pool.

• status.paletteClusterStatuses - Contains status information for each Palette cluster in the Compute Pool. Each entry includes:

  • health: The overall health of the Palette cluster:
    • Healthy: The cluster is operating normally.
    • UnHealthy: The cluster has health issues.
    • Unknown: The cluster health status is not available.
    • Maintenance Mode: The cluster is in maintenance mode.
  • cloudConfigUUID: The UUID of the cloud configuration used to provision the Palette cluster. This is used to update the machine pools of the cluster.
  • allocatedMachinePools: Tracks which Edge Hosts are allocated to each machine pool within the cluster. Each entry includes:
    • name: The machine pool name (e.g., "control-plane-pool", "worker-pool-nvidia-amd64-0").
    • nodeType: The type of nodes in this machine pool. Possible values:
      • ControlPlaneOnly: Only control plane nodes.
      • WorkerOnly: Only worker nodes.
      • ControlPlaneAndWorker: Nodes serve as both control plane and worker.
    • hosts: Map of Edge Host UIDs to their details (architecture, CPU count, memory, GPU count and memory by variant, status, and status timestamp). Each host includes:
      • status: Provisioning and health status of the host. Possible values: Initial, Provisioning, Healthy, Unhealthy, Failed, Deleting, Unknown.
      • statusUpdatedAt: Timestamp of the last status change for the host.
    • labels: Kubernetes labels applied to nodes in this pool (e.g., ["control-plane"], ["worker", "gpu-model-nvidia-a100-sxm4-80gb"]).
    • workerPoolRequirementsName: (Worker pools only) The name of the WorkerPool requirement that this machine pool was created from. Used to match allocated pools back to their requirements for scaling operations.

Related Resources

• Create and Manage Compute Pools

• Troubleshooting Compute Pools

• Compute Pool Concepts