Create and Manage Scaling Policies

This guide describes how to create and manage Scaling Policies. A Scaling Policy defines autoscaling behavior for a Compute Pool by configuring CPU and GPU utilization thresholds, scaling durations, resource bounds, and cooldown periods.

When a Compute Pool references a Scaling Policy, PaletteAI continuously monitors resource utilization and automatically adds or removes nodes to match workload demand.

You can create Scaling Policies using the PaletteAI UI or using YAML Kubernetes manifests.

Prerequisites

Before you create a Scaling Policy, confirm that you have the following resources available.

UI Workflow

• The spectrocloud.com/scalingpolicies:create permission. If the Create Scaling Policy button is not displayed, your role does not include the required permission.

• A Compute Pool in Running status to enable autoscaling on.

• Prometheus running and accessible from the hub cluster. The ScalingPolicy controller queries Prometheus for CPU and GPU utilization metrics. Refer to Configure Prometheus Agent Monitoring to configure global.metrics, spoke-side Prometheus agents, and GPU metric collection prerequisites.

YAML Workflow

• Access to the hub cluster with permissions to create ScalingPolicy resources in the target namespace.

• A Compute Pool in Running status to enable autoscaling on.

• Prometheus running and accessible from the hub cluster. The ScalingPolicy controller queries Prometheus for CPU and GPU utilization metrics. A PrometheusAvailable condition is set on the ScalingPolicy status when the connection is confirmed.

  Refer to Configure Prometheus Agent Monitoring to configure global.metrics, spoke-side Prometheus agents, and GPU metric collection prerequisites.

Create a Scaling Policy

A Scaling Policy must include at least one CPU or GPU scaling configuration.

UI Workflow

1. Log in to the PaletteAI console.

2. Select the Project in which you want to create the Scaling Policy.

3. From the left main menu, select Settings, and then select the Autoscaling Policies tab.

4. Select Create Scaling Policy. The Create Auto-Scaling Policy dialog appears.

5. Enter a Scaling Policy Name.

6. In the Metrics section, select Add Metric. Choose the metric to scale on, either CPU or a GPU variant, and configure the Scale Up Threshold, Scale Up Duration, Scale Down Threshold, and Scale Down Duration values. Repeat this step for each metric you want the policy to evaluate.

7. (Optional) In the Resource Bounds section, select Add Resource Bound and set the minimum and maximum CPU or GPU counts the policy may scale between.

8. (Optional) Adjust the Cool Down Duration and Timeout Duration values.

9. Select Create. The new policy appears in the Autoscaling Policies table.

info

Tenant administrators can also manage Scaling Policies for all Projects from the Tenant Settings > Autoscaling Policies menu, which provides Tenant and Projects tabs.

YAML Workflow

A ScalingPolicy is a namespaced resource in the spectrocloud.com/v1alpha1 API group. You must specify at least one of cpu or gpu scaling configurations.

CPU Scaling Policy

Use this configuration when your Compute Pool uses CPU-based workloads.

cpu-scaling-policy.yaml

apiVersion: spectrocloud.com/v1alpha1
kind: ScalingPolicy
metadata:
  name: my-cpu-policy
  namespace: my-project-namespace
spec:
  cpu:
    scaleUpThreshold: 80
    scaleDownThreshold: 20
    scaleUpDuration: 5m
    scaleDownDuration: 10m
  cpuResourceBounds:
    minCPUCount: 4
    maxCPUCount: 64
  cooldownDuration: 15m
  abortDuration: 30m

Apply the manifest to the hub cluster:

kubectl apply --filename cpu-scaling-policy.yaml

GPU Scaling Policy

Use this configuration when your Compute Pool uses GPU-based workloads. You must configure one gpu entry per GPU variant. Each GPU variant in gpuResourceBounds must have a corresponding entry in gpu.

gpu-scaling-policy.yaml

apiVersion: spectrocloud.com/v1alpha1
kind: ScalingPolicy
metadata:
  name: my-gpu-policy
  namespace: my-project-namespace
spec:
  gpu:
    - variant: 'NVIDIA H100 PCIe | 80 GB'
      scaleUpThreshold: 85
      scaleDownThreshold: 15
      scaleUpDuration: 3m
      scaleDownDuration: 8m
    - variant: 'NVIDIA A100-SXM4-80GB | 80 GB'
      scaleUpThreshold: 85
      scaleDownThreshold: 15
      scaleUpDuration: 3m
      scaleDownDuration: 8m
  gpuResourceBounds:
    - variant: 'NVIDIA H100 PCIe | 80 GB'
      minGPUCount: 1
      maxGPUCount: 16
    - variant: 'NVIDIA A100-SXM4-80GB | 80 GB'
      minGPUCount: 1
      maxGPUCount: 8
  cooldownDuration: 15m
  abortDuration: 30m

Apply the manifest to the hub cluster:

kubectl apply --filename gpu-scaling-policy.yaml

Combined CPU and GPU Scaling Policy

You can configure both CPU and GPU scaling in a single policy. This is useful when worker pools in the same Compute Pool contain both CPU-only and GPU nodes.

combined-scaling-policy.yaml

apiVersion: spectrocloud.com/v1alpha1
kind: ScalingPolicy
metadata:
  name: my-combined-policy
  namespace: my-project-namespace
spec:
  cpu:
    scaleUpThreshold: 80
    scaleDownThreshold: 20
    scaleUpDuration: 5m
    scaleDownDuration: 10m
  gpu:
    - variant: 'NVIDIA H100 PCIe | 80 GB'
      scaleUpThreshold: 85
      scaleDownThreshold: 15
      scaleUpDuration: 3m
      scaleDownDuration: 8m
  cpuResourceBounds:
    minCPUCount: 4
    maxCPUCount: 64
  gpuResourceBounds:
    - variant: 'NVIDIA H100 PCIe | 80 GB'
      minGPUCount: 1
      maxGPUCount: 16
  cooldownDuration: 15m
  abortDuration: 30m

Apply the manifest to the hub cluster:

kubectl apply --filename combined-scaling-policy.yaml

For a description of every field, including defaults and validation rules, refer to the ScalingPolicy Configuration reference.

Reference a Scaling Policy from a Compute Pool

A Compute Pool opts in to autoscaling by referencing a Scaling Policy.

UI Workflow

To attach a Scaling Policy while creating a Compute Pool:

1. On the Node config page, select Scaling Policy in the left panel under Autoscaling.

2. Turn on Auto-Scaling Policy.

3. Select the Scaling Policy, and continue the creation workflow.

To attach or change the Scaling Policy on an existing Compute Pool:

1. From the left main menu, select Compute Pools, and then select the Compute Pool.

2. In the top-right, select Settings > Compute Pool Settings.

3. Turn on Auto-Scaling Policy, select the Scaling Policy, and Save your changes.

YAML Workflow

Add a scalingPolicyRef to the Compute Pool's clusterVariant configuration. Set scalingPolicyRef.namespace to the namespace where the ScalingPolicy is deployed. The Compute Pool and the ScalingPolicy can be in different namespaces.

Dedicated Compute Pool:

spec:
  clusterVariant:
    dedicated:
      scalingPolicyRef:
        name: my-cpu-policy
        namespace: my-project-namespace

Shared Compute Pool:

spec:
  clusterVariant:
    shared:
      scalingPolicyRef:
        name: my-cpu-policy
        namespace: my-project-namespace

Apply the updated ComputePool manifest or patch the existing resource:

kubectl patch computepool my-compute-pool \
  --namespace my-project-namespace \
  --type merge \
  --patch '{"spec":{"clusterVariant":{"dedicated":{"scalingPolicyRef":{"name":"my-cpu-policy","namespace":"my-project-namespace"}}}}}'

After the reference is applied, PaletteAI begins evaluating metrics and the Scaling Policy status reflects the associated Compute Pool.

Validate

UI Workflow

1. From the left main menu, select Settings, and then select the Autoscaling Policies tab. Confirm the policy appears in the table along with its associated Compute Pools.

2. From the left main menu, select Compute Pools, and then select the Compute Pool that references the policy. On the Overview tab, confirm the Auto-Scaling field displays the policy name.

YAML Workflow

1. Confirm the ScalingPolicy resource exists and that Prometheus is available:

   kubectl get scalingpolicy my-cpu-policy --namespace my-project-namespace

   Example Output

   NAME            PROMETHEUS_AVAILABLE   PROCESSED_COMPUTEPOOLS   AGE
   my-cpu-policy   True                   1                        5m

   The PROMETHEUS_AVAILABLE column indicates whether the controller can reach Prometheus to query metrics. If it shows False, verify that Prometheus is running and accessible from the hub cluster.

2. Inspect the full status to confirm the associated Compute Pool is listed:

   kubectl describe scalingpolicy my-cpu-policy --namespace my-project-namespace

   Under Status, confirm that the processed pool count is greater than zero and that the name of your Compute Pool appears in the computePools list.

3. Confirm the ComputePoolEvaluation resource is created for the associated Compute Pool:

   kubectl get computepoolevaluation --namespace my-project-namespace

   A ComputePoolEvaluation resource is created for each Compute Pool that references the Scaling Policy. It records the current scaling decision and target resource counts. For an example-driven guide to reading this resource, refer to Debug Scaling Decisions with ComputePoolEvaluation.

Update a Scaling Policy

You can update a Scaling Policy you created or cloned in your project namespace at any time. Changes take effect on the next reconciliation cycle. Pre-defined policies (aggressive, balanced, conservative) in the platform namespace (mural-system) cannot be modified by users. To change their behavior, clone one to your project namespace and update the clone.

UI Workflow

1. From the left main menu, select Settings, and then select the Autoscaling Policies tab.

2. Beside the policy, select the three-dot menu, and choose Edit. To customize a pre-defined policy, choose Clone instead, and then edit the clone.

3. Adjust the metrics, resource bounds, and durations as needed, and Save your changes. A warning is displayed when the policy is in use, because editing it impacts all associated Compute Pools.

YAML Workflow

Update the manifest and re-apply it:

kubectl apply --filename cpu-scaling-policy.yaml

Or patch a specific field directly:

kubectl patch scalingpolicy my-cpu-policy \
  --namespace my-project-namespace \
  --type merge \
  --patch '{"spec":{"cooldownDuration":"20m"}}'

info

Updating a Scaling Policy does not interrupt active scaling operations. The new configuration applies after the current scaling action completes.

Delete a Scaling Policy

Before you delete a Scaling Policy, remove the reference from all Compute Pools that use it. PaletteAI prevents deletion of a Scaling Policy that is still referenced by active Compute Pools.

UI Workflow

1. For each Compute Pool that references the policy, open the Compute Pool, select Settings > Compute Pool Settings, turn off Auto-Scaling Policy, and Save your changes.

2. From the left main menu, select Settings, and then select the Autoscaling Policies tab.

3. Beside the policy, select the three-dot menu, and choose Delete.

4. In the confirmation dialog, select Delete.

YAML Workflow

1. Remove the scalingPolicyRef from each referencing Compute Pool:

   kubectl patch computepool my-compute-pool \
     --namespace my-project-namespace \
     --type merge \
     --patch '{"spec":{"clusterVariant":{"dedicated":{"scalingPolicyRef":null}}}}'

2. Delete the Scaling Policy:

   kubectl delete scalingpolicy my-cpu-policy --namespace my-project-namespace

   If any active Compute Pool still references the policy, the deletion is rejected with an error listing the referencing resources. Remove the remaining references and retry.

Next Steps

• Learn how Scaling Policies work, including the pre-defined profiles, in Scaling Policies.

• View all ScalingPolicy fields, defaults, and validation rules in ScalingPolicy Configuration.

• Learn about autoscaling behavior in Compute Pool concepts.