An AIWorkload is the custom resource that represents an App Deployment or Model Deployment on the PaletteAI hub cluster. When you complete the App Deployment or Model Deployment workflow, PaletteAI creates an AIWorkload resource in your Project namespace. The resource captures everything the deployment needs: the Profile Bundles to deploy, the workloads to generate from them, and the Compute Pool that runs them.
An App Deployment represents an AI/ML application deployed using a Profile Bundle; the Profile Bundle must contain a Workload Profile with the type Application. An App Deployment is the primary method that data scientists and ML engineers use to deploy their workloads onto Compute Pools.
A Compute Pool is a group of shared Compute resources that PaletteAI turns into one or more Kubernetes clusters where your AI/ML applications and models run. In the hub-spoke architecture, each Compute Pool becomes a spoke cluster on which applications and models are deployed.
Compute resources are how PaletteAI turns your physical machines into Kubernetes clusters that run AI/ML applications and models.
Definitions is an umbrella term that encapsulates Components, Traits, and Policies. They are reusable building blocks that describe how to deploy Workloads.
An Environment is an abstraction that dictates what clusters a workload should be deployed to.
Multi-Instance GPU (MIG) is an NVIDIA feature that partitions a supported GPU into as many as seven isolated GPU instances. Each instance has dedicated compute, memory, and memory bandwidth, so several workloads can share one physical GPU without competing for the same resources. MIG helps you increase GPU utilization when individual workloads do not need a full GPU.
A Scaling Policy is a reusable set of autoscaling rules for Compute Pools. It defines the CPU and GPU utilization thresholds that trigger scaling, how long utilization must hold before PaletteAI acts, the minimum and maximum resources a pool may scale between, and how long to wait between scaling actions. One policy can be attached to many Compute Pools, so teams can standardize scaling behavior instead of tuning each pool individually.
A Workload represents an application or task that runs on your infrastructure. PaletteAI creates Workloads from AIWorkload resources by using three building blocks called Definitions: