Hardware Support

PaletteAI is hardware-agnostic. You can run the PaletteAI control plane on commodity hardware and register a mix of CPU-only and GPU-equipped machines as the compute capacity that serves your AI/ML workloads. At the same time, PaletteAI supports silicon-optimized software so that you can take full advantage of accelerators from different vendors, including NVIDIA and AMD GPUs.

This page explains where PaletteAI runs, what hardware you can register for Compute Pools, and which Profile Bundles are optimized for specific accelerators.

Control Plane Hardware

The PaletteAI control plane, also called the hub cluster, requires no specialized hardware. It runs on any conformant Kubernetes cluster that meets the baseline resource requirements and does not need GPUs. The control plane orchestrates deployments and manages resources; it does not run AI/ML inference workloads itself.

The baseline system requirements are as follows.

• Use a Kubernetes cluster as the PaletteAI hub.

• Access to the hub cluster using the built-in Kubernetes cluster-admin ClusterRole.

• Minimum Kubernetes versions

  Cluster Type	Kubernetes Version
  Hub	>= 1.31.0
  Spoke	>= 1.31.0

• Minimum resource requests

  Cluster Type	CPU	Memory	Storage
  Hub	3388m	2732 Mi	10Gi
  Spoke	1216m	972 Mi	10Gi

• Ensure the hub cluster can reach the public AWS Elastic Container Registry (ECR) that hosts the mural and mural-crds charts.

• Access to the hub cluster kubeconfig file.

• Install Flux controllers on the hub cluster if you plan to use the recommended Flux-managed workflow.

• Install the following tools on the machine you use to install or upgrade PaletteAI:

  • curl or wget to download the Helm values file.

  • A text editor, such as vi, to edit the Helm values file.

  • kubectl version >= 1.31.0.

  • Helm version >= 3.17.0 if you plan to use the manual Helm workflow instead of the recommended Flux-managed workflow.

• Configure the hub cluster Kubernetes API server to trust Dex as an identity provider. PaletteAI deploys Dex as part of the installation. This requirement applies only to the hub cluster, not to spoke clusters. For details, refer to Configure Kubernetes API Server to Trust OpenID Connect (OIDC) Provider.

For platform-specific prerequisites and installation steps, refer to the guide for your platform: Vanilla Kubernetes, Appliance, AWS, AWS EKS, or GKE.

Workload Cluster Hardware

Workload clusters, also called spoke clusters, run your AI/ML applications and models. Depending on your workloads, a workload cluster can be one of the following.

• CPU-only - Suitable for lightweight inference, data preprocessing, and applications that do not require GPU acceleration.
• GPU-accelerated - Required for most large language model (LLM) inference and training workloads.
• Mixed - A combination of CPU-only and GPU-equipped nodes in the same Compute Pool. Within the workload cluster, PaletteAI places the cluster's Kubernetes control-plane nodes on CPU-only hosts when possible to maximize GPU availability for workloads.

You choose the hardware for a workload cluster by registering the appropriate machines and grouping them into a Compute Pool. Refer to Register an Edge Node and Create and Manage Compute Pools for the registration and pool-creation workflows.

Register GPU Nodes for Compute Pools

PaletteAI discovers registered machines through the Compute resource and groups them by hardware profile. For PaletteAI to discover a host at all, the host must first have the palette.ai: true label applied. Without this label, PaletteAI does not discover the machine, regardless of any GPU tags. After a host is discoverable, Palette's Edge agent automatically detects and reports GPU metadata, including the GPU vendor and family. When the agent cannot detect a value, you can supply it with Edge host tags such as gpu-family, gpus, and gpu-memory.

By default, PaletteAI treats machines with GPUs as worker candidates and machines without GPUs as candidates for the workload cluster's Kubernetes control-plane nodes. This node role is internal to each workload cluster and is unrelated to the PaletteAI hub described in Control Plane Hardware. You can override this behavior with the palette.ai/control-plane and palette.ai/worker tags. For the full set of discovery and role-eligibility tags, refer to Machine Discovery.

PaletteAI supports GPU accelerators from multiple vendors, including NVIDIA and AMD. The vendor and family that PaletteAI reports for each machine determine which GPU-optimized Profile Bundle you pair with the Compute Pool.

GPU-Optimized Profile Bundles

To use GPUs in a workload cluster, the Profile Bundle that backs the Compute Pool must deploy the GPU operator for your hardware vendor. The GPU operator manages the device plugins that expose the GPUs to Kubernetes and, depending on the vendor and Profile Bundle, may also manage driver installation. Refer to the vendor-specific requirements below.

You can browse and import ready-to-use Profile Bundles from PaletteAI Studio. Refer to Import a Profile Bundle for the import workflow.

NVIDIA GPUs

For NVIDIA hardware, the GPU operator to use is the NVIDIA GPU Operator, which manages NVIDIA driver installation and the NVIDIA device plugin.

PaletteAI ships a vLLM Prod Stack Profile Bundle out of the box that deploys the NVIDIA GPU Operator. Because the NVIDIA GPU Operator manages driver installation, you do not need to install NVIDIA drivers on the hosts manually when you use this bundle. Confirm the driver behavior against the bundle definition for your PaletteAI version.

AMD GPUs

For AMD hardware, the GPU operator to use is the AMD GPU Operator, which manages the AMD device plugin and integrates with the ROCm software stack.

PaletteAI does not ship an out-of-the-box Profile Bundle for AMD. Instead, an AMD Enterprise AI Profile Bundle is available in PaletteAI Studio. Import it the same way you import any other Profile Bundle.

Unlike the NVIDIA workflow, the AMD Enterprise AI Profile Bundle expects the ROCm and amdgpu kernel packages to be installed on each host rather than by the operator. Install these packages on every AMD GPU host through a Bring Your Own OS (BYOOS) after-install stage so that the drivers are present before the AMD GPU Operator starts.

info

The AMD after-install stage runs as part of the host's operating system provisioning. Match the ROCm and amdgpu package versions to your AMD accelerator model and kernel. Consult the AMD ROCm installation documentation for the versions supported on your hardware.

Next Steps

After you understand the hardware options, choose an installation method for the PaletteAI hub and then register your workload nodes.

• Install the PaletteAI hub using the guide for your platform: Vanilla Kubernetes, Appliance, AWS, AWS EKS, or GKE.
• Register an Edge Node to add compute capacity.
• Create and Manage Compute Pools to group your machines and pair them with a GPU-optimized Profile Bundle.