Skip to main content
Version: v1.3.x

Deploy an Inference Launchpad Cluster

This guide describes how to deploy an Inference Launchpad cluster from PaletteAI. An Inference Launchpad cluster is a standard Compute Pool whose Profile Bundle opts the spoke in to PaletteAI Inference Launchpad by carrying the palette.ai/launchpad-enabled=true annotation. Once you have such a Compute Pool, deploying a vLLM-backed Model Deployment to it is the same workflow as any other Model Deployment — the Inference Launchpad gateway on the spoke automatically proxies the model.

The Inference Launchpad-specific inputs are your choice of Profile Bundle and two networking variables that bundle exposes: modelRootDomain (the wildcard root domain the gateway serves models under, for example *.example.com) and launchpadDomain (the fully qualified hostname of the gateway itself, for example launchpad.example.com). Together they form the URLs the gateway publishes. This guide points you at the standard PaletteAI Compute Pool and Model Deployment flows and calls out where the Inference Launchpad inputs fit.

Prerequisites​

  • A Launchpad-enabled Profile Bundle available at the scope you plan to deploy in. To obtain a PaletteAI Inference Launchpad Profile Bundle, contact your Spectro Cloud representative.

  • A content bundle carrying the required packs imported to an OCI Pack registry that is accessible by the Palette instance defined in the scope's integration. The packs are not available in the public Spectro Cloud registry and require an additional license to access.

  • The values you plan to enter for the two Inference Launchpad networking variables — the wildcard root domain the gateway will serve models under (modelRootDomain) and the fully qualified hostname of the gateway itself (launchpadDomain). Both must be resolvable, and the root domain must be set up as a wildcard DNS record.

  • Wildcard TLS materials for the root domain — a full-chain certificate and its private key — which the Profile Bundle asks for at the Variables step.

  • The prerequisites for Create and Manage Compute Pools and for Create and Manage Model Deployments. Inference Launchpad does not add new permissions or infrastructure requirements to those flows.

Confirm the Profile Bundle Is Launchpad-Enabled​

Before you create the Compute Pool, verify that the Profile Bundle you plan to reference carries the Launchpad-enabled annotation. Without it, the gateway is not deployed to the spoke.

  1. From the left main menu, select Profile Bundles.

  2. Open the Profile Bundle you plan to use.

  3. Expand the Metadata section and confirm that the Annotations list includes palette.ai/launchpad-enabled: true.

If the annotation is absent on a custom Profile Bundle, edit the bundle and add it under Annotations.

Create the Compute Pool​

Create a Compute Pool the standard way. The Inference Launchpad-specific choices are:

  • At the Profile Bundle step, select the Launchpad-enabled bundle you confirmed above.
  • At the Variables step, set the Inference Launchpad networking variables:
    • modelRootDomain — The wildcard root domain the gateway will serve models under (for example, *.example.com). Must resolve as a wildcard DNS record.
    • launchpadDomain — The fully qualified hostname of the gateway itself (for example, launchpad.example.com). This is the OpenAI-compatible API endpoint clients call.
    • tlsCert and tlsKey — The full-chain wildcard TLS certificate and its private key, matching modelRootDomain.

Every other field — Node config, Resource groups, Autoscaling — is unchanged. For the full walkthrough, refer to Create and Manage Compute Pools.

When the Compute Pool reaches Running status, the Inference Launchpad gateway is installed on the spoke cluster and ready to serve models.

Deploy a Model to the Compute Pool​

Deploy a vLLM-backed Model Deployment to the Compute Pool the standard way. The Inference Launchpad gateway automatically proxies vLLM-backed Model Deployments on a Launchpad-enabled Compute Pool; you do not need to attach an Inference Quota, set gateway routing fields, or add anything else to the Model Deployment for it to be served through the gateway.

For the full walkthrough, refer to Create and Manage Model Deployments. Select a vLLM-based Workload Profile — such as the one shipped in the AI Launchpad Profile Bundle — so the gateway can integrate with the model.

(Optional) To enable prefix caching, set the vllmConfEnablePrefixCaching Workload Profile variable to true on the Model Deployment. Refer to Configure Model Pricing.

Validate​

  1. Open the Model Deployment's Overview page. When the gateway is serving the model, the page displays the Inference Launchpad URL and, for chat-capable models, a chat launcher.

  2. Issue an API key at the target scope and call the Inference Launchpad URL with the key as a bearer token in the Authorization header. Refer to the PaletteAI Inference Launchpad documentation for the request format.

Next Steps​