7 docs tagged with "tasks"

This guide describes how to configure model pricing at the System scope. Model pricing is the install-wide catalog of per-model rates (USD per 1,000,000 tokens) that the Inference Launchpad gateway uses for cost accounting and that the Cost dimension of Inference Quotas is measured against. PaletteAI federates the catalog to every Launchpad-enabled spoke; each served model that appears in the catalog gets an explicit rate, and every other served model falls back to the Default rate. Without a Default and an explicit price for a model, that model's requests do not contribute to the Cost dimension of any quota.

Settings define integrations and configuration values used by Projects and Compute Pools. For integration types, prerequisites, and examples, refer to Settings and Integrations.

This guide describes how to deploy an Inference Launchpad cluster from PaletteAI. An Inference Launchpad cluster is a standard Compute Pool whose Profile Bundle opts the spoke in to PaletteAI Inference Launchpad by carrying the palette.ai/launchpad-enabled=true annotation. Once you have such a Compute Pool, deploying a vLLM-backed Model Deployment to it is the same workflow as any other Model Deployment — the Inference Launchpad gateway on the spoke automatically proxies the model.

This page covers how to reference outputs and user variables using PaletteAI's macro system in workload profiles. The different source types for macros include:

This guide describes how to create, copy, edit, revoke, and delete API Keys for PaletteAI Inference Launchpad. An API Key is a bearer credential that a client presents to the Inference Launchpad gateway and that draws against exactly one Inference Quota.

This guide describes how to create, edit, and delete Inference Quotas. An Inference Quota is a usage budget that the PaletteAI Inference Launchpad gateway enforces on inference requests. Each quota caps the volume a caller can consume over a rolling window, measured in Requests, Tokens, or Cost.

This section describes how to manage PaletteAI Inference Launchpad from PaletteAI. Inference Launchpad can also run standalone; the tasks in this section apply only when you use PaletteAI as its central control plane. When you do, PaletteAI is the single place where clusters are provisioned, usage budgets are defined, credentials are issued, and model prices are catalogued, and the hub federates each of those to every Launchpad-enabled Compute Pool.