Token Metering

This section describes how to manage Token Metering from PaletteAI: creating and editing the Inference Quotas that cap metered usage, issuing the API Keys that attribute each caller's traffic to a quota, and configuring the Model Pricing that converts token counts into cost figures.

Token Metering tracks AI model token consumption across requests, providing visibility into input, output, and total token usage for monitoring, attribution, and cost analysis. Metering data powers the enforcement features built on top of it — usage budgets, credentials, and pricing.

PaletteAI enforces token metering through the PaletteAI Inference Launchpad, an inference gateway that runs on a Kubernetes cluster and serves models over an OpenAI-compatible API. Launchpad can also run standalone; the tasks in this section apply when you use PaletteAI as its central control plane. When you do, PaletteAI is the single place where clusters are provisioned, usage budgets are defined, credentials are issued, and model prices are catalogued, and the hub federates each of those to every Launchpad-enabled Compute Pool.

For the full metering model — quota types, the scope hierarchy, and how metered usage maps to keys and budgets — refer to Token Metering. For the gateway behavior itself, refer to the PaletteAI Inference Launchpad documentation.

Deploy a Launchpad-Enabled Cluster

A Launchpad-enabled cluster is a standard Compute Pool whose Profile Bundle opts the spoke in to Inference Launchpad. Once you have such a Compute Pool, deploying a vLLM-backed model to it is the same as any other Model Deployment — the gateway automatically proxies it and meters every request. To bring up a Launchpad-enabled cluster end to end, including the Profile Bundle and networking inputs the bundle asks for, refer to Deploy an Inference Launchpad Cluster.

Manage Inference Quotas

An Inference Quota is a usage budget the gateway enforces on inference requests. Quotas live at the System, Tenant, or Project scope, either as the scope-total budget that caps all metered usage at that scope or as a per-model budget that narrows a specific set of Model Deployments. To create a new per-model quota, edit the limits on an existing quota, or delete a quota, refer to Manage Inference Quotas.

Manage API Keys

An API Key is a bearer credential that a client presents to the gateway to authenticate an inference request. Each key draws against exactly one Inference Quota, so every request it authorizes is metered against that quota. OIDC groups control who can view the key and its secret in the PaletteAI control plane; the gateway authenticates the bearer token only. To issue a new key, copy its value out for distribution, edit its OIDC groups or expiration, revoke it, or delete it, refer to Manage API Keys.

Configure Model Pricing

Model pricing is the install-wide catalog of per-model rates (USD per 1,000,000 tokens) the gateway uses to turn metered token counts into the estimated Cost figures that Inference Quotas are measured against. It is a System-admin-only task and is set once at the System scope. To edit the default rate, add or update per-model rates, or delete a rate, refer to Configure Model Pricing.