Token Metering

Token Metering tracks AI model token consumption across requests, providing visibility into input, output, and total token usage for monitoring, attribution, and cost analysis. Every inference call that passes through the gateway is metered: requests, tokens, and estimated cost are recorded against the caller's quota so you can monitor consumption, attribute usage to the model that generated it, and cap spend before it happens.

Metering is only the starting point. The same counts that Token Metering records power the enforcement and billing features built on top of it:

• Inference Quotas — Usage budgets that cap how much of an AI resource — requests, tokens, or cost — callers can consume over a rolling window. Defined at System, Tenant, or Project scope, either as the scope-total budget that caps all inference at that scope or as a per-model budget that narrows a specific set of Model Deployments.
• API Keys — Bearer credentials that a client presents to the gateway. Each key draws against exactly one Inference Quota, so its traffic is metered to the model and budget you intended.
• Model Pricing — The install-wide catalog of per-model rates the gateway uses to convert token counts into the estimated Cost figures that quotas and metered usage report. Configured once at the System scope.

The enforcement side of metering — creating quotas, issuing keys, and setting prices — is managed from the PaletteAI console or by applying YAML Kubernetes manifests. For step-by-step instructions, refer to Manage Inference Quotas, Manage API Keys, and Configure Model Pricing.

How Metering Works

PaletteAI enforces token metering through the PaletteAI Inference Launchpad, an inference gateway that runs on a Kubernetes cluster and serves models to callers over an OpenAI-compatible API. The gateway applies authentication and metering on every request: it authenticates the caller's API key, identifies the model deployment that serves the request, records the request against the caller's quota, and reports the token usage after the response completes.

Metering covers three dimensions:

Dimension	What It Measures
Requests	Number of inference API calls received by the gateway.
Tokens	Total input and output tokens consumed by inference calls.
Cost	Estimated cost in USD, derived from per-model prices published to the gateway.

Each dimension is limited over one or more windows. Two windows are available in the UI: Hour and Day. A quota can leave a dimension unset — an unset dimension is not enforced. If any single dimension hits its limit, the quota is exhausted and further usage is blocked until the window resets.

The gateway itself — how models are exposed, how requests are routed, and how metering telemetry is reported — is documented in the PaletteAI Inference Launchpad documentation.

Inference Quotas

An Inference Quota is a usage budget the gateway enforces on inference requests. Each quota caps consumption on one or more of the metering dimensions over one or more rolling windows.

Types of Inference Quotas

There are two types of Inference Quotas. Both live at the same scope; the difference is what they cap.

Type	Description	Use Case
Scope-total	One quota per scope that caps the aggregate usage of every model at that scope. Every request from a model at that scope draws against this quota, including requests already counted by any per-model quota.	A single, scope-wide budget for all inference the scope is allowed to consume.
Per-model	An additional quota that caps only the models that opt in through their AIWorkload.spec.inferenceQuotaRef. Requests to those models draw against the per-model quota and every scope-total quota at or above the model's scope.	A tighter cap on a specific model — for example, an expensive frontier model — that must not consume the scope-total budget on its own.

Only one scope-total quota exists at each scope, created automatically when the scope is set up. You can add any number of per-model quotas.

Scope Hierarchy

Inference Quotas follow the PaletteAI scope hierarchy: System is the root, Tenants sit below System, and Projects sit below Tenants. A parent's scope-total quota clamps its descendants:

• The System scope-total quota caps the total inference across system-scoped Compute Pools and every Tenant.
• Each Tenant scope-total quota caps the total inference across tenant-scoped Compute Pools and the Projects it contains.
• Each Project scope-total quota caps the inference issued from that Project.

Per-model quotas at a lower scope cannot exceed the corresponding limit at a parent scope. When you set a limit in the UI, the form displays the maximum value allowed by the parent so you can size the quota within the ceiling.

Budget Reductions, Ceilings, and Out-of-Policy Descendants

Two fields control hierarchy behavior:

• spec.budget — What this quota may consume. Lowering a scope-total budget automatically clamps descendant budgets that exceed the new value for that dimension and window.
• spec.budgetLimits — Ceilings on what descendant scope-total and per-model quotas may set in their budgets. Lowering a ceiling does not rewrite those budgets; over-limit descendants become Out of policy until you edit them down or raise the ceiling.

Out-of-policy deployments stay running, but the gateway denies inference against them, and editing those deployments is disabled until the quota is back within policy. Configure budgets and ceilings in Manage Inference Quotas.

Deleting a per-model quota removes the additional cap for the models it covered; those models fall back to the scope-total quota. Any API keys attached to a deleted per-model quota are also deleted. The scope-total quota itself is created and managed with its scope and cannot be deleted while the scope exists.

API Keys

An API Key is a bearer credential that a client presents to the gateway to authenticate an inference request. Each key draws against exactly one Inference Quota, so every request the key authorizes counts toward that quota's metered usage.

The API Key Value

The controller generates the key value on your behalf when the API Key is created. Every value begins with the prefix pai- so it can be quickly identified as being issued by PaletteAI.

The value never appears on the API Key custom resource itself. It is stored in a Kubernetes Secret that the controller creates alongside the API Key, and the UI reads the value out of that Secret each time you press the Copy button. Copy the value from the drawer whenever you need it, hand it to the caller through a secure channel, and treat the API Key like any other bearer token. The value cannot be rotated in place; if the current value is compromised, delete the key and issue a new one.

Inference Quota Binding

Each API Key is bound to one Inference Quota at the moment you create it, and the binding is immutable. What the key can call depends on which type of quota you bind it to:

• Attached to a scope-total quota, the key can call any model at that scope.
• Attached to a per-model quota, the key can call only the models covered by that quota — that is, the AIWorkloads in the quota's own namespace that reference it through spec.inferenceQuotaRef.

To move a key to a different quota, delete it and create a new key against the target quota.

OIDC Groups

OIDC groups on an API Key are a PaletteAI control-plane concern. They decide which identity-provider groups may view the key and its secret value in the console (and through the Kubernetes API). Every API Key must list at least one OIDC group. The gateway does not evaluate OIDC groups; it authenticates inference callers with the bearer token only.

Groups accumulate down the scope hierarchy: every group listed on a higher-scoped key is automatically granted the same control-plane access on every lower-scoped key beneath it, on top of that lower-scoped key's own groups. A member of a System-scope key's group can therefore view any Tenant- or Project-scope key without being named on that key's oidcGroups, and a member of a Tenant-scope key's group can view any Project-scope key in that Tenant. Inheritance supplements the required own groups — it does not replace them.

The Create API Key dialog offers a picker of the OIDC groups PaletteAI already knows about at the current scope. If the group you want does not appear in the list, type its name directly into the field. On child scopes, the dialog also shows a read-only Inherited OIDC Groups section listing the groups already granted access by ancestor keys — because those callers already have control-plane access, you do not need to repeat those groups on the new key.

Expiration and Lifecycle States

Each API Key has one of the following expiration settings, chosen when the key is created:

• Never — The key does not expire on its own.
• 7 / 30 / 60 / 90 days — Predefined windows counted from creation. The dialog shows the exact date each option resolves to.
• Custom — A calendar date you pick.

Expiration is enforced by the controller. When the current time passes the key's expiresAt, the controller marks the key Expired and the gateway begins rejecting requests it authorizes. You can update the expiration on an existing key without recreating it.

Each key surfaces one of the following states on the Inference Quotas tab.

State	Meaning
Active	The key is valid and the gateway accepts requests it authorizes.
Expired	The current time is past expiresAt. The gateway rejects requests. The Copy button is disabled for expired keys.
Revoked	The key was administratively revoked. The gateway rejects requests.

Editable Fields

You can edit an existing API Key's OIDC groups and expiration in place. The key's name and the Inference Quota it draws against are set at creation and cannot be changed. To rebind a key to a different quota or rename it, delete it and create a new key.

Request-Time Authentication

The client passes the key value as a bearer token in the Authorization header when it calls the gateway. The exact request shape — endpoint URL, model routing, and response format — is documented in the PaletteAI Inference Launchpad documentation.

Manage Token Metering in the PaletteAI Console

Inference Quotas and API Keys are managed from Settings at each scope, under the Model Management section. Model Pricing is managed only at the System scope, on the Model Pricing tab of the same section.

Scope	Navigation
System	System Settings → Model Management → Inference Quotas
Tenant	Tenant Settings → Model Management → Inference Quotas
Project	Project Settings → Model Management → Inference Quotas
System	System Settings → Model Management → Model Pricing

At every scope, the tab lists the scope-total quota, the per-model quotas defined at that scope, and — at the System and Tenant scopes — a sub-tab that lists descendants (Tenants under System, Projects under Tenant) so you can review their quotas without leaving the page. The API Keys button on each quota opens a drawer listing the keys attached to that quota. Each quota row reports its metered utilization, so you can watch consumption approach the limits you set.

Permissions

• spectrocloud.com/inferencequotas:{create,update,delete} gate the corresponding actions on the Inference Quotas tab.
• spectrocloud.com/inferenceapikeys:{create,update,delete,get} gate the API key actions on the API Keys drawer. The get permission is also required to copy a key's value.

If an action or button described in the documentation is not displayed, your role does not include the required permission. Contact your administrator to request access.

Enable the Gateway on a Cluster

The gateway that meters inference requests is deployed by a Launchpad-enabled Profile Bundle. Any Compute Pool whose Profile Bundle carries the palette.ai/launchpad-enabled=true annotation runs the gateway on its spoke cluster and receives the InferenceQuota and InferenceAPIKey resources PaletteAI federates from the hub. Spectro Cloud's AI Launchpad Profile Bundle already carries this annotation. For custom bundles, refer to Profile Bundles. To bring up a Launchpad-enabled Compute Pool end to end, refer to Deploy an Inference Launchpad Cluster.

Next Steps

• Manage Inference Quotas — Step-by-step instructions to create, update, and delete quotas.
• Manage API Keys — Step-by-step instructions to create, copy, update, and delete API keys.
• Configure Model Pricing — Set the per-model rates that cost metering uses.
• PaletteAI Inference Launchpad documentation — What the gateway does with the quotas and keys PaletteAI provides.