Configure Model Pricing

This guide describes how to configure model pricing at the System scope. Model pricing is the install-wide catalog of per-model rates (USD per 1,000,000 tokens) that the Inference Launchpad gateway uses for cost accounting and that the Cost dimension of Inference Quotas is measured against. PaletteAI federates the catalog to every Launchpad-enabled spoke; each served model that appears in the catalog gets an explicit rate, and every other served model falls back to the Default rate. Without a Default and an explicit price for a model, that model's requests do not contribute to the Cost dimension of any quota.

Only System administrators can edit model pricing. The catalog lives on the cluster-scoped System resource at spec.modelPricing.

You can configure model pricing from the PaletteAI console or by applying YAML.

Prerequisites

UI Workflow

• Platform operator access — the System scope is not visible without it.

• The spectrocloud.com/systems:update permission for editing the pricing catalog. If a button described in this guide is not displayed, your role does not include the required permission.

• The per-model rates you plan to enter, expressed in USD per 1,000,000 tokens. Rates come from the model provider's published pricing page or your negotiated contract.

YAML Workflow

• Access to the PaletteAI hub cluster with permissions to read and update the cluster-scoped System resource.

• The per-model rates you plan to enter, expressed in USD per 1,000,000 tokens.

Rate Fields

Every rate — the Default and each per-model entry — carries the same four fields, all expressed in USD per 1,000,000 tokens. Input and Output are independent per-dimension rates. Cache read and Cache write are only ever billed when the backend reports cache tokens, and their zero or unset value falls back to Input rather than disabling the dimension.

The vLLM backend reports cache tokens only when prefix caching is enabled at deploy time. For the AI Launchpad Profile Bundle, set the vllmConfEnablePrefixCaching Workload Profile variable to true on the Model Deployment. When prefix caching is off, cache read and cache write rates never contribute to the Cost dimension regardless of what you enter here.

Field	Wire field	What It Meters
Input	in	Fresh input tokens sent to the model. Zero or unset means the dimension is billed at zero.
Output	out	Output tokens returned by the model. Zero or unset means the dimension is billed at zero.
Cache read	cachedIn	Prompt-cache-read tokens. Zero or unset falls back to the Input rate.
Cache write	cacheWrite	Prompt-cache-write tokens. Zero or unset falls back to the Input rate.

To bill cache tokens at a different rate than fresh input, enter a non-zero value for Cache read or Cache write. To bill cache tokens at zero, set Input to zero (this also zeroes fresh-input billing); there is no way to zero a cache dimension without also zeroing Input.

Edit the Default Rate

The Default rate applies to any served model absent from the per-model catalog. PaletteAI materializes it into an explicit rate at federation time, so it governs pricing for every otherwise-unpriced model on every Inference Launchpad spoke.

UI Workflow

1. Log in to the PaletteAI console.

2. From the Projects Menu, select All Projects. From the left main menu, select System Settings.

3. Select the Model Management section, and then select the Model Pricing tab.

4. In the Default Pricing section at the top of the page, select Edit.

5. Enter the Input, Output, Cache read, and Cache write rates. Leaving Input or Output empty bills that dimension at zero; leaving Cache read or Cache write empty applies the Input rate to cache tokens when the backend reports them.

6. Select Save.

YAML Workflow

Edit spec.modelPricing.default on the System resource. The System resource is a cluster-scoped singleton whose name matches the PaletteAI instance name (the global.instanceName Helm value).

Example System modelPricing

apiVersion: spectrocloud.com/v1alpha1
kind: System
metadata:
  name: <instance-name>
spec:
  modelPricing:
    default:
      in: 0.50
      out: 1.50

Apply the manifest:

Example command

kubectl apply --filename system.yaml

Add or Edit a Per-Model Rate

Per-model rates override the Default for the models they name. The Inference Launchpad gateway matches a request's model value to the catalog exactly, so each entry's model name must match the served-model name your Model Deployment exposes.

UI Workflow

1. On the Model Pricing tab, in the Model rates section, select Add Prices to open the drawer.

2. Filter or search for the model or models you want to price and select them.

3. Enter the Input, Output, Cache read, and Cache write rates for each selected model.

4. Select Save.

To edit an existing per-model rate, open the three-dot menu on the row and select Edit. To edit several rows at once, select their checkboxes and use the Bulk Edit action from the selection bar.

YAML Workflow

Add or update entries under spec.modelPricing.prices:

Example System modelPricing with per-model entries

apiVersion: spectrocloud.com/v1alpha1
kind: System
metadata:
  name: <instance-name>
spec:
  modelPricing:
    default:
      in: 0.50
      out: 1.50
    prices:
      - model: meta-llama/Llama-3.1-8B-Instruct
        in: 0.20
        out: 0.60
      - model: mistralai/Mistral-7B-Instruct-v0.3
        in: 0.15
        out: 0.45
        cachedIn: 0.05

The model value must match the model name the gateway sees on the request exactly — an empty or mismatched value never matches, and the served model falls back to the Default rate. Apply the manifest with kubectl apply.

Delete a Per-Model Rate

Deleting a per-model entry causes that model to fall back to the Default rate on the next federation cycle. The catalog itself, and other entries, are unaffected.

UI Workflow

1. On the Model Pricing tab, in the row for the model you want to delete, open the three-dot menu and select Delete.

2. To delete several rows at once, select their checkboxes and use the Delete action from the selection bar.

3. Confirm the deletion.

YAML Workflow

Remove the entry from spec.modelPricing.prices and re-apply the System manifest, or use kubectl edit:

Example command

kubectl edit system <instance-name>

Validate

Confirm the catalog is being federated and applied:

UI Workflow

1. On the Model Pricing tab, verify the Default and per-model rows show the values you set.

2. Issue an inference call through the Inference Launchpad gateway for a model covered by the catalog. Refer to the PaletteAI Inference Launchpad documentation for the request format.

3. Open an Inference Quota with a Cost limit and confirm the utilization on the Cost dimension advances after the call.

YAML Workflow

Confirm the catalog is set on the System resource:

Example command

kubectl get system <instance-name> --output jsonpath='{.spec.modelPricing}'

Then issue an inference call and confirm the observed usage on an Inference Quota's Cost dimension advances:

Example command

kubectl get inferencequota <quota-name> --namespace <namespace> \
  --output jsonpath='{.status.used.cost}'

Next Steps

• Manage Inference Quotas — Cap the Cost dimension the model catalog feeds.
• Manage API Keys — Issue credentials that draw against those quotas.
• PaletteAI Inference Launchpad — Concept overview and how cost accounting fits into the gateway.