Skip to main content
Version: v1.2.x

Configure Tenant Settings

Use Tenant Settings to manage Tenant-scoped configuration in the PaletteAI UI. From this page, you can manage Tenant configuration such as display information, integrations, OpenID Connect (OIDC) group access, target namespace access control, model access control lists, per-Tenant and per-Project GPU limits, Tenant compute inventory, Compute Configs, Scaling Policies, and Model as a Service Mappings.

To modify settings for a single Project at the project scope, refer to Configure Project Settings. For more information on the different scopes, refer to Multi-Tenancy.

Prerequisites​

  • At least one Project exists in the Tenant, and you can access that Project in the PaletteAI UI. Tenant visibility in the UI depends on Project access.

  • To update Tenant-wide settings, you must have the Tenant admin role. For details, refer to Tenant Role Permissions.

  • To edit entries in the Projects sub-tabs, you must have the required Project permissions. For details, refer to Project Role Permissions.

Configure Tenant Settings​

  1. Log in to PaletteAI and select the Tenant you want to manage.

  2. From the Projects Menu in the top-left, select All Projects.

  3. From the left main menu, select Tenant Settings.

Manage Basic Information​

Use the Basic Information tab to update Tenant display values that appear elsewhere in the PaletteAI UI.

  1. Update Tenant Name or Description. To add or remove Annotations and Labels, expand Metadata.

  2. Select Save changes.

Create Tenant Settings Ref​

A Tenant Settings Ref holds integrations for Tenant-scoped resources. Each Tenant has at most one Settings Ref. Create the Settings Ref once, then add or update integrations as your environment grows. Hugging Face and NVIDIA NGC integrations can be shared with Projects; Palette integrations apply only at the Tenant scope and are not shared with Projects.

For background on how System, Tenant, and Project Settings interact, refer to Settings and Integrations.

Enablement​

When no Tenant Settings Ref exists, the Tenant sub-tab shows an empty state titled Tenant-level settings are not available.

  1. On the Settings Ref tab, on the Tenant sub-tab, select Create Tenant-level Settings.

  2. In the Settings Ref Name field, enter a name for the settings resource.

    The name must be 63 characters or fewer and contain only lowercase alphanumeric characters or hyphens, beginning and ending with an alphanumeric character. The name cannot be changed after creation.

  3. Add at least one Palette, Hugging Face, or NVIDIA Inference Microservices (NIMs) integration. The create dialog requires at least one integration before you can save. For field-level guidance per integration type, refer to Add an Integration.

  4. Select Confirm.

After the Tenant Settings Ref is created, the Tenant sub-tab shows the Settings Ref header and the Integrations table. Continue with Manage Tenant Settings Ref to add more integrations or update existing ones.

Manage Tenant Settings Ref​

Add or update the integrations in the Tenant Settings Ref, and review the integrations configured in each Project. The Settings Ref tab opens to the Tenant sub-tab by default. The page header shows Settings Ref followed by the name you assigned, with a Ready or Not Ready status badge. Not Ready indicates that one or more integrations do not have a valid secret.

Add an Integration​

Each integration represents a connection to an external system, such as Palette, Hugging Face, or NVIDIA NIMs. Each integration type can be added once per Tenant Settings Ref. Palette integrations are used for Tenant-scoped operations only and cannot be shared with Projects. Hugging Face and NVIDIA NIMs integrations can be shared with specific Projects in the Tenant and optionally locked so Projects cannot override them.

  1. On the Tenant sub-tab, select Add Integration to open the Add Integration drawer.

  2. Choose an Integration Type — Palette, Hugging Face, or NVIDIA NIMs. The form fields update to match the selected type.

  3. Complete the fields for the chosen integration type:

    • Palette integration — used to provision compute resources from your Palette tenant. Provide:

      • Integration Name — display name, for example, Palette-Acme.
      • Host URL — the Palette host URL, for example, https://console.spectrocloud.com.
      • (Optional) Tenant — the Palette tenant slug, for example, acme. PaletteAI resolves the tenant automatically from the Palette API.
      • Project ID — the default Palette project UID that PaletteAI uses when provisioning clusters.
      • (Optional) Toggle Skip SSL Certificate Verification on to bypass certificate validation when connecting to the Palette server.
      • API Key — the Palette API key. Select Validate to verify the credentials against the Palette server.
      • (Optional) Toggle Cert Secret Ref on to reference a Kubernetes Secret containing a custom certificate authority bundle for the Palette server.

    • Hugging Face integration — enables Hugging Face model management features at the Project level. Provide:

      • Integration Name — display name, for example, HuggingFace-Acme.
      • API Key — your Hugging Face Hub API key. Select Validate to verify the key against the Hugging Face API.

    • NVIDIA NIMs integration — enables NVIDIA NGC model management features at the Project level. Provide:

      • Integration Name — display name, for example, NVIDIA-Acme.
      • API Key — your NVIDIA NGC API key. Select Validate to check that a key is provided. The Validate action does not verify the key format or contact NVIDIA NGC.

  4. (Optional) Expand Metadata to add Kubernetes labels and annotations to the integration.

  5. (Hugging Face and NVIDIA NIMs only; not available for Palette) Configure shared access for the integration:

    • Under Projects, choose which Projects in the Tenant can use this integration.
    • To prevent Projects from configuring their own version of the integration, enable Enforce tenant integration across all projects. Locked integrations cannot be replaced by a Project-level configuration.

    When you enforce the integration and one or more selected Projects already have the same integration type configured, PaletteAI shows a warning. Select Go to impacted projects in the warning to switch to the Projects sub-tab filtered to those Projects.

  6. Select Confirm to add the integration.

Tenant-Level Palette Constraints​

When configuring a Palette integration at the tenant level:

  • Unique Palette tenant: Each PaletteAI tenant must use a unique Palette Tenant. If another PaletteAI tenant already uses the same Palette tenant UID, the newest tenant's Settings is flagged with PaletteIntegrationDuplicated (tenantUID). The oldest tenant's Settings remains healthy.

  • Unique Palette project within tenant: The Palette project configured in tenant Settings cannot also be used by any child project Settings within the same PaletteAI tenant.

  • Child project inheritance: All child projects that configure their own Palette integration must use credentials belonging to the same Palette Tenant as this tenant. If a project uses a different Palette Tenant, the project is flagged with PaletteTenantMismatch — the tenant Settings remains healthy.

tip

If your Settings shows FailedToVerifyUniqueness, the Palette API has not yet resolved the tenant identity. This is usually temporary. Check back after the next reconciliation cycle or verify your Palette API connectivity.

Edit or Delete an Integration​

Use the action menu on each row of the Integrations table to edit or delete an integration.

  • Edit & Share opens the integration drawer so you can update fields, change which Projects the integration is shared with, and toggle the lock setting. To replace the API key for an existing integration, clear the masked value and re-enter the key, and then select Validate before saving.

  • Delete removes the integration. When the integration is used by active model deployments, PaletteAI requires you to confirm by typing the integration name. Affected deployments are marked out of policy until they are updated or deleted.

Palette integrations cannot be deleted from the Tenant Settings Ref. To rotate Palette credentials, use Edit to update the existing entry.

Edit Basic Information​

To update the Tenant Settings Ref's labels and annotations, select the Settings dropdown next to the Settings Ref name, and then select Edit basic information. The Settings Ref name itself cannot be changed after creation.

Review Project Integrations​

Use the Projects sub-tab to review the integrations configured in each Project, including Hugging Face and NVIDIA NGC integrations shared from the Tenant Settings Ref.

  1. Select the Projects sub-tab.

  2. Search or filter to locate the Project. To open or close every Project row at once, select Expand all or Collapse all.

  3. Expand a Project to review its integrations table. Each row shows the Integration Name, Type, Last modified timestamp, Scope badge (Tenant for an integration shared from the Tenant Settings Ref, Project for an integration owned by the Project), and Shared with Projects.

  4. (Optional) Select Edit in the action menu on a Project-scoped row to update a Project integration. Tenant-scoped rows are read-only here; manage them from the Tenant sub-tab.

Manage User Access​

On the User Access tab, control which OIDC groups administer the Tenant and which OIDC groups map to Project Viewer, Editor, and Admin roles.

PaletteAI stores canonical Kubernetes group names in Tenant.spec.tenantRoleMapping.groups, not the raw display names from your identity provider (IdP). Each entry must be scoped to the Tenant's own metadata.name:

  • pai:tenant:<tenant-name>:role:tnt-adm — bare tenant-admin tier
  • pai:tenant:<tenant-name>:role:tnt-adm-<suffix> — tenant-admin tier with a suffix derived from the IdP group name you configured (slugified for DNS compatibility)

When you create or edit Tenant admin groups in the PaletteAI UI, you enter the IdP group name (for example, Okta Admins). PaletteAI persists the matching canonical string (for example, pai:tenant:default:role:tnt-adm-okta-admins) on the Tenant resource. In YAML workflows, list the canonical strings directly under tenantRoleMapping.groups.

Hue binds these canonical strings on tenant-admin ClusterRoleBindings. Kubernetes RBAC grants access only when the effective group on the API request exactly matches a binding subject.

When IdP groups must match the canonical strings

Configure the API server to receive the canonical pai:tenant:… group names—not only the human-readable IdP names—in these cases:

  1. User impersonation is disabled and the hub cluster trusts Dex (or another OIDC issuer) directly. Group claims from the user's token are evaluated by RBAC unchanged.
  2. canvas.impersonationProxy.groupsMode is passthrough (the chart default). Dex group claims are forwarded as-is; groupMap is not applied.
  3. kubectl, GitOps, or other API clients authenticate outside the PaletteAI UI. Impersonation mapping applies only to requests Canvas makes on behalf of a logged-in user.

In these setups, configure Dex or your IdP to emit the canonical strings in the JWT groups claim, or switch to impersonation with groupsMode: map (below). Renaming every IdP group to the long canonical form is rarely practical; groupMap is the supported approach when impersonation is enabled.

When you can keep human-readable IdP group names

If canvas.impersonationProxy.enabled is true and groupsMode is map, Canvas translates IdP group names to Kubernetes group names when it calls the API for a logged-in user. Add one groupMap entry per Tenant admin IdP group. The key must be the exact group string Dex returns in the JWT; the value must be the canonical string on the Tenant:

Example groupMap for tenant admin groups
canvas:
  impersonationProxy:
    enabled: true
    userMode: passthrough
    groupsMode: map
    groupMap:
      'Okta Admins':
        - pai:tenant:default:role:tnt-adm-okta-admins
      'Platform SRE':
        - pai:tenant:primary-dev:role:tnt-adm-platform-sre

Obtain each canonical value from kubectl get tenant <tenant-name> -o jsonpath='{.spec.tenantRoleMapping.groups}' or from the User Access tab after you save Tenant admin groups (the UI shows the persisted canonical strings). Every Tenant admin group you add through the UI needs a matching groupMap entry unless the IdP already emits the canonical name.

For local Dex users, assign Kubernetes groups with dexGroupMap (keyed by user email) instead of groupMap. dexGroupMap requires groupsMode: map.

After you change groupMap or dexGroupMap, apply the update through helm upgrade or the canvas-config secret so Canvas reloads the configuration. Confirm effective groups on the PaletteAI whoami page (/ai/whoami).

Important: groupMap affects only Kubernetes API requests made through Canvas. It does not change Dex JWT tokens and does not apply to direct kubectl OIDC login unless you configure equivalent mapping for that client.

Project roleMapping groups

Project Viewer, Editor, and Admin groups (Project.spec.roleMapping) are separate from tenant admin groups. They are not rewritten to the pai:tenant:… format. Map them through groupMap only when you use impersonation and the IdP names differ from the strings stored on the Project.

For step-by-step impersonation setup, refer to Configure User Impersonation. For access diagnostics, refer to Troubleshooting Tenants.

  1. On the Tenant sub-tab, add or remove entries in Tenant Admin Groups using the IdP group name (for example, Okta Admins), and then select Save changes.

    After you save, the inputs show the canonical pai:tenant:… strings persisted on the Tenant. Blank entries are ignored. Configure at least one IdP group before you save changes.

  2. Select the Projects sub-tab to review each Project's OIDC Viewer, Editor, and Admin group assignments.

  3. Search or filter to locate the Project, and then select Edit to update that Project's OIDC group assignments.

    The edit drawer updates the Viewer Groups, Editor Groups, and Admin Groups fields for the selected Project.

Manage Target Namespaces​

On the Target Namespaces tab, configure which Kubernetes namespaces on spoke clusters are permitted deployment targets for workloads. Refer to Namespace Isolation for how the policy behaves.

  1. On the Tenant sub-tab, add or remove namespace entries in the Allow list (namespaces where workloads and their resources can be deployed) and Deny list (namespaces where workloads and their resources cannot be deployed), and then select Save changes.

    Alternatively, enable Allow all (*) to permit all namespaces by default and deny specific ones, or Deny all (*) to block all namespaces by default and allow specific ones.

  2. Select the Projects sub-tab to review and edit each Project's target namespace configuration.

Manage Models List​

On the Models List tab, control which models from the configured Hugging Face and NVIDIA NGC integrations are available for Model Deployments in the Tenant and its Projects. Allowed models appear in the model selection step of the Model Deployment workflow; denied models do not.

  1. On the Tenant sub-tab, configure the access control list for each model source by adding allowed or denied entries, and then save your changes.

  2. Select the Projects sub-tab to review each Project's model access control lists, including Tenant configuration shared with the Project.

Manage GPU Partitioning​

On the GPU Partitioning (MIG) tab, control whether Multi-Instance GPU (MIG) options are displayed in deployment and Compute Pool workflows:

  • Enable Multi-Instance GPU for Tenant-scoped deployments — Show MIG options for workloads deployed at the Tenant scope.

  • Enable Multi-Instance GPU for all Projects in this Tenant — Show MIG options for every Project, or turn the toggle off to select individual Projects.

Save changes when finished. These toggles control only whether MIG options appear in the console; they do not change the MIG configuration of existing Compute Pools.

Manage Resource Limits​

On the Resource Limits tab, set GPU limits and requests for the Tenant, and review or adjust per-Project GPU quotas.

  1. On the Tenant sub-tab, update GPU Limits and GPU Requests, and then select Save changes.

    Each entry must use a unique GPU variant and a limit value. If a warning indicates that Projects exceed the Tenant GPU limits, increase the Tenant limits or reduce Project limits.

  2. (Optional) In the Tenant GPU reservations section, use Add reservation to reserve GPUs for Tenant-level Compute Pools and workloads. Each reservation value must be at most the Tenant limit for that GPU variant.

  3. Select the Projects sub-tab to review GPU limits and requests across Projects.

  4. Search or filter to locate the Project, and then select Edit to adjust GPU limits and requests.

Manage Compute​

On the Compute tab, review and configure worker and control plane nodes that the Tenant's Projects have discovered through their Palette integrations. The tab opens to the Tenant sub-tab by default. Configure default NICs and per-host static IPs before creating Compute Pools. For project-scoped configuration, refer to Manage Compute in Project Settings.

  1. On the Tenant sub-tab, review every node from all Compute resources in the Tenant's own namespace.

    Project compute nodes are displayed separately on the Projects sub-tab. The table includes Edge Host, Status (In use, Free, Unhealthy, Unavailable, or Unknown), Type (Worker or Control Plane), NIC (the node's default NIC), Architecture, GPU Variant, GPU count, Total GPU Memory, CPU, Memory, Compute Pool, and Resource Groups. On the Projects sub-tab, a Project column also appears.

  2. Filter the list with Filter by name, or with the Status, Compute Pool, Type, and GPU Variant selectors. The Projects sub-tab also offers a Project filter.

  3. When some nodes are missing a default NIC (Network Interface Card), a warning banner appears with the message: "Some nodes are missing a default NIC used for cluster network routing. Nodes without one won't be selected for workload placement. Default NICs are auto-filled when there's a single option. For multi-NIC nodes, select a default NIC below."

    • Select Resolve unassigned NICs from the banner to assign default NICs to multiple nodes at once.

    • Select Edit default NIC from the action menu on individual node rows to change the default NIC and optionally configure a static IP address.

  4. To configure a default NIC for a node, select Edit default NIC from the action menu, choose the NIC from the dropdown, and optionally enable Static IP to provide IP Address, Default gateway, Subnet mask, and DNS Server values. Changing the default NIC will disrupt networking on the node and cause connectivity loss for running workloads while the change is applied.

  5. Select the Projects sub-tab to review compute nodes grouped by their owning Project.

Manage Multi-Instance GPU​

Govern where Multi-Instance GPU (MIG) configuration appears in the user interface. Tenant decisions apply to the Tenant and its Projects, and a Project cannot override the Tenant's MIG governance. The System scope sits above the Tenant: a System administrator can enable or disable MIG across all Tenants and Projects, and that decision takes precedence over Tenant settings. These controls affect only where MIG configuration is exposed; they do not change how the NVIDIA GPU Operator applies MIG.

  1. Open the Tenant's MIG settings.

  2. Enable MIG for Tenant-scoped Compute Pool flows to expose MIG configuration on Compute Pools that the Tenant owns. When disabled, the Tenant-scoped flows hide MIG configuration.

  3. Choose which Projects may configure MIG in their Project-scoped Compute Pool flows.

    • Allow all Projects to expose MIG configuration. This is the default.
    • Restrict MIG configuration to a specific list of Projects.
    • Allow no Projects to expose MIG configuration.

  4. Save your changes.

info

MIG configuration also depends on the hardware. Even when a Tenant allows MIG for a Project, PaletteAI offers MIG configurations only for Compute Pools whose GPUs support MIG. Refer to Multi-Instance GPU for the hardware requirements.

Create Compute Config​

Create a Tenant-scoped Compute Config that Projects can adopt as a shared infrastructure baseline. Each Compute Config captures cluster defaults such as Secure Shell (SSH) keys, Network Time Protocol (NTP) servers, and the network overlay.

Enablement​

  1. On the Compute Config tab, on the Tenant sub-tab, select Create Compute Config to open the Create tenant compute config wizard.

  2. Complete the wizard steps:

    • Basic Information — set the Compute Config Name and Deletion Policy, optionally add SSH Keys, and pick the Projects that can use this Compute Config under Shared access.
    • Edge Configuration — configure NTP servers and the network overlay.
    • Node Pool Defaults — configure the control plane pool and worker pool default settings, such as node count, architecture, CPU, memory, labels, annotations, taints, single-node cluster mode, and worker-node eligibility.
    • Review — confirm the configuration and select Create.

    When the Tenant has at least one Project, you must share the Compute Config with at least one Project under Shared access.

Manage Compute Configs​

Review and adjust the Tenant Compute Configs and inspect Project-scoped Compute Configs. The Compute Config tab opens to the Tenant sub-tab by default.

  1. On the Tenant sub-tab, review the Tenant Compute Configs.

    The table includes the Compute Config Name, the virtual IP (VIP), Control plane nodes, Deployment Type, Shared with, and an action menu.

  2. Edit, clone, or delete a Compute Config from the action menu on each row.

    Cloning populates the wizard from the source. Deletion is blocked while a Compute Config is in use by a Project.

  3. Select the Projects sub-tab to inspect Compute Configs scoped to each Project.

    Search by name or filter by Project, then expand a Project row to review its Project-scoped Compute Configs together with any Tenant Compute Configs shared into the Project. The Scope column distinguishes Tenant from Project sources.

Create Scaling Policy​

Create a Tenant Scaling Policy that Projects can reference from their Compute Pools to autoscale workloads.

For background on autoscaling and the YAML reference, refer to Create and Manage Scaling Policies.

Enablement​

  1. On the Autoscaling Policies tab, on the Tenant sub-tab, select Create Autoscaling Policy at the top of the table to open the Create Auto-Scaling Policy dialog.

  2. Provide the following:

    • Scaling Policy Name — a unique Kubernetes-compliant name.
    • Metrics — add one or more CPU or GPU metrics with scale-up and scale-down thresholds and durations. At least one CPU or GPU metric is required.
    • Resource Bounds — optional minimum and maximum CPU counts, plus minimum and maximum GPU counts per variant. Each GPU variant in Resource Bounds must have a matching entry in Metrics.
    • Cool Down Duration — minimum wait time between scaling actions. Defaults to 15 minutes.
    • Timeout Duration — maximum duration for an ongoing scaling operation. Defaults to 30 minutes.
    • Shared access — under Projects, choose which Projects can use this policy.

  3. Select Create to save the policy.

Manage Scaling Policies​

Review the Tenant Scaling Policies, inspect their adoption across Compute Pools, and edit, clone, or delete existing policies. The Autoscaling Policies tab opens to the Tenant sub-tab by default.

  1. On the Tenant sub-tab, review the Tenant Scaling Policies.

    The table includes Scaling Policy Name, Shared with, Metrics, In use Compute Pools, Cool Down, Timeout, Scope, and an action menu. System policies appear with a system Scope badge and only support Clone.

  2. Select the In use Compute Pools count to open a drawer that lists the Compute Pools referencing the policy along with each pool's owning Project.

  3. Edit, clone, or delete a policy from the action menu on each row.

    Editing modifies the policy in place. Cloning opens the dialog populated from the source policy. Deletion is disabled while the policy is referenced by any Compute Pool.

  4. Select the Projects sub-tab to inspect Scaling Policies scoped to each Project.

    Search by name or filter by Project, then expand a Project row to view its Project-scoped Scaling Policies. Edit, clone, or delete a Project policy from the action menu on each row.

Create Model as a Service Mappings​

Create a mapping on the Model as a Service Mappings tab to route incoming Model Deployments to a chosen Profile Bundle based on the model's source and attributes. When a Model Deployment matches the source and filters defined for a mapping, PaletteAI deploys the model using the target Profile Bundle. The tab has Tenant and Projects sub-tabs; you create new mappings on the Tenant sub-tab, while the Projects sub-tab lets you review, edit, or delete existing Project-level mappings.

Enablement​

  1. On the Model as a Service Mappings tab, on the Tenant sub-tab, select Add Mapping to open the Add Mapping drawer. The drawer has two columns.

  2. In the Model Source column on the left, choose Hugging Face or NVIDIA NIMs as the source type, then define the Model Match filter. Each mapping supports one filter, which is a key-value pair that a model must match for the mapping to apply.

    For Hugging Face, the filter key is apps with values vllm or ollama. For NVIDIA NIMs, the filter is fixed to nims = true and cannot be edited.

  3. In the Target Profile Bundle column on the right, pick the Profile Bundle that PaletteAI uses when a model matches the filter.

    The table lists each Profile Bundle's Name, Version, and Type. Only Application and Fullstack Profile Bundles are eligible; Infrastructure bundles do not appear.

  4. Select Confirm to save the mapping, or Cancel to discard it. Confirmed mappings are saved to the Tenant automatically.

Manage Model as a Service Mappings​

Review existing mappings, and edit or delete them. Changes are saved to the Tenant automatically.

  1. On the Tenant sub-tab, review the mappings table.

    The table includes Target Profile Bundle, Source Type, Model Filters, and an action menu. Filter the list by Source Type or use Filter by name to search.

  2. Edit or delete an existing mapping from the action menu on each row.

  3. Select the Projects sub-tab to review Project-level mappings. You can edit or delete existing Project mappings from the action menu, but you cannot create new mappings on this sub-tab.

Validate​

  1. Refresh the Tenant Settings page and confirm your saved changes persist on the applicable Tenant or Projects sub-tab.

Next Steps​