Project Scope

A Project is a workspace where teams deploy and manage AI/ML applications. It is the narrowest scope and where most day-to-day work happens. Every Project belongs to a parent Tenant. Projects provide the following benefits:

• Isolated workspace — Each Project has its own namespace, and resources in one Project cannot reference another Project's namespaces. This keeps changes in one Project from affecting another. For details, refer to the Namespace Isolation section.

• Team access control — Map OIDC groups to viewer, editor, or admin roles within the Project.

• Resource scoping — By default, the resources you create (such as Compute Pools, App and Model Deployments, and Profile Bundles) live in the Project's namespace and are usable only within that Project. Certain resources can also be created at the Tenant scope and shared with Projects. Refer to Tenant-Scoped Resources for more information.

• Target namespace control — Define which namespaces on spoke clusters workloads can be deployed to.

• GPU budgets — Set GPU limits for the Project (must be within Tenant limits). For how Project and Tenant quotas interact, refer to Project-Tenant Quota Interaction.

• Optional Settings override — Use different Palette credentials than the parent Tenant, if needed.

Project Configuration

Projects reference their parent Tenant and define team access. You can create a Project through the UI or declaratively through a Kubernetes manifest. To learn how to create a Project, refer to the Create and Manage Projects guide.

info

Model Management (Model as a Service mappings and model access control lists) is configured after Project creation through the Project Settings page, not during the Project creation wizard.

Example Project manifest

apiVersion: spectrocloud.com/v1alpha1
kind: Project
metadata:
  name: project-a
  namespace: project-a
spec:
  displayName: 'Project A'

  settingsRef:
    name: dev-settings

  # Parent Tenant
  tenantRef:
    name: primary-dev

  # Default compute configuration
  computeConfigRef:
    name: edge-compute-config
    namespace: project-a

  # Team access by OIDC group
  roleMapping:
    viewer:
      - fin-ops
      - auditors
    editor:
      - ml-engineers
      - data-team
    admin:
      - admin
      - sre
    additionalRoles:
      operator:
        - operator

  # GPU quotas for this Project
  gpuResources:
    limits:
      'NVIDIA A100-SXM4-80GB | 80 GB': 64
      'NVIDIA H100 PCIe | 80 GB': 48
      'Default': 24
    requests:
      'NVIDIA A100-SXM4-80GB | 80 GB': 6
      'NVIDIA H100 PCIe | 80 GB': 6
      'Default': 4

  # Target namespace restrictions
  admissionConfiguration:
    workloadConfiguration:
      deploymentNamespaces:
        allow:
          - project-a
          - staging
        disallow:
          - cluster-scoped
          - '*'

Project Roles

Projects provide three core permission levels.

Role	Capabilities
Viewer	Read-only access to all Project resources
Editor	Create and manage App Deployments, Model Deployments, Profile Bundles, and Workload Profiles
Admin	Full access including Project settings and permissions

In addition to the three core roles, Projects support custom RBAC personas for more fine-grained access control. Custom personas can be defined using the roleMapping.additionalRoles field, which maps custom persona tier IDs to OIDC groups. Each custom persona tier ID must match the palette.ai/project-rbac-tier annotation value on a project RBAC template ConfigMap. This allows teams to extend the core roles with additional custom personas tailored to their specific access control requirements. For details on how to configure custom personas, refer to Project RBAC Policy ConfigMaps.

Refer to Roles and Permissions for an overview of each role, or to the Role Permissions reference for the full Kubernetes permission matrix.

Namespace Isolation

Projects can restrict which namespaces on spoke clusters workloads can be deployed to. This provides security boundaries and prevents workloads from deploying to sensitive system namespaces.

Target namespace restrictions are configured through admissionConfiguration.workloadConfiguration.deploymentNamespaces using an allow list and a disallow list. In the PaletteAI UI, these appear as the Allow list and Deny list.

• allow (Allow list) - Namespaces where workloads and their resources can be deployed.

• disallow (Deny list) - Namespaces where workloads and their resources cannot be deployed.

Cluster-Scoped Resources

The special keyword cluster-scoped controls whether workloads can create cluster-scoped resources (resources without a namespace, such as ClusterRoles or CustomResourceDefinitions). Include cluster-scoped in the allow list to permit creation of cluster-scoped resources.

Policy Behavior

Target namespace policies follow these rules:

• If the allow list contains *, all namespaces are allowed except those in the disallow list.

• If the disallow list contains *, all namespaces are blocked except those in the allow list. The disallow list takes precedence over the allow list.

• A namespace cannot appear in both lists. PaletteAI rejects such a configuration at admission.

• If you leave both lists unset, PaletteAI defaults them to allow only the Project's own namespace (allow: [<project-namespace>] and disallow: ['*']). To deploy into the Project's own namespace, keep that namespace in the allow list.

Example target namespace configuration in Project object

spec:
  admissionConfiguration:
    workloadConfiguration:
      deploymentNamespaces:
        allow:
          - project-a # Project namespace
          - staging # Additional namespace
        disallow:
          - cluster-scoped # Deny cluster-scoped resources
          - '*' # Deny all other namespaces

GPU Quotas

A Project's GPU budget operates within its parent Tenant's limits. Set gpuResources.limits and gpuResources.requests on the Project to cap GPU usage across the Project's Compute Pools and the deployments that run on them. The Tenant can constrain the Project further through tenant-level controls.

Quota Types

Field	Available on	Purpose
limits	- Project - Tenant	Maximum total GPU usage per GPU variant within the scope, summed across all of its Compute Pools. Deployments consume from this budget through the Compute Pool they run on.
requests	- Project - Tenant	Maximum GPUs that a single Compute Pool can request per GPU variant.
tenantReservations	- Tenant	Portion of the Tenant limits reserved exclusively for tenant-namespace resources. The remainder (limits - tenantReservations) is the budget shared by all Projects.
projectLimits	- Tenant	Per-Project ceiling that the Tenant admin can impose, keyed by Project name and then by GPU variant. Caps a Project's usage in addition to its own limits.

All quota keys are GPU variant names (for example, NVIDIA A100-SXM4-80GB | 80 GB, NVIDIA H100 PCIe | 80 GB). Use the special key Default to set a per-variant catch-all for any GPU variant not listed explicitly.

Example GPU quota

gpuResources:
  limits:
    'NVIDIA A100-SXM4-80GB | 80 GB': 32 # Up to 32 A100s total within this scope
  requests:
    'NVIDIA A100-SXM4-80GB | 80 GB': 8 # A single Compute Pool can request up to 8 A100s

Default Quotas

Use the Default key to set quotas for GPU variants not explicitly listed. The Default key acts as a per-variant catch-all cap. Each unlisted GPU variant individually gets up to the specified limit.

gpuResources:
  limits:
    'NVIDIA A100-SXM4-80GB | 80 GB': 32
    'Default': 16 # Each unlisted GPU variant is limited to 16 total
  requests:
    'NVIDIA A100-SXM4-80GB | 80 GB': 8
    'Default': 4 # Each unlisted GPU variant is limited to 4 per request

Project-Tenant Quota Interaction

• At admission, when you create or update a Project, its limits.<variant> and requests.<variant> must each be less than or equal to the parent Tenant's limits.<variant> and requests.<variant> for the same variant. If the Tenant only specifies a Default, that catch-all is used.
• When a Compute Pool is created or updated, a single Project's GPU usage for a variant is capped at the lowest of the values that are set (each with a Default fallback): the Project's own limits.<variant>, the Tenant's limits.<variant>, and the Tenant's projectLimits[<this Project>].<variant>.
• The Tenant's limits.<variant> is also enforced at aggregate scope: combined usage across all Projects plus the tenant namespace must not exceed it.
• If a Project does not specify limits for a variant, it adds no ceiling of its own, but the Tenant's limits.<variant> (or Default) still caps that Project individually, and the Tenant's aggregate limits and any projectLimits ceiling continue to bound usage.
• If a variant is constrained by neither the Tenant nor the Project, that is, there is no explicit entry and no Default, PaletteAI does not restrict GPU usage for that variant.

Oversubscription

PaletteAI automatically tracks GPU usage in real time under status.gpuUsage in the respective Tenant and Project Custom Resource Definitions (CRDs). At the Tenant level, status.tenantGpuUsage separately tracks usage by tenant-namespace resources.

GPU tracking at Tenant scope

status:
  gpuUsage:
    project-a:
      'NVIDIA A100-SXM4-80GB | 80 GB': 12
      'NVIDIA H100 PCIe | 80 GB': 8
    project-b:
      'NVIDIA A100-SXM4-80GB | 80 GB': 4
      'NVIDIA H100 PCIe | 80 GB': 4
  tenantGpuUsage:
    'NVIDIA A100-SXM4-80GB | 80 GB': 4

GPU tracking at Project scope

status:
  gpuUsage:
    'NVIDIA A100-SXM4-80GB | 80 GB': 12
    'NVIDIA H100 PCIe | 80 GB': 8

Use the following commands to check GPU usage at the Tenant and Project levels.

kubectl get tenant <tenant-name> --output jsonpath='{.status.gpuUsage}'
kubectl get tenant <tenant-name> --output jsonpath='{.status.tenantGpuUsage}'
kubectl get project <project-name> --namespace <project-namespace> --output jsonpath='{.status.gpuUsage}'

When concurrent requests cause usage to exceed limits, the following sequence occurs.

1. The Tenant TenantNotOversubscribed condition or the Project NotOversubscribed condition becomes False.

2. New GPU requests are blocked until usage drops below limits. App Deployments and Compute Pools requesting GPUs remain in a pending state.

3. After operators delete existing workloads and usage drops below limits, the condition clears and new requests can proceed.

The Tenant status.oversubscribedVariants field records which GPU variants are oversubscribed and at which scope:

• Tenant - Overall tenant usage (Projects + tenant namespace) exceeds limits for the variant. Blocks all new GPU resources for that variant.
• TenantNamespace - Tenant-namespace usage exceeds tenantReservations for the variant. Blocks only new tenant-namespace GPU resources for that variant; Projects can continue consuming GPUs up to their cap.

Example oversubscribed variants

status:
  oversubscribedVariants:
    'NVIDIA A100-SXM4-80GB | 80 GB': Tenant
    'NVIDIA H100 PCIe | 80 GB': TenantNamespace

Use the following commands to check the oversubscription status at the Tenant and Project levels.

kubectl get tenant <tenant-name> --output jsonpath='{.status.conditions[?(@.type=="TenantNotOversubscribed")]}'
kubectl get tenant <tenant-name> --output jsonpath='{.status.oversubscribedVariants}'
kubectl get project <project-name> --namespace <project-namespace> --output jsonpath='{.status.conditions[?(@.type=="NotOversubscribed")]}'

info

Aggregate Tenant limits, tenant-namespace reservations (tenantReservations), and per-Project ceilings (projectLimits) are set by a Tenant admin. For those controls, refer to Tenant Scope GPU Quotas.