Skip to main content

Kubernetes Installation

This page guides you through the process of installing PaletteAI on any Kubernetes cluster, whether it is managed by cloud providers (EKS, GKE, AKS) or self-managed in on-premises or edge environments. The deployment method covered uses the hub-as-spoke pattern, which allows the hub cluster to also act as a spoke cluster, and deploys Zot for the Open Container Initiative (OCI) registry. By acting as a spoke cluster, AI/ML applications can be deployed directly on the hub cluster. To learn more about hub and spoke clusters, refer to our Hub-Spoke Model guide.

Prerequisites​

To install PaletteAI, you must have an existing Kubernetes cluster and the ability to install the Mural Helm chart, which is hosted publicly via AWS ECR. An external OCI registry is recommended for production deployments but is not required.

  • The following requirements apply to the machine where you are preparing the Helm chart values and initiating the installation from:

    • Helm version >= 3.17

    • Kubectl version >= 1.31

    • curl or wget to download the Helm chart values file

    • Network access to the existing Kubernetes API server for the cluster on which PaletteAI will be installed

  • The following requirements apply to your existing Kubernetes cluster on which PaletteAI will be installed:

    • Cluster admin rights

    • Kubernetes version >= 1.32.0

    • The following resources available:

      • 3388m CPU

      • 2732 Mi Memory

      • 10Gi Storage

    • The Kubernetes API server must trust Dex as an identity provider. Dex is deployed as a part of the PaletteAI installation. Learn more about configuring the Kubernetes API server to trust Dex by reviewing the Configure Kubernetes API Server to Trust OIDC Provider guide.

    • A load balancer implemented for Kubernetes Ingress resources, such as:

      • Cloud provider load balancers (AWS ALB/NLB, GCP Load Balancer, Azure Load Balancer)

      • On-premises solutions (MetalLB, HAProxy, F5)

      • Other load balancer controllers compatible with Kubernetes Ingress

Installation​

  1. Download the latest Helm chart values file. This example uses curl.

    curl --output values.yaml --silent https://docs.palette-ai.com/resources/assets/hosted/helm/values.yaml

  2. Open the Helm chart values file in a text editor of your choice and complete the following sections. This example uses vi.

    vi values.yaml

  1. Determine if you need to disable any of the open-source components, such as Ingress Nginx. PaletteAI requires these components, but if your cluster comes with one of the components already installed, you may use your existing instance. For example, some Kubernetes management platforms include Ingress Nginx with every cluster. In this case, disable Ingress Nginx by setting ingress-nginx.enabled: false so that PaletteAI can use the existing instance.

global​

  1. The global configuration is used to configure overarching settings for the PaletteAI deployment. Review and modify the following values as necessary.

    1. Set global.dns.domain to the primary domain for the deployment. Do not include a protocol. For example, use example.org, not https://example.org.

      global:
        dns:
          domain: 'example.acme.org'

    2. If you are not planning on using the ingress-nginx controller, set global.dns.rootIngress.enabled to false.

      global:
        dns:
          rootIngress:
            enabled: false

    3. In global.auditLogging.basicAuth, change the default username and password for audit logging. The session secret is used for encoding and decoding the PaletteAI session cookie. Credentials are not stored in the browser. The cookie is used to map the session to the user so that the server can retrieve the user's credentials.

      global:
        auditLogging:
          basicAuth:
            username: REPLACE_WITH_YOUR_USERNAME
            password: REPLACE_WITH_YOUR_PASSWORD
      Complete global configuration section

alertmanager​

  1. Navigate to the alertmanager section. Update credentials for the alertmanager instance based on the credentials you configured in the global section.

    You must provide a Base64 encoded string for the Authorization header. Use the interactive encoder below to generate your Base64 encoded string and copy the value to the clipboard.

    Base64 Encoded String:

    Alternatively, generate the Base64 encoded string using the following command. Replace username and password with the username and password you configured in the global section.

    echo -n "username:password" | base64

    Following is the livenessProbe and readinessProbe sections with the Base64 encoded string. Replace REPLACE_WITH_YOUR_BASE64_ENCODED_STRING with the Base64 encoded string you generated.

    alertmanager:
      livenessProbe:
        httpGet:
          path: /-/healthy
          port: http
          scheme: HTTPS
          httpHeaders:
            - name: Authorization
              value: 'Basic REPLACE_WITH_YOUR_BASE64_ENCODED_STRING'
      readinessProbe:
        httpGet:
          path: /-/ready
          port: http
          scheme: HTTPS
          httpHeaders:
            - name: Authorization
              value: 'Basic REPLACE_WITH_YOUR_BASE64_ENCODED_STRING'
    Complete alertmanager configuration section

dex​

  1. Dex authenticates users to PaletteAI through SSO. You can configure Dex to connect to an upstream OIDC provider or to a local user database. For this installation, you will configure Dex to connect to an upstream OIDC provider. If you want to configure an OIDC provider later, you can do so; however, Dex still requires some basic configuration.

    a. Set dex.config.issuer to your domain. Do not remove the /dex path.

    dex:
      config:
        issuer: 'https://replace.with.your.domain/dex'

    b. This next part may be deferred for later, but we strongly recommend configuring at least one connector. Set the dex.config.connectors to the connectors you want to use. The Dex documentation has examples for each of the connectors.

    Below is an example of an OIDC connector that connects to AWS Cognito. The oidc type can be used for any OIDC provider that does not have a native Dex connector. Different OIDC providers may require different configurations.

    Example AWS Cognito configuration
    dex:
      config:
        connectors:
          - type: oidc
            id: aws
            name: AWS Cognito
            config:
              issuer: https://cognito-idp.us-east-1.amazonaws.com/us-east-1_xxxxxx
              clientID: xxxxxxxxxxxxxxx
              clientSecret: xxxxxxxxxxxxxxxxx
              redirectURI: https://replace.with.your.domain/dex/callback # Dex's callback url for authorized code flow that will redirect to our application's callback url
              getUserInfo: true
              userNameKey: email
              insecureSkipEmailVerified: true
              insecureEnableGroups: true
              scopes:
                - openid
                - email
                - profile
              promptType: consent
              claimMapping:
                groups: groups

    c. Once you have configured the connectors, proceed to the dex.config.staticClients section. Replace REPLACE_WITH_A_UNIQUE_STRING with a unique string and replace.with.your.domain with your domain. Do not remove the /ai/callback or /callback paths.

    dex:
      config:
        staticClients:
          - id: mural
            redirectURIs:
              - 'https://replace.with.your.domain/ai/callback'
            name: 'mural'
            secret: 'REPLACE_WITH_A_UNIQUE_STRING'
            public: false
            trustedPeers:
              - kubernetes
          - id: kubernetes
            redirectURIs:
              - 'https://replace.with.your.domain/callback'
            name: kubernetes
            secret: 'REPLACE_WITH_A_UNIQUE_STRING'
            public: false
            trustedPeers:
              - mural

    1. Next, configure the dex.config.staticPasswords section. We strongly recommend changing the default user and password to strong values. The following example is the default user and password in bcrypt format. Remember to use a bcrypt hash generator to generate the password hash. The userID can be any unique string.

      warning

      If you did not configure any OIDC connectors, you must configure at least one static user, which is used to access the PaletteAI UI. Static Dex users automatically inherit admin privileges through the service account. Dex does not support groups for local static users. To use groups for local static users, you must use the User Impersonation feature.

      dex:
        config:
          staticPasswords:
            - email: 'admin@example.com'
              hash: '$2a$12$Ot2dJ0pmdIC2oXUDW/Ez1OIfhkSzLZIbsumsxkByuU3CUr02DtiC.'
              username: 'admin'
              userID: '08a8684b-db88-4b73-90a9-3cd1661f5466'

    2. Lastly, configure the dex.ingress section to expose Dex. For host, replace replace.with.your.domain with your domain. Do not change the className or the path. Because TLS is terminated at the load balancer, the tls section is empty.

      dex:
        ingress:
          enabled: true
          className: 'nginx'
          annotations: {}
          hosts:
            - host: replace.with.your.domain
              paths:
                - path: /dex
                  pathType: ImplementationSpecific
          tls: []
      Complete dex configuration section

canvas​

  1. Canvas controls part of the user interface. Review and modify the following values as necessary.

    1. Set canvas.ingress.enabled to true. Set the canvas.ingress.domain to your domain, omitting the protocol.

      canvas:
        ingress:
          enabled: true
          annotations: {}
          ingressClassName: nginx
          domain: example.acme.org
          tls: []
          paths:
            - path: /ai
              pathType: ImplementationSpecific
              backend:
                service:
                  name: canvas
                  port:
                    number: 2999

    2. Set canvas.enableHTTP to true. This supports TLS termination at the load balancer. canvas.ingress.tls remains empty as a result.

      canvas:
        enableHTTP: true

    3. The last portion of the Canvas configuration is the OIDC configuration. If you deferred configuring OIDC for Dex, you may do the same for Canvas configure it later.

      In the canvas.oidc section, enter a unique string for the sessionSecret. For redirectURL, replace replace.with.your.domain with your domain. Do not remove the /ai/callback path.

      Set skipSSLCertificateVerification to true to allow self-signed certificates generated by cert-manager. If you configured the ClusterIssuer to use a Certificate Authority (CA) provided by your organization, set skipSSLCertificateVerification to false.

      canvas:
        oidc:
          sessionSecret: 'REPLACE_WITH_A_UNIQUE_STRING'
          sessionDir: '/app/sessions'
          issuerK8sService: 'https://dex.mural-system.svc.cluster.local:5554/dex'
          skipSSLCertificateVerification: true
          redirectURL: 'https://replace.with.your.domain/ai/callback'

      If you did not configure your Kubernetes cluster to trust Dex as an OIDC provider and do not plan to, then you must configure the impersonationProxy section to enable user impersonation. The example below shows how to configure the local Dex user admin@example.com to be mapped to an example Kubernetes group admin. Review our User Impersonation page to learn more about how to configure user impersonation for OIDC groups and other use cases.

      canvas:
        impersonationProxy:
          enabled: true
          userMode: passthrough
          userMap: {}
          groupsMode: map
          groupMap: {}
          dexGroupMap:
            'admin@example.com': [ 'admin' ]
      Complete canvas configuration section

ingress-nginx​

  1. If your cluster already includes Ingress Nginx, set ingress-nginx.enabled to false.

    ingress-nginx:
      enabled: false

    Otherwise, configure ingress-nginx.controller to use the HTTP listener and terminate TLS at the load balancer. Change the value for targetPorts.https to http.

    ingress-nginx:
      controller:
        service:
          targetPorts:
            http: http
            https: http
    LoadBalancer Service Alternative

    Instead of using Ingress Nginx, you can disable ingress-nginx entirely and configure Canvas to use a LoadBalancer service type instead. To use this method, set the following values in your Helm chart:

    • ingress-nginx.enabled: false
    • canvas.ingress.enabled: false
    • canvas.service.type: LoadBalancer
    Complete ingress-nginx configuration section

flux2​

  1. The last portion of the values.yaml file to update is the flux2 section.

    Set flux2.policies.create to false to disable the Flux2 network policies. These policies, if enabled, prevent ingress traffic from reaching their target services.

    flux2:
      policies:
        create: false
    info

    This step is not required if the hub and all spoke clusters are configured to use a common, external OCI registry. An external OCI registry is configured in the fleetConfig.spokes[*].ociRegistry and hue.ociRegistry sections of the values.yaml file.

    Complete flux2 configuration section

cert-manager​

  1. Install cert-manager before installing the mural-crds and Mural Helm charts. Palette AI requires cert-manager to automatically provision and manage Transport Layer Security (TLS) certificates in Kubernetes.

    Refer to the cert-manager documentation for more information.

    Add the cert-manager Helm repository.

    helm repo add jetstack https://charts.jetstack.io
    helm repo update jetstack

    Install cert-manager.

    helm install cert-manager jetstack/cert-manager \
      --namespace cert-manager \
      --create-namespace \
      --version v1.19.1 \
      --set crds.enabled=true \
      --wait

    Verify that cert-manager is running.

    kubectl get pods -n cert-manager

    You should observe output similar to the following.

    NAME                                      READY   STATUS    RESTARTS   AGE
    cert-manager-7d9f7c8c7d-xz4qw            1/1     Running   0          1m
    cert-manager-cainjector-5d9c8c7d-abc12   1/1     Running   0          1m
    cert-manager-webhook-6b8c7d9f-def34      1/1     Running   0          1m

    FIPS-Compliant Installation

    For environments requiring FIPS compliance, create a cert-manager-values.yaml file with FIPS-approved images.

    crds:
      enabled: true

    image:
      repository: us-docker.pkg.dev/palette-images-fips/palette/spectro-cert-manager/cert-manager-controller
      tag: v1.19.1

    acmesolver:
      image:
        repository: us-docker.pkg.dev/palette-images-fips/palette/spectro-cert-manager/cert-manager-acmesolver
        tag: v1.19.1

    cainjector:
      image:
        repository: us-docker.pkg.dev/palette-images-fips/palette/spectro-cert-manager/cert-manager-cainjector
        tag: v1.19.1

    startupapicheck:
      image:
        repository: us-docker.pkg.dev/palette-images-fips/palette/spectro-cert-manager/cert-manager-startupapicheck
        tag: v1.19.1

    webhook:
      image:
        repository: us-docker.pkg.dev/palette-images-fips/palette/spectro-cert-manager/cert-manager-webhook
        tag: v1.19.1

    Install cert-manager using the values file.

    helm install cert-manager jetstack/cert-manager \
      --namespace cert-manager \
      --create-namespace \
      --version v1.19.1 \
      --values cert-manager-values.yaml \
      --wait

Helm Install​

  1. Install the mural-crds chart. This chart contains the Custom Resource Definitions (CRDs) required by PaletteAI and must be installed before the PaletteAI Mural chart.

    helm install mural-crds oci://public.ecr.aws/mural/mural-crds:0.3.2 \
      --namespace mural-system --create-namespace --wait
    Example output
    NAME: mural-crds
    LAST DEPLOYED: Tue May 27 09:34:33 2025
    NAMESPACE: mural-system
    STATUS: deployed
    REVISION: 1

    Next, install PaletteAI using the Mural Helm chart and the values.yaml file you configured in the previous steps.

    helm install mural oci://public.ecr.aws/mural/mural:0.6.2 \
      --namespace mural-system --create-namespace --values values.yaml --wait
    Example output
    NAME: mural
    LAST DEPLOYED: Tue May 27 09:39:48 2025
    NAMESPACE: mural-system
    STATUS: deployed
    REVISION: 1

DNS​

  1. Once PaletteAI is deployed, fetch the URL of the load balancer the ingress-nginx controller deployed.

    kubectl get service ingress-nginx-controller --namespace mural-system
    Example output
    NAME                       TYPE           CLUSTER-IP       EXTERNAL-IP                                                               PORT(S)                      AGE
    ingress-nginx-controller   LoadBalancer   10.104.129.101   a9d221d65b2fd41b3929574458e8ce05-1177779699.us-east-1.elb.amazonaws.com   80:31952/TCP,443:30926/TCP   41m
  2. Create a DNS record pointing the domain configured in the Mural chart values to the load balancer created by the Ingress Nginx Service. Use an A record for IP addresses or a CNAME/alias record for hostnames, depending on your DNS provider's capabilities.

You have now deployed PaletteAI on your Kubernetes cluster. The cluster is configured to trust Dex as an identity provider. Assuming you have configured Dex with an OIDC connector, you can now log in to PaletteAI using your identity provider. Alternatively, you can use the default Dex local user to log in to PaletteAI.

If you need to make changes to your installation, review the Helm Chart Configuration Reference page and modify your values.yaml file as necessary. To trigger the update, use the following command.

helm upgrade mural oci://public.ecr.aws/mural/mural:0.6.2 \
  --namespace mural-system --values values.yaml --wait

Proceed to the Validate section to validate that PaletteAI is deployed and configured correctly.

Validate​

Take the following steps to verify that PaletteAI is deployed and configured correctly.

  1. Open a browser and navigate to the domain URL you configured for PaletteAI.

  2. Login with the default username and password. If you configured Dex with an OIDC connector, log in with your identity provider.

Next Steps​

Once PaletteAI is installed on your cluster, you must integrate Palette with PaletteAI using PaletteAI's Settings resource. This resource requires a Palette tenant, project, and API key in order to communicate with Palette and deploy AI/ML applications to the appropriate location.

Proceed to the Integrate with Palette guide to learn how to prepare your Palette environment.