Skip to main content
Version: v1.4.x

Access an AI Virtual Machine

You reach an AI Virtual Machine with virtctl, the KubeVirt command line tool, against the Compute Pool's cluster. virtctl proxies through the cluster's KubeVirt API, so there is no separate SSH endpoint to expose and no firewall rule to add. The PaletteAI console provides the kubeconfig for the cluster, but it does not host a terminal or a console session, so virtctl is required.

Prerequisites​

Get the Compute Pool Cluster Kubeconfig​

  1. From the left main menu, open Compute Pools, and select the Compute Pool that runs your AI Virtual Machine.
  2. Select the Clusters tab.
  3. In the Kubeconfig column, download the kubeconfig file for the cluster.

Point kubectl at the file, and list the virtual machines on the cluster:

export KUBECONFIG=/path/to/<cluster-name>.kubeconfig
kubectl get virtualmachines --namespace default
Example Output
NAME                        AGE
gpu-dev-vm-4bbfa220         5m
cpu-sandbox-93cece96        2d

AI Virtual Machines run in the default namespace on the Compute Pool cluster. The virtual machine has its own name on that cluster, which PaletteAI derives, so it can differ from the AI Virtual Machine's name in the console. Use the name the command prints in the commands that follow.

Install virtctl​

Install virtctl at the same KubeVirt minor version as the Compute Pool's cluster. Download the release archive for your platform from the KubeVirt releases page, or install the plugin through Krew:

kubectl krew install virt

If you install through Krew, the plugin is invoked as kubectl virt rather than virtctl, so run kubectl virt where the commands below use virtctl. For more on the client tool, refer to the KubeVirt virtctl documentation.

Confirm the install, using kubectl virt in place of virtctl if you installed through Krew:

virtctl version --client

Connect over a Serial Console​

The serial console needs nothing beyond the credentials, so it is the quickest way in. Connect with virtctl console, substituting the virtual machine name from the list above:

virtctl console <vm-name> --namespace default

The command attaches to the running instance and drops you at the guest's login prompt, where you sign in as pai with the password pai. Leave the console with Ctrl+] or Ctrl+5.

warning

The built-in templates expire the pai password, so the first sign-in over any of these connections forces you to set a new password, and pai stops working from that point. Keep the new password, and keep a console or VNC session available as a way back in. Refer to Change AI Virtual Machine Credentials.

Connect over SSH​

SSH hands the connection to the SSH client on your machine, so the first connection asks you to confirm the virtual machine's host key. Accept the key when it is offered, or let the client accept a key it has not seen before without stopping to ask:

virtctl ssh pai@vm/<vm-name> --namespace default

The built-in templates create the pai user with the password pai and enable password sign-in. To use an SSH key instead of the password, add --identity-file.

Connect over VNC​

VNC attaches to the running instance, so the virtual machine must be in the Running state. Start a session with:

virtctl vnc <vm-name> --namespace default

virtctl vnc starts a proxy and hands it to a VNC viewer on your machine, such as remote-viewer or tiger-vnc. If no viewer is installed, add --proxy-only to print the local port and point your own viewer at it. Use --vnc-path to name a viewer that is not on your PATH.

Like the serial console, VNC works when SSH does not, for example while cloud-init is still finishing on first boot.

Troubleshoot the Connection​

  • The virtual machine is not in the Running state, so there is nothing to connect to. Refer to Manage an AI Virtual Machine.
  • SSH stops at a host key prompt. The first connection asks you to confirm the virtual machine's host key. Accept it, or pass it through with --local-ssh-opts='-o StrictHostKeyChecking=accept-new'.
  • SSH refuses the pai password. The password expires at first sign-in, so use the password you set, or connect over the serial console and set a new one. Refer to Change AI Virtual Machine Credentials.
  • The virtual machine name is not found. Read it from kubectl get virtualmachines, since the name on the cluster can differ from the console name.
  • kubectl cannot reach the cluster. Confirm the kubeconfig file is the one downloaded for this Compute Pool's cluster.

For failures that come from the AI Virtual Machine itself, refer to Troubleshoot AI Virtual Machines.

Next Steps​