Press CTRL+K to open search

Enabling Application Machines with GPUs in On-Premise Podman Deployments

  • Updated on Oct 21, 2026
  • Published on May 6, 2026
  • 3 minute(s) read
Prev Next

This procedure sets up an application machine with a GPU to run ORCA (Optical Reasoning and Cognition Agent) VLMs in an on-premises Podman deployment of Hyperscience. It is for system administrators.

To use ORCA VLMs, your instance needs at least one application machine that has both a GPU (graphics processing unit) and a CPU (central processing unit).

Availability

Application machines with GPUs are supported in v41 and later, and only for ORCA use cases.

Prerequisites

Complete the following checks and stage 1 before you extract the Hyperscience bundle on the application machine. For the installation procedure, see Technical Installation Instructions.

Verify the GPU hardware

Make sure that your GPU meets the hardware requirements in Infrastructure Requirements.

Verify that the lspci command is enabled

  1. Install the pciutils package. On RHEL, run the following command:

    yum -y install pciutils

  2. Run lspci to confirm that the command is enabled.

Verify that your GPU supports CUDA

CUDA is a parallel computing platform and programming model created by NVIDIA. To check whether your GPU supports CUDA, run the following command:

lspci | grep -i nvidia

For more information, see NVIDIA’s CUDA GPUs - Compute Capability and NVIDIA CUDA Installation Guide for Linux.

Verify that you have a supported version of Linux

Follow the instructions in NVIDIA’s NVIDIA CUDA Installation Guide for Linux to check your version of Linux. Then, make sure your Linux version is supported by the latest CUDA Toolkit by reviewing NVIDIA’s NVIDIA CUDA Toolkit Release Notes.

Make sure that your Linux distribution and version are supported by Hyperscience. Compatibility with the CUDA Toolkit alone is not enough. For the supported distributions and versions, see Infrastructure Requirements.

Verify that gcc is installed

The CUDA Toolkit requires the gcc compiler. To check that gcc is installed, follow the instructions in NVIDIA’s NVIDIA CUDA Installation Guide for Linux.

Verify the kernel headers and development packages

The CUDA driver requires the kernel headers and development packages for the kernel version that your system runs. It requires them when you install the driver and whenever the driver is rebuilt. For example, if your system runs kernel version 3.17.4-301, you need the 3.17.4-301 kernel headers and development packages.

To install them, or to confirm that they are installed, run the following command on RHEL:

sudo dnf install kernel-devel-$(uname -r) kernel-headers-$(uname -r)

For the commands for other Linux distributions, see NVIDIA’s NVIDIA CUDA Installation Guide for Linux.

1. Install the CUDA driver and the NVIDIA Container Toolkit

Prepare for the driver installation

Make sure your system meets the prerequisites for the driver installation:

Install the CUDA driver

Run the following command on RHEL:

sudo dnf module install nvidia-driver:latest-dkms

We recommend the latest CUDA driver. For the minimum driver version that your version of Hyperscience requires, see Infrastructure Requirements.

Install the NVIDIA Container Toolkit

Complete the steps in the “Installing with Yum or Dnf” section of NVIDIA’s Installing the NVIDIA Container Toolkit.

Configure Podman to use NVIDIA devices

  1. Complete the steps in the “Procedure” section of NVIDIA’s Support for Container Device Interface.

  2. In /usr/share/containers/containers.conf, set NVIDIA as the runtime (runtime = "nvidia"). If another runtime is set, comment it out.

  3. In the same file, add nvidia = ["/usr/bin/nvidia-container-runtime"] to [engine.runtimes].

Your file should look similar to the following:

runtime = "nvidia"
...
[engine.runtimes]
...
nvidia = [
"/usr/bin/nvidia-container-runtime",
]

Optionally, reboot the system:

sudo reboot

Checkpoint: The CUDA driver and the NVIDIA Container Toolkit are installed, and /usr/share/containers/containers.conf sets nvidia as the runtime.

2. Initialize ORCA on the GPU machine

Initialize ORCA on the application machine that has the GPU. The system then uses that machine for tasks that require ORCA. Run the following commands on the GPU machine.

  1. If you have not completed the standard installation steps on this machine, run them:

    ./run.sh init
./run.sh

  2. Run the ORCA-specific installation step for your ORCA base model. Run this step only on the GPU machine.

    • For ORCA 1.0:

      ./run.sh ipm VISION_LANGUAGE_MODEL_GPU

    • For any other ORCA base model, such as ORCA 2:

      ./run.sh ipm VISION_LANGUAGE_MODEL_GPU_2

Next steps

To install an ORCA base model, see Installing ORCA VLMs.

© Hyperscience