---
title: "Enabling Trainers with GPUs in On-Premise Kubernetes Deployments"
slug: "enabling-trainers-with-gpus-in-on-premise-kubernetes-deployments"
updated: 2026-05-06T19:21:33Z
published: 2026-05-06T19:21:33Z
canonical: "help.hyperscience.ai/enabling-trainers-with-gpus-in-on-premise-kubernetes-deployments"
---

> ## Documentation Index
> Fetch the complete documentation index at: https://help.hyperscience.ai/llms.txt
> Use this file to discover all available pages before exploring further.

# Enabling Trainers with GPUs in On-Premise Kubernetes Deployments

With Hyperscience, you can classify and extract data from Unstructured documents. However, automating these processes requires more computing resources than our automation capabilities for Structured and Semi-structured documents do.

In order to process Unstructured documents in on-premise deployments of Hyperscience, you need to add a trainer that has both a GPU (graphics processing unit) and a CPU (central processing unit). GPUs have specialized cores that allow the system to perform multiple computations in parallel, reducing the time required to complete the complex operations required to train models for Unstructured documents. When you attach a trainer whose machine has a GPU, you can maximize the benefits of Long-form Extraction. To learn more about this feature, see [Long-form Extraction](https://help.hyperscience.ai/latest/docs/long-form-extraction).

> [!NOTE]
> **Support for trainers with GPUs in Hyperscience**
> 
> - Trainers with GPUs are supported in Docker, Podman, and Kubernetes deployments of **Hyperscience v41 and later**. Use of GPUs with v40 or earlier is not supported.
> - A trainer with a GPU is required to train Long-form Extraction models.
> - While GPUs are not required to train Field Identification or Table Identification models, you can use a trainer with a GPU to train these models.
> - If you need to run training for ORCA model definitions, you can do so if your trainer machine has a GPU with the required specifications. **The use of ORCA “out of the box” (i.e., without training) does not require a trainer with a GPU.**
> - The training of other types of models with a GPU is not supported. However, you can train them with a CPU on a trainer machine that has both a GPU and a CPU.

This article describes how to enable a trainer with both a GPU and a CPU in an on-premise Kubernetes deployment of Hyperscience.

In order for our training to take advantage of the GPU on the on-premise Kubernetes cluster, we have to make sure that the Kubernetes nodes are configured with correct drivers or AMIs and that the containers can access the GPU.

## Bare metal

### 1. Make sure your GPU hardware meets the requirements.

See [Kubernetes Installation Overview](/deployment/docs/kubernetes-installation-requirements#gpus-for-trainers) for more information.

### 2. Prepare for driver installation.

#### a. Verify that your GPU supports CUDA.

*CUDA* is a parallel computing platform and programming model created by NVIDIA. Machine learning often uses CUDA-based libraries, SDKs, and other tools.

You can find out whether your GPU supports CUDA by running the following command:

```bash
lspci | grep -i nvidia
```

For more information, see NVIDIA’s [CUDA GPUs - Compute Capability](https://developer.nvidia.com/cuda-gpus) and [NVIDIA CUDA Installation Guide for Linux](https://docs.nvidia.com/cuda/cuda-installation-guide-linux/index.html#verify-you-have-a-cuda-capable-gpu).

#### b. Verify that you have a supported version of Linux.

Follow the instructions in NVIDIA’s [NVIDIA CUDA Installation Guide for Linux](https://docs.nvidia.com/cuda/cuda-installation-guide-linux/index.html#verify-you-have-a-supported-version-of-linux) to check your version of Linux. Then, make sure your Linux version is supported by the latest CUDA Toolkit by reviewing NVIDIA’s [NVIDIA CUDA Toolkit Release Notes](https://docs.nvidia.com/cuda/cuda-toolkit-release-notes/index.html#cuda-toolkit-major-component-versions).

#### c. Verify that the system has *gcc* installed.

The `gcc` compiler is required for development using the CUDA Toolkit. To make sure it is installed, follow the instructions in NVIDIA’s [NVIDIA CUDA Installation Guide for Linux](https://docs.nvidia.com/cuda/cuda-installation-guide-linux/index.html#verify-the-system-has-gcc-installed).

#### d. Verify that the system has the current Kernel headers and development packages installed.

Kernel headers are header files that specify the interface between the Linux kernel and userspace libraries and programs. The CUDA driver requires that the kernel headers and development packages for the running version of the kernel be installed at the time of the driver installation, as well whenever the driver is rebuilt. For example, if your system is running kernel version 3.17.4-301, the 3.17.4-301 kernel headers and development packages must also be installed.

To verify that these requirements are met, run the following command:

```bash
apt-get install linux-headers-$(uname -r)
```

For more information and commands for various Linux distributions, see NVIDIA’s [NVIDIA CUDA Installation Guide for Linux](https://docs.nvidia.com/cuda/cuda-installation-guide-linux/index.html#verify-the-system-has-the-correct-kernel-headers-and-development-packages-installed).

### 3. Install the CUDA drivers.

Follow these steps to make sure that you have the most current and correct CUDA drivers installed.

#### a. Remove any CUDA drivers installed on the system.

Compatibility between the CUDA Toolkit and CUDA drivers is crucial. In our Docker-based system, the CUDA Toolkit is installed in the trainer images, and you need to make sure that they match the CUDA drivers that are installed in the host OS. The current CUDA Toolkit version installed should be compatible with the latest available CUDA driver. For details on toolkit and driver versions, see NVIDIA’s [NVIDIA CUDA Toolkit Release Notes](https://docs.nvidia.com/cuda/cuda-toolkit-release-notes/index.html#id5).

![CUDADriversAndToolkits.png](https://cdn.us.document360.io/87894cef-4958-4f3f-be6f-b75a78c82548/Images/Documentation/20747131784717.png)

An example command for removing these drivers appears below:

```bash
apt-get clean; apt-get update; apt-get purge -y cuda*; apt-get purge -y nvidia-*; apt-get -y autoremove
```

You can tailor this command to match your Linux distribution.

#### b. Install *nvidia-container-toolkit* or *nvidia-docker2*.

The Docker host needs to be prepared before it can expose your GPU hardware to the containers. Although containers share your host’s kernel, they cannot access information on the system packages you have installed. A plain container will lack the device drivers that interface with your GPU.

**(Ubuntu 21 and later) Install *nvidia-container-toolkit*.**

You can activate support for NVIDIA GPUs by installing NVIDIA’s Docker Container Toolkit:

```bash
distribution=$(. /etc/os-release;echo $ID$VERSION_ID) \
        && curl -fsSL https://nvidia.github.io/libnvidia-container/gpgkey | gpg --dearmor -o /usr/share/keyrings/nvidia-container-toolkit-keyring.gpg \
        && curl -s -L https://nvidia.github.io/libnvidia-container/$distribution/libnvidia-container.list | \
              sed 's#deb https://#deb [signed-by=/usr/share/keyrings/nvidia-container-toolkit-keyring.gpg] https://#g' | \
              tee /etc/apt/sources.list.d/nvidia-container-toolkit.list
        apt-get update && apt-get install -y nvidia-container-toolkit
```

**(Ubuntu 20) Install *nvidia-docker2*.**

To activate support for NVIDIA GPUs, install `nvidia-docker2` by running the following command:

```bash
distribution=$(. /etc/os-release;echo $ID$VERSION_ID) \
        && curl -s -L https://nvidia.github.io/nvidia-docker/gpgkey | sudo apt-key add - \
        && curl -s -L https://nvidia.github.io/nvidia-docker/$distribution/nvidia-docker.list | sudo tee /etc/apt/sources.list.d/nvidia-docker.list
        apt-get update && apt-get install -o Dpkg::Options::="--force-confold"   -y nvidia-docker2
```

#### **c. Configure the container runtime.**

Run the `nvidia-ctk` command shown below to configure the container runtime.

The `nvidia-ctk` command modifies the `etc/docker/daemon.json` file on the host. The file is updated so that Docker can use the NVIDIA Container Runtime.

```bash
nvidia-ctk runtime configure --runtime=docker
systemctl restart docker
```

Inspect your `/etc/docker/daemon.json` file to confirm that the configured container runtime has been changed. The NVIDIA Toolkit will handle the injection of GPU device connections when new containers start.

#### d. Install the latest CUDA drivers.

Running the following command installs the latest CUDA driver versions, which should be compatible with the container toolkit:

```bash
distribution=$(. /etc/os-release;echo $ID$VERSION_ID | sed -e 's/\.//g')
        wget https://developer.download.nvidia.com/compute/cuda/repos/$distribution/x86_64/cuda-keyring_1.0-1_all.deb
        dpkg -i cuda-keyring_1.0-1_all.deb
        apt-get update && apt-get -y install cuda-drivers
```

While we recommend installing the latest version of the CUDA driver, only the minimum required version for your version of Hyperscience is required. See [Kubernetes Installation Overview](/deployment/docs/kubernetes-installation-requirements#gpus-for-trainers) for more information.

#### e. Deploy the *nvidia-device-plugin* DaemonSet.

To deploy the DaemonSet, follow the instructions in the “Enabling GPU Support in Kubernetes” section of the [NVIDIA device plugin for Kubernetes](https://github.com/NVIDIA/k8s-device-plugin#enabling-gpu-support-in-kubernetes) documentation in GitHub.

## AWS EKS

1. You can use the pre-built GPU AMIs made available by AWS with all the necessary drivers installed. For more information, see the “Amazon EKS optimized accelerated Amazon Linux AMIs” section of AWS’s [Amazon EKA optimized Amazon Linux AMIs](https://docs.aws.amazon.com/eks/latest/userguide/eks-optimized-ami.html#gpu-ami).
  - For a list of pre-built GPU AMIs based on Kubernetes version, see Amazon EKS AMI’s [Releases](https://github.com/awslabs/amazon-eks-ami/releases) list in GitHub.
2. Deploy `nvidia-plugin-daemonset` by following the instructions in the “Enabling GPU Support in Kubernetes” section of the [NVIDIA device plugin for Kubernetes](https://github.com/NVIDIA/k8s-device-plugin#enabling-gpu-support-in-kubernetes) documentation in GitHub.

## Azure AKS

To enable GPU for Azure AKS, follow the steps in Microsoft’s [Use GPUs for compute-intensive workloads on Azure Kubernetes Service (AKS)](https://learn.microsoft.com/en-us/azure/aks/gpu-cluster). Doing so sets up the GPU nodes and installs the `nvidia-device-plugin` DaemonSet.

## Google GKE

Follow Google Cloud’s [Run GPUs in GKE Standard node pools](https://cloud.google.com/kubernetes-engine/docs/how-to/gpus) to set up GPU node pools with drivers and the DaemonSet.

## Next steps

After you’ve enabled your trainer, follow the steps in “Long-form Extraction” ( [v41](/v41/docs/long-form-extraction#prerequisites) | [v42](/v42/docs/long-form-extraction#prerequisites) | [v43](/v43/docs/long-form-extraction#prerequisites) ) to apply the trainer to your Long-form Extraction use case.