Enable Nvidia Runtime on Docker

Make Sure nvidia succeed installed on your machine :

[root@ecs-jellyfin ~]$ nvidia-smi
Sun Oct 20 00:52:05 2024
+-----------------------------------------------------------------------------+
| NVIDIA-SMI 470.182.03   Driver Version: 470.182.03   CUDA Version: 11.4     |
|-------------------------------+----------------------+----------------------+
| GPU  Name        Persistence-M| Bus-Id        Disp.A | Volatile Uncorr. ECC |
| Fan  Temp  Perf  Pwr:Usage/Cap|         Memory-Usage | GPU-Util  Compute M. |
|                               |                      |               MIG M. |
|===============================+======================+======================|
|   0  Tesla T4            On   | 00000000:00:0D.0 Off |                    0 |
| N/A   32C    P8    10W /  70W |      0MiB / 15109MiB |      0%      Default |
|                               |                      |                  N/A |
+-------------------------------+----------------------+----------------------+

+-----------------------------------------------------------------------------+
| Processes:                                                                  |
|  GPU   GI   CI        PID   Type   Process name                  GPU Memory |
|        ID   ID                                                   Usage      |
|=============================================================================|
|  No running processes found                                                 |
+-----------------------------------------------------------------------------+

Installing the NVIDIA Container Toolkit on host

https://docs.nvidia.com/datacenter/cloud-native/container-toolkit/latest/install-guide.html#installing-with-yum-or-dnf

curl -s -L https://nvidia.github.io/libnvidia-container/stable/rpm/nvidia-container-toolkit.repo | tee /etc/yum.repos.d/nvidia-container-toolkit.repo
sudo yum-config-manager --enable nvidia-container-toolkit-experimental
sudo yum install -y nvidia-container-toolkit
which nvidia-container-runtime

Configure on Docker

https://docs.nvidia.com/datacenter/cloud-native/container-toolkit/latest/install-guide.html#configuration

# Set up the NVIDIA package repository
sudo mkdir -p /etc/systemd/system/docker.service.d
sudo tee /etc/systemd/system/docker.service.d/override.conf <<EOF
[Service]
ExecStart=
ExecStart=/usr/bin/dockerd --host=fd:// --add-runtime=nvidia=/usr/bin/nvidia-container-runtime
EOF
# Reload systemd and restart Docker service
sudo systemctl daemon-reload
sudo systemctl restart docker
# confirm the runtime has been added with the following command:
sudo systemctl show --property=ExecStart docker
# The ExecStart line should show something like:
# ExecStart=/usr/bin/dockerd --host=fd:// --add-runtime=nvidia=/usr/bin/nvidia-container-runtime

Install NVIDIA Docker Packages

root@ecs-jellyfin ~]$ sudo yum install -y nvidia-docker2
root@ecs-jellyfin ~]$ sudo systemctl restart docker
root@ecs-jellyfin ~]$ docker info | grep -i runtime
 Runtimes: nvidia runc io.containerd.runc.v2
 Default Runtime: runc

Run Application to try

[root@ecs-jellyfin ~]$ cat /opt/jellyfin/compose.yml
services:
  jellyfin:
    image: lscr.io/linuxserver/jellyfin:latest
    container_name: jellyfin
    runtime: nvidia
    environment:
      - PUID=0
      - PGID=0
      - TZ=Etc/UTC
      - NVIDIA_VISIBLE_DEVICES=all
      - JELLYFIN_PublishedServerUrl=http://192.168.0.5 # optional
    volumes:
      - ./config:/config
      - /media:/media/tvshows
    ports:
      - 8096:8096
    restart: unless-stopped
[root@ecs-jellyfin ~]$ docker-compose -f /opt/jellyfin/compose.yml up -d
[+] Running 1/0
 ✔ Container jellyfin  Running
[root@ecs-jellyfin alist]$ docker run --rm --gpus all nvidia/cuda:12.6.2-cudnn-devel-ubi9 nvidia-smidocker run --rm --gpus all nvidia/cuda:12.6.2-cudnn-devel-ubi9 nvidia-smi

OR Just run Application on the host

# Install jellyfin

https://repo.jellyfin.org/?path=/server/centos/latest-stable/amd64

[root@ecs-jellyfin j]$ sudo dnf install --nogpgcheck https://dl.fedoraproject.org/pub/epel/epel-release-latest-$(rpm -E %rhel).noarch.rpm
[root@ecs-jellyfin j]$ sudo dnf install --nogpgcheck https://mirrors.rpmfusion.org/free/el/rpmfusion-free-release-$(rpm -E %rhel).noarch.rpm https://mirrors.rpmfusion.org/nonfree/el/rpmfusion-nonfree-release-$(rpm -E %rhel).noarch.rpm
[root@ecs-jellyfin j]$ sudo dnf repolist && dnf install curl wget gnupg2 ca-certificates libva libvdpau ffmpeg jellyfin -y
[root@ecs-jellyfin j]$ systemctl start jellyfin.service
[root@ecs-jellyfin j]$ systemctl status jellyfin.service