Getting started with Docker

Lets install Docker. this will allow you to use docker containers. In Debian, we will add the docker repository

sudo apt-get update
sudo apt-get install ca-certificates curl
sudo install -m 0755 -d /etc/apt/keyrings
sudo curl -fsSL https://download.docker.com/linux/debian/gpg -o /etc/apt/keyrings/docker.asc
sudo chmod a+r /etc/apt/keyrings/docker.asc

echo \
  "deb [arch=$(dpkg --print-architecture) signed-by=/etc/apt/keyrings/docker.asc] https://download.docker.com/linux/debian \
  $(. /etc/os-release && echo "$VERSION_CODENAME") stable" | \
  sudo tee /etc/apt/sources.list.d/docker.list > /dev/null
sudo apt-get update

Now we install the packages from the repository

sudo apt-get install docker-ce docker-ce-cli containerd.io docker-buildx-plugin docker-compose-plugin

Now we can test the installation

sudo docker run hello-world

lets check our groups we should add our user to the docker group

sudo usermod -aG docker $USER

Now lets make a folder where we can store data for our docker containers. AI models and supporting data or dependencies take a large amount of space so its a good idea to use a location not in the system directories.

mkdir <your storage area>

Now that we have docker running we can use a variety of pre-packaged applications without the need for specific environments or more complex setups. To give these containers access to our NVIDIA GPU we will need to install the Nivida Container Toolkit. Add the repository

curl -fsSL https://nvidia.github.io/libnvidia-container/gpgkey | sudo gpg --dearmor -o /usr/share/keyrings/nvidia-container-toolkit-keyring.gpg \
  && curl -s -L https://nvidia.github.io/libnvidia-container/stable/deb/nvidia-container-toolkit.list | \
    sed 's#deb https://#deb [signed-by=/usr/share/keyrings/nvidia-container-toolkit-keyring.gpg] https://#g' | \
    sudo tee /etc/apt/sources.list.d/nvidia-container-toolkit.list

sudo apt update

And now we can install the packages

export NVIDIA_CONTAINER_TOOLKIT_VERSION=1.17.8-1
  sudo apt-get install -y \
      nvidia-container-toolkit=${NVIDIA_CONTAINER_TOOLKIT_VERSION} \
      nvidia-container-toolkit-base=${NVIDIA_CONTAINER_TOOLKIT_VERSION} \
      libnvidia-container-tools=${NVIDIA_CONTAINER_TOOLKIT_VERSION} \
      libnvidia-container1=${NVIDIA_CONTAINER_TOOLKIT_VERSION}

Now we will configure the toolkit. We are using Docker, so we will use

sudo nvidia-ctk runtime configure --runtime=docker

Once this is complete you should get a prompt recommending docker daemon is restarted

sudo systemctl restart docker