From Zero to GPU-Accelerated Containers

Primary SEO Target Keywords: Docker setup for machine learning, Docker GPU passthrough, NVIDIA Container Toolkit tutorial, containerize PyTorch, Docker architecture deep dive.

Meta Description: Master Docker fundamentals and GPU containerization. Step-by-step setup guide for Linux/WSL2, core command cheat sheet, and layer-caching best practices for AI workloads.

1. Why Docker Matters in Machine Learning Infrastructure

In traditional software development, “it works on my machine” is an annoyance. In machine learning infrastructure, it is a operational catastrophe. Differences in Python sub-versions, compiled C++ extensions, CUDA drivers, and shared library paths (.so files) regularly break model reproduction between local workstations and cloud clusters.

Docker solves dependency drift by isolating the application, software runtime, and system libraries into a self-contained unit called a container.

Virtual Machines vs. Docker Containers

Characteristic	Virtual Machines (VMs)	Docker Containers
Architecture	Guest OS on top of Hypervisor	Shares host Linux Kernel via Namespaces & Cgroups
Startup Time	Minutes	Milliseconds to Seconds
Resource Overhead	Heavy (requires dedicated RAM/CPU for OS)	Near-Zero (native process execution)
GPU Passthrough	Complex PCIe passthrough required	Native device node injection (/dev/nvidia*)
Image Size	10 GB – 50 GB+	200 MB – 3 GB (Optimized)

2. The 4 Core Building Blocks of Docker

To read and write Docker configurations effectively, you must understand four primary concepts:

1. Dockerfile: A text blueprint containing sequential instructions (FROM, RUN, COPY, CMD) used to assemble an image.
2. Image: An immutable, read-only template containing the application code, dependencies, and environment binaries. Images are built in stacked layers.
3. Container: A runnable, isolated instance of an image. If an image is a class in Object-Oriented Programming, a container is an instantiated object.
4. Volume / Bind Mount: A mechanism to map a folder on the host filesystem directly into the container filesystem, enabling persistent data storage and live code syncing.

3. Step-by-Step Setup Guide: Installing Docker & GPU Drivers

This guide covers setup for Ubuntu 22.04 / 24.04 LTS and Windows 11 (via WSL2).

Step 1: Install Docker Engine on the Host

Run the following commands in your terminal to install the official Docker Engine:

Bash

# Update package index and install prerequisites
sudo apt-get update
sudo apt-get install -y ca-certificates curl gnupg

# Add Docker's official GPG key
sudo install -m 0755 -d /etc/apt/keyrings
curl -fsSL https://download.docker.com/linux/ubuntu/gpg | sudo gpg --dearmor -o /etc/apt/keyrings/docker.gpg
sudo chmod a+r /etc/apt/keyrings/docker.gpg

# Set up the stable repository
echo \
  "deb [arch=$(dpkg --print-architecture) signed-by=/etc/apt/keyrings/docker.gpg] https://download.docker.com/linux/ubuntu \
  $(. /etc/os-release && echo "$VERSION_CODENAME") stable" | \
  sudo tee /etc/apt/sources.list.d/docker.list > /dev/null

# Install Docker Engine and CLI tools
sudo apt-get update
sudo apt-get install -y docker-ce docker-ce-cli containerd.io docker-buildx-plugin docker-compose-plugin

Step 2: Configure Non-Root User Access (Optional but Recommended)

By default, running docker commands requires sudo. Manage Docker as a non-root user:

Bash

sudo usermod -aG docker $USER
newgrp docker

Sanity Check: Run docker run hello-world to verify the installation.

Step 3: Install the NVIDIA Container Toolkit (For GPU Access)

To pass physical host GPUs into isolated Docker containers, install the NVIDIA Container Toolkit:

Bash

# Configure the repository
curl -fsSL https://nvidia.github.io/libnvidia-container/gpgkey | sudo gpg --dearmor -o /usr/share/keyrings/nvidia-container-toolkit-keyring.gpg \
  && curl -s -L https://nvidia.github.io/libnvidia-container/stable/deb/nvidia-container-toolkit.list | \
    sed 's#deb [^ ]*#& [signed-by=/usr/share/keyrings/nvidia-container-toolkit-keyring.gpg]#g' | \
    sudo tee /etc/apt/sources.list.d/nvidia-container-toolkit.list

# Install the toolkit
sudo apt-get update
sudo apt-get install -y nvidia-container-toolkit

# Configure Docker daemon to register the NVIDIA runtime
sudo nvidia-ctk runtime configure --runtime=docker

# Restart Docker daemon
sudo systemctl restart docker

4. Docker CLI Command Cheat Sheet

Command	Action / Purpose
docker build -t <name> .	Builds an image tagged <name> from the current directory Dockerfile.
docker run --rm -it <image>	Launches an interactive container terminal and deletes it upon exit (--rm).
docker run --gpus all <image>	Launches container with all host physical GPUs passed through.
docker ps -a	Lists all active and stopped containers.
docker images	Displays all cached Docker images on the host machine.
docker exec -it <id> bash	Opens a live interactive shell inside a running container.
docker system prune -a	Deletes unused images, stopped containers, and build caches to free disk space.

5. Practical Hands-On Tutorial: Building Your First GPU Machine Learning Container

Now that Docker and the NVIDIA Container Toolkit are configured, let’s create a minimal containerized PyTorch workspace and run a hardware verification script.

Step 1: Create the Project Files

Create a project directory on your host machine:

Bash

mkdir docker-ml-demo && cd docker-ml-demo

Create a file named verify_environment.py:

Python

import torch

print("--- System Environment Check ---")
print(f"PyTorch Version: {torch.__version__}")
print(f"CUDA Available:  {torch.cuda.is_available()}")

if torch.cuda.is_available():
    print(f"GPU Device Name: {torch.cuda.get_device_name(0)}")
    print(f"Device Count:    {torch.cuda.device_count()}")
else:
    print("WARNING: Running on CPU. No GPU detected!")

Create a requirements.txt file:

Plaintext

torch==2.11.*
numpy

Create the Dockerfile:

Dockerfile

# 1. Base Image with CUDA Runtime Support
FROM nvidia/cuda:13.0.0-runtime-ubuntu24.04

# 2. Prevent Interactive Prompts During Package Installations
ENV DEBIAN_FRONTEND=noninteractive

# 3. System Dependencies (Cleaned in the same layer to minimize size)
RUN apt-get update && apt-get install -y --no-install-recommends \
    python3.12 \
    python3-pip \
    && rm -rf /var/lib/apt/lists/*

# 4. Install Python Dependencies FIRST (Leverages Docker Layer Caching)
COPY requirements.txt /tmp/requirements.txt
RUN pip install --no-cache-dir -r /tmp/requirements.txt --index-url https://download.pytorch.org/whl/cu130

# 5. Copy Application Source Code LAST
COPY verify_environment.py /workspace/verify_environment.py
WORKDIR /workspace

# 6. Default Execution Command
CMD ["python3", "verify_environment.py"]

Step 2: Build the Image

Build the container image and tag it as cuda-verifier:

Bash

docker build -t cuda-verifier .

Step 3: Run the Container with GPU Passthrough

Execute the container while passing host physical GPUs via the --gpus all flag:

Bash

docker run --rm --gpus all cuda-verifier

Expected Output:

Plaintext

--- System Environment Check ---
PyTorch Version: 2.11.0+cu130
CUDA Available:  True
GPU Device Name: NVIDIA RTX 4090
Device Count:    1

6. Docker Layer Caching Rules for Machine Learning

Understanding Docker’s layer cache is essential to avoid waiting 15 minutes for builds to complete every time you edit a line of code.

The Order Rule: Dependencies Before Code

Docker evaluates Dockerfile lines sequentially from top to bottom. If a line changes, that line and all subsequent lines invalidate their cache and must rebuild from scratch.

• Anti-Pattern (Slow Builds):
  DockerfileCOPY . /workspace # Editing code here invalidates the cache! RUN pip install -r requirements.txt # Re-installs 5GB PyTorch wheel EVERY build
• Best Practice (Fast Builds):
  DockerfileCOPY requirements.txt /tmp/ # Only invalidates if dependencies change RUN pip install -r /tmp/requirements.txt COPY . /workspace # Code edits complete in milliseconds!

By copying requirements.txt before the rest of the source directory, updating your training scripts or model architectures lets Docker reuse the multi-gigabyte PyTorch installation layer cached on disk.