Docker for GenAI Applications

Containerize ML models and APIs for consistent, reproducible deployments

What is Docker?

Docker is a platform for developing, shipping, and running applications in containers. Containers package code and dependencies together, ensuring your GenAI application runs consistently across development, testing, and production environments. For ML/AI workloads, Docker solves the notorious "works on my machine" problem.

Key Innovation: Docker containers are lightweight, start in seconds, and share the host OS kernel—unlike VMs which require a full OS per instance. This makes them ideal for scaling inference endpoints.

Portable

Run anywhere

Isolated

No conflicts

Fast

Seconds to start

Scalable

Horizontal scaling

Core Concepts

Image

Read-only template with instructions

Container

Running instance of an image

Dockerfile

Recipe to build an image

Registry

Storage for images (Docker Hub, GCR)

Dockerfile for GenAI Applications

Python ML API Dockerfile

# Use official Python slim image
FROM python:3.11-slim

# Set working directory
WORKDIR /app

# Install system dependencies for ML libraries
RUN apt-get update && apt-get install -y \
    build-essential \
    curl \
    && rm -rf /var/lib/apt/lists/*

# Copy requirements first (cache layer)
COPY requirements.txt .

# Install Python dependencies
RUN pip install --no-cache-dir -r requirements.txt

# Copy application code
COPY src/ ./src/
COPY models/ ./models/

# Set environment variables
ENV PYTHONUNBUFFERED=1
ENV PORT=8080

# Expose port
EXPOSE 8080

# Run the application
CMD ["uvicorn", "src.main:app", "--host", "0.0.0.0", "--port", "8080"]

Multi-Stage Build (Optimized)

# Stage 1: Build
FROM python:3.11-slim AS builder

WORKDIR /app
COPY requirements.txt .
RUN pip install --no-cache-dir --user -r requirements.txt

# Stage 2: Production
FROM python:3.11-slim

WORKDIR /app

# Copy only installed packages from builder
COPY --from=builder /root/.local /root/.local
ENV PATH=/root/.local/bin:$PATH

# Copy application
COPY src/ ./src/

# Non-root user for security
RUN useradd -m appuser && chown -R appuser /app
USER appuser

EXPOSE 8080
CMD ["uvicorn", "src.main:app", "--host", "0.0.0.0", "--port", "8080"]

Essential Docker Commands

Building & Running

# Build image from Dockerfile
docker build -t mygenai-app:latest .

# Run container
docker run -d -p 8080:8080 --name genai-api mygenai-app:latest

# Run with environment variables
docker run -d -p 8080:8080 \
  -e OPENAI_API_KEY=$OPENAI_API_KEY \
  -e MODEL_NAME=gpt-4 \
  mygenai-app:latest

# Run with volume mount (for development)
docker run -d -p 8080:8080 \
  -v $(pwd)/src:/app/src \
  mygenai-app:latest

Management Commands

# List running containers
docker ps

# View logs
docker logs -f genai-api

# Execute command in container
docker exec -it genai-api /bin/bash

# Stop container
docker stop genai-api

# Remove container
docker rm genai-api

# List images
docker images

# Remove image
docker rmi mygenai-app:latest

Docker Compose for GenAI Stack

Tip: Docker Compose is perfect for local development with multiple services (API + database + vector store).

# docker-compose.yml
version: '3.8'

services:
  genai-api:
    build: .
    ports:
      - "8080:8080"
    environment:
      - OPENAI_API_KEY=${OPENAI_API_KEY}
      - DATABASE_URL=postgresql://user:pass@postgres:5432/genai
      - REDIS_URL=redis://redis:6379
    depends_on:
      - postgres
      - redis
    volumes:
      - ./src:/app/src  # Hot reload for development
  
  postgres:
    image: postgres:15-alpine
    environment:
      POSTGRES_USER: user
      POSTGRES_PASSWORD: pass
      POSTGRES_DB: genai
    volumes:
      - postgres_data:/var/lib/postgresql/data
    ports:
      - "5432:5432"
  
  redis:
    image: redis:7-alpine
    ports:
      - "6379:6379"
  
  chromadb:
    image: chromadb/chroma:latest
    ports:
      - "8000:8000"
    volumes:
      - chroma_data:/chroma/chroma

volumes:
  postgres_data:
  chroma_data:

Best Practices for GenAI

Warning: Never include API keys, model weights, or sensitive data in Docker images. Use environment variables, secrets management, or mount volumes.

Use slim base images: python:3.11-slim not python:3.11 (smaller, faster)
Multi-stage builds: Separate build and runtime stages to reduce image size
Layer caching: Copy requirements.txt before source code for better cache
Non-root user: Run containers as non-root for security
.dockerignore: Exclude unnecessary files (venv, __pycache__, .git)
Health checks: Add HEALTHCHECK instruction for orchestrators
Pin versions: Use specific tags, not :latest in production
GPU support: Use NVIDIA Container Toolkit for GPU workloads

.dockerignore for ML Projects

# .dockerignore
.git
.gitignore
.env
.env.*
*.md
LICENSE

# Python
__pycache__
*.pyc
*.pyo
.pytest_cache
.mypy_cache
venv/
.venv/

# IDE
.vscode/
.idea/

# Data & Models (use volumes or download at runtime)
data/
*.pt
*.pth
*.h5
*.onnx

# Tests (unless needed in image)
tests/

# Docker
Dockerfile
docker-compose*.yml
.dockerignore

GPU Support for ML Training

NVIDIA GPU Dockerfile

# Use NVIDIA CUDA base image
FROM nvidia/cuda:12.1-runtime-ubuntu22.04

# Install Python
RUN apt-get update && apt-get install -y python3.11 python3-pip

WORKDIR /app
COPY requirements.txt .
RUN pip install --no-cache-dir -r requirements.txt

COPY . .

# Run with GPU access
# docker run --gpus all mytraining:latest
CMD ["python", "train.py"]

Push to Container Registry

# Google Container Registry (GCR)
docker tag mygenai-app:latest gcr.io/my-project/genai-app:latest
docker push gcr.io/my-project/genai-app:latest

# Google Artifact Registry (Recommended)
docker tag mygenai-app:latest us-central1-docker.pkg.dev/my-project/repo/genai-app:latest
docker push us-central1-docker.pkg.dev/my-project/repo/genai-app:latest

# Docker Hub
docker tag mygenai-app:latest username/genai-app:latest
docker push username/genai-app:latest

Learn More

Essential Resources

Test Your Knowledge

Score 8/10 or higher to pass