Just Enough Docker

Containers and Docker fundamentals — essential knowledge for any data role.

What this video covers

Docker packages an app together with everything it needs to run, so it behaves the same on your laptop, your teammate’s laptop, and a production server. This lesson goes from the idea (what a container is and how it differs from a virtual machine or a Python venv) to the hands-on flow: write a Dockerfile, build an image, run a container, look inside it, run several containers together with Docker Compose, and share images through Docker Hub. Everything you need is on this page — read it end to end, or skim the cheatsheet at the bottom if you already know Docker.

The whole flow at a glance

1. Write your app and list its dependencies (requirements.txt).
2. Describe how to package it in a Dockerfile.
3. Build an image: docker build.
4. Run a container from the image: docker run.
5. Check on it: docker ps, docker logs, docker exec.
6. Combine several containers (app + database) with docker compose.
7. Share the image through a registry like Docker Hub: docker push.

Part 1 — The idea

What is Docker, and why we need it

Code that works on your laptop often breaks somewhere else — a different OS, different library versions, a missing dependency. Docker fixes this by bundling your code plus everything it needs into a container, which runs the same on any machine.

• Image — the packaged blueprint (code + dependencies). Read-only.
• Container — a running copy of an image.

Why analysts care: dbt, Airflow, and Python pipelines usually run in containers in production. If it works in your container, it works there too.

Container vs. virtual machine

A virtual machine runs a full guest operating system on top of yours — heavy (gigabytes) and slow to boot (minutes). A container shares your machine’s OS kernel and packages only the app — light (megabytes) and starts in seconds. That’s why you can run many containers on one machine but only a few VMs.

• VM = a whole separate computer, simulated inside yours.
• Container = just your app, boxed up, running on your OS.

Container vs. Python venv

You may already know venv: it isolates Python packages per project, but it still uses your machine’s OS, Python version, and system tools. Docker isolates everything — OS, system libraries, Python version, and packages.

venv = a clean room for packages. Docker = a clean room for the whole machine your app thinks it’s running on.

Containers are disposable

A container is just a running copy of an image. If it crashes, don’t repair it — delete it and start a fresh one from the same image, which comes up in the same known-good state. That makes containers easy to restart and easy to scale (start more copies), and it’s the idea behind self-healing systems like Kubernetes.

Think of it as image = recipe, container = dish. A ruined dish? Cook it again from the recipe.

Part 2 — Build your first image

Prepare a small project

A typical Python project folder has your script, a requirements.txt listing the packages it needs, and (soon) a Dockerfile. Working in a venv keeps that package list clean:

python3 -m venv venv

source venv/bin/activate      # Mac/Linux
venv\Scripts\activate         # Windows

(venv) appears in your prompt once it’s active, and pip install now stays inside the project. Save what you installed so the container can install the same things:

pip freeze > requirements.txt

Tip — Vim. When you need to edit a file without leaving the terminal (for example inside a container that has no GUI), vim is usually available:

vim requirements.txt

• i — insert mode (start typing)
• Esc — leave insert mode
• :wq — save and quit
• :q! — quit without saving

The Dockerfile

A Dockerfile is a plain text file with the step-by-step recipe for building an image: what to start from, what to install, what to copy in, and what to run.

FROM python:3.11-slim
WORKDIR /app
COPY requirements.txt .
RUN pip install -r requirements.txt
COPY . .
CMD ["python", "main.py"]

• FROM — the base image to build on (here, a slim Python 3.11).
• WORKDIR — the folder inside the container where the next commands run.
• COPY — copy files from your machine into the image.
• RUN — run a command while building the image (e.g. install dependencies).
• CMD — the command that runs when the container starts.

Requirements are copied and installed before the rest of the code so Docker can reuse that cached layer — dependencies are only reinstalled when requirements.txt changes, which makes rebuilds fast.

.dockerignore

A .dockerignore file lists what Docker should skip when copying files into the image — like .gitignore, but for COPY . .. It keeps images small and keeps secrets out of them.

venv/
__pycache__/
.git/
.env
*.pyc

Typical things to ignore: your local venv, .git history, .env files with secrets, and cache folders.

Build the image

docker build reads the Dockerfile and produces an image. Run it from the folder that contains the Dockerfile:

docker build -t my-app .

• -t my-app — tag (name) the image; add a version with my-app:1.0.
• . — the build context: the current folder, whose files COPY can use.

Check that it exists:

docker images

Part 3 — Run and inspect containers

Run a container

docker run starts a container from an image:

docker run my-app

Common options:

docker run --rm -it my-app          # interactive terminal, auto-delete when it exits
docker run -d -p 8080:80 my-app     # run in background, map host port 8080 to container port 80
docker run --name my-container my-app

• --rm — remove the container when it stops.
• -it — interactive terminal (lets you type inside the container).
• -d — detached: run in the background.
• -p host:container — expose a container port on your machine.
• --name — give the container a readable name.

Monitor containers

docker ps                       # running containers
docker ps -a                    # all containers, including stopped ones
docker logs my-container        # output the app printed
docker logs -f my-container     # follow logs live (Ctrl+C to stop watching)
docker stats                    # live CPU / memory usage per container
docker inspect my-container     # full details (ports, env vars, mounts) as JSON
docker stop my-container        # stop it

docker logs is the first thing to check when a container exits or misbehaves.

Get inside a container

docker exec runs a command inside an already running container. With -it and a shell, it drops you in:

docker exec -it my-container bash

Use sh instead of bash if the image is minimal (e.g. Alpine). Type exit to leave — the container keeps running. Handy for checking files or seeing exactly what your app sees.

Part 4 — Several containers together: Docker Compose

Real projects usually need more than one container — say, your app plus a database. Docker Compose describes all of them in one docker-compose.yml and starts everything with one command, instead of typing a long docker run for each.

services:
  app:
    build: .
    ports:
      - "8080:80"
    environment:
      - DB_HOST=db
    volumes:
      - ./data:/app/data
    depends_on:
      - db

  db:
    image: postgres:16
    environment:
      - POSTGRES_PASSWORD=example
    volumes:
      - db-data:/var/lib/postgresql/data

volumes:
  db-data:

• services — each entry is one container (app, db).
• build / image — build from a local Dockerfile, or pull a ready-made image.
• ports — same as -p in docker run.
• environment — environment variables passed into the container.
• volumes — share a folder with your machine, or keep data after the container is removed (here, the database files survive a restart).
• depends_on — start db before app.

Services reach each other by name — the app connects to the database at host db.

docker compose up             # build (if needed) and start everything
docker compose up -d          # same, in the background
docker compose ps             # status of the services
docker compose logs -f app    # follow one service's logs
docker compose down           # stop and remove the containers

Part 5 — Sharing images: Docker Hub

Docker Hub (hub.docker.com) is a public registry of images — think GitHub, but for Docker images. Every FROM python:3.11-slim or image: postgres:16 above was pulled from there.

• Official images — maintained and verified (python, postgres, nginx, ubuntu); the safe starting point.
• Tags — versions of an image (postgres:16, python:3.11-slim). With no tag you get latest, which changes over time — pin a version for reproducibility.
• Your own images — with a free account you can push them, so teammates and servers pull the exact same image.

docker pull postgres:16                      # download an image
docker login                                 # sign in to Docker Hub
docker tag my-app your-username/my-app:1.0   # name it under your account
docker push your-username/my-app:1.0         # upload it

Company images usually live in a private registry (AWS ECR, GitHub Container Registry) — same commands, different address.

Cheatsheet

I want to…	Command
Build an image	docker build -t my-app .
List images	docker images
Run a container	docker run --rm -it my-app
Run in the background with a port	docker run -d -p 8080:80 --name my-container my-app
List running / all containers	docker ps / docker ps -a
Read or follow logs	docker logs my-container / docker logs -f my-container
Live CPU / memory	docker stats
Get a shell inside	docker exec -it my-container bash
Stop a container	docker stop my-container
Start all services in Compose	docker compose up -d
Stop and remove them	docker compose down
Download / upload an image	docker pull <image> / docker push <image>

Next steps

Continue with Just Enough Python — the language you’ll use in containers, notebooks, and pipelines throughout the course.