Modules, packages, and virtual environments

You will not write everything yourself. You will import it. pandas, requests, boto3, Airflow: all of them are packages you install and import. This topic is how that works, and how to keep projects from breaking each other.

import

A module is a .py file. A package is a folder of modules. Python ships with a large standard library, ready to import:

import math
math.sqrt(144)                       # 12.0

from datetime import date, timedelta
date(2026, 1, 6) + timedelta(days=7)

import json
json.loads('{"a": 1}')               # {'a': 1}

import os
os.environ.get("DB_HOST")            # read an environment variable

Two styles. import math then math.sqrt(). Or from math import sqrt then sqrt(). Both fine. The convention for the big data libraries is an alias: import pandas as pd, import numpy as np.

Standard-library modules you will use in data work: json, csv, datetime, os, pathlib, logging, re, statistics, collections.

Your own modules

Any file you write is importable. Put clean_name() in utils.py, and in another file:

from utils import clean_name

That is how a folder of scripts becomes a small project.

Installing packages

Anything not in the standard library comes from PyPI, the Python package index:

pip install pandas requests

pip is the default installer. uv is a newer, much faster one that does the same job (uv pip install pandas). Poetry is a third option, common in data teams. Pick one, learn it, stay consistent.

Virtual environments: one per project

Here is the problem. Project A needs pandas 1.5. Project B needs pandas 2.2. Install both globally and one of them breaks. This is dependency hell.

The fix: every project gets its own isolated environment.

cd my-project
python -m venv .venv                 # create it (once)
source .venv/bin/activate            # turn it on (Mac/Linux)
.venv\Scripts\activate               # turn it on (Windows)
pip install pandas requests
pip freeze > requirements.txt        # record exact versions

Now pip install puts packages inside .venv, and nothing else on your machine is touched. Delete the folder, and the environment is gone.

requirements.txt lists the exact versions. A colleague, a CI server, or an Airflow worker runs pip install -r requirements.txt and gets the same setup. That is the goal: your code runs on any machine.

Tools you will hear about:

Tool	What it does
venv	Built in. Creates environments.
pip	Built in. Installs packages.
uv	Fast replacement for both. Growing fast.
Poetry	Both, plus dependency resolution. Common in data teams.
Conda	Both, plus non-Python packages. Common in data science.
pyenv	Manages several Python versions on one machine.

The rules

1. Every project, every GitHub repo, gets its own environment.
2. Never pip install into the system Python.
3. Pin versions in requirements.txt (or pyproject.toml).
4. Add .venv/ to .gitignore. The environment is rebuilt, never committed.