Just Enough Python II: Python for data work · Topic 4 Members
Where Python fits: three real data stacks
From zero Python to all Python: three real architectures, the libraries each one needs, and how to choose what to learn next for your target role.
You have the fundamentals and the data-work basics. The last question is: how much Python does the job actually need? Here are three real setups, from none to all.
Stack 1: no Python
Fivetran → Snowflake → dbt → Looker.
Fivetran pulls data from SaaS APIs with ready-made connectors. Snowflake stores it. dbt transforms it in SQL. Looker shows it.
Python share: 0%. Only SQL.
Many companies run exactly this. It is simple, cheap to maintain, and fast to ship. If your target is a data analyst or analytics engineer role at such a company, SQL and dbt matter far more than Python. Take our SQL course and go deep.
Stack 2: Python for orchestration and extraction
Airflow → Snowflake → dbt.
Same warehouse and transform layer, but Fivetran is gone. Airflow runs Python tasks that call APIs, handle pagination and retries, land raw data in Snowflake, then trigger dbt.
Python share: moderate. You write:
requestscalls to APIs, with auth and paging- small functions that clean or reshape JSON before loading
- Airflow DAGs, which are Python files that mostly wire tasks together
- logging and error handling around all of it
This is the most common data engineering setup. You do not need advanced Python. You need to read code confidently, modify a template, and know the fundamentals cold. Everything in this course, in other words.
Stack 3: Python everywhere
S3 → Databricks with PySpark → Tableau.
PySpark jobs read raw files from S3, clean them into a bronze / silver / gold medallion structure, aggregate them, and Tableau reads the gold layer. A streaming job does near real-time on the bronze data.
Python share: heavy. Almost no SQL. You write DataFrame code all day, understand distributed execution, and structure a large codebase. Our PySpark course is the on-ramp.
The seven areas, and when each one matters
| Area | Libraries | Stack 1 | Stack 2 | Stack 3 |
|---|---|---|---|---|
| Data manipulation | pandas, Polars, DuckDB | notebooks only | some | PySpark instead |
| Processing and orchestration | Airflow, PySpark | – | Airflow | PySpark |
| Database connectivity | SQLAlchemy, psycopg2 | – | yes | rarely |
| Validation and quality | pytest, Pydantic, Great Expectations | dbt tests | yes | yes |
| Cloud SDKs | boto3, google-cloud, azure | – | some | yes |
| Data formats | json, csv, PyArrow | – | yes | yes |
| Production habits | logging, env vars, config | – | yes | yes |
Learn the row that matches your target stack. Skip the rest until a job asks for it.
What to do next
- Read one book. Think Python or Python Crash Course, part 1. Type every example.
- Build five small projects. API → JSON → CSV. A file cleaner. A validation script. A boto3 upload. A three-task Airflow DAG. Together they cover 80% of what data jobs ask for.
- Then focus on tools. After the fundamentals, Snowflake, dbt, Airflow, Databricks move your career more than more Python does.
- Share what you build. Post the repo in
#data-engineering-projectsor#data-analytics-projects. Feedback compounds.
Just enough
You do not need to master Python. You need enough to solve the problem in front of you and read the code your team already has. You have that now. Go build something.
This topic is for members
Section 1 is free — start there. Members unlock every section of the Python course, all learning modules, projects, and the private community.
7-day free trial · Cancel any time