Data Engineering advanced ⏱ 15–20 hours

End-to-End ETL Pipeline on AWS

Build a complete ETL pipeline on AWS: pull from Adzuna API, transform, and load into a data warehouse with full orchestration.

AWSETLPythonAPIData Pipeline

View project on GitHub

What you’ll build

A production-style ETL pipeline that fetches real job listing data from the Adzuna API, transforms it in Python, and loads it into a cloud warehouse on AWS. Covers the full stack from API credentials to queryable data.

Skills you’ll practice

Working with REST APIs: authentication, pagination, rate limiting
Python data transformation and cleaning
AWS services: S3, Lambda/EC2, and warehouse integration
End-to-end pipeline orchestration and error handling

The full project guide is for members

Join Surfalytics to unlock every step of this project, all other projects, the full course, and the private community.

Join Surfalytics — Start free trial See pricing

7-day free trial · Cancel any time