Data Engineering intermediate ⏱ 12–15 hours
Data Ingestion Patterns: API to Snowflake
A 6-week deep dive into multiple data ingestion approaches using the GitHub API as a source and Snowflake as the target.
SnowflakePythonAPIIngestionAirflow
View project on GitHub
What you’ll build
Six different ingestion approaches to load the same GitHub API data into Snowflake: raw Python, Airbyte, Fivetran, custom connectors, and Airflow orchestration. By the end you understand the trade-offs behind every ingestion decision.
Skills you’ll practice
- REST API ingestion: pagination, auth, rate limits
- Multiple Snowflake load patterns: PUT + COPY, Snowpipe, Connector
- Comparing ELT tools: Airbyte vs custom vs managed connectors
- Incremental ingestion and deduplication strategies
The full project guide is for members
Join Surfalytics to unlock every step of this project, all other projects, the full course, and the private community.
7-day free trial · Cancel any time