Surfalytics

Free Interactive PySpark Course

Just enough PySpark for data roles

Real Python with the Spark DataFrame API runs on these pages. Every topic mirrors our free SQL course — same dataset, same tasks — so you translate skills you already have instead of starting from zero.

This is the API you will use as Spark in Databricks — and the same DataFrame style behind Snowpark in Snowflake, AWS Glue, and Microsoft Fabric.

Real Python, in-browser Run any DataFrame code Automatic answer checking
Start topic 1 — it's free New to SQL? Start there first

Note: the Python engine is a ~30 MB download on your first visit, then your browser caches it.

Course program

Every topic is theory plus hands-on exercises — the same tasks you solved in SQL, now as DataFrame pipelines. The first section is free.

DataFrame basics I

  1. 01 DataFrames and select() Meet the DataFrame: pick columns with select(), see all columns, and rename with alias(). Free
  2. 02 Expressions and lit() Compute new values inside select(): arithmetic on columns, literals with lit(), and joining strings with concat_ws(). Free
  3. 03 Built-in functions (F.*) Transform values with pyspark.sql.functions: upper, lower, length, and date parts with year() and month(). Free
  4. 04 Removing duplicates: distinct() Return each unique row once with distinct() and dropDuplicates(). Free
  5. 05 Filtering rows: filter() Keep only the rows you need with filter(): column comparisons and combining conditions with & and |. Free
  6. 06 NULLs, ranges, and isin() Handle missing values, filter by ranges, and check membership with isin(). Free
  7. 07 Pattern matching: like() Search text with like(), ilike(), startswith(), and contains(). Free
  8. 08 Regular expressions: rlike() Match complex text patterns with rlike() when like() is not enough. Free
  9. 09 Sorting: orderBy() and limit() Sort DataFrames with orderBy(), control direction with asc()/desc(), and take the top N rows with limit(). Free
  10. 10 Grouping: groupBy() and agg() Summarize data with groupBy().agg(): count, avg, sum, min, max — and filter groups like SQL's HAVING. Free

The practice dataset

The same dataset as the SQL course: students, cohorts, courses, enrollments, and job offers. If you did the SQL exercises, you already know these tables.

N 1 N 1 N 1 N 1 cohorts id PK INT name TEXT start_date DATE format TEXT students id PK INT name TEXT city TEXT country TEXT cohort_id FK INT background TEXT joined_at DATE courses id PK INT title TEXT category TEXT difficulty TEXT hours INT enrollments id PK INT student_id FK INT course_id FK INT enrolled_at DATE completed_at DATE score INT job_offers id PK INT student_id FK INT company TEXT role TEXT city TEXT salary_usd INT remote BOOLEAN offer_date DATE

How the tables join

Relationship Cardinality Join condition
students.cohort_id → cohorts.id many students → one cohort JOIN cohorts ON students.cohort_id = cohorts.id
enrollments.student_id → students.id many enrollments → one student JOIN students ON enrollments.student_id = students.id
enrollments.course_id → courses.id many enrollments → one course JOIN courses ON enrollments.course_id = courses.id
job_offers.student_id → students.id many job offers → one student JOIN students ON job_offers.student_id = students.id

In short: each student belongs to one cohort. Students enroll in courses through enrollments, and their job offers land in job_offers.

Frequently asked questions

Is this PySpark course really free?

Yes. The first section, DataFrame basics I, is completely free — no account needed. Later sections are part of the Surfalytics membership.

Do I need to install Spark or Python?

No. Real Python runs directly in your browser with the PySpark DataFrame API. You write code and see results instantly — nothing to download or set up.

Where will I use PySpark at work?

The PySpark DataFrame API is how you use Spark in Databricks. The same DataFrame style powers Snowpark in Snowflake, AWS Glue, and Microsoft Fabric — learn it once, use it everywhere.

Should I learn SQL or PySpark first?

SQL first. This course mirrors our free SQL course task by task, on the same dataset, so you can translate what you already know instead of starting over.

Want the full path to a data job?

SQL and PySpark are the start. Surfalytics members get the full curriculum, hands-on projects, mock interviews, and a private community.

See how membership works