Data formats: JSON and CSV

Data arrives in files and API responses. Two formats cover most of it. You need to read both, walk through them, and write them back.

JSON

JSON is how APIs talk. It maps directly onto Python: objects become dicts, arrays become lists, true becomes True, null becomes None.

import json

raw = '{"company": "Shopify", "offers": [{"role": "Data Analyst", "salary_usd": 95000}]}'

data = json.loads(raw)                   # text  -> Python (loads = "load string")
data["offers"][0]["salary_usd"]          # 95000

text = json.dumps(data, indent=2)        # Python -> text (dumps = "dump string")

Walking nested JSON is dict access and list indexing, one level at a time. When a key may be missing, use .get():

bonus = data.get("bonus", 0)

Files use the versions without the s:

with open("offers.json") as f:
    data = json.load(f)

with open("out.json", "w") as f:
    json.dump(data, f, indent=2)

The with block opens the file and closes it for you, even if something fails inside. Always open files this way.

CSV

CSV is what spreadsheets and most databases export. Rows of text, separated by commas. The csv module reads it as dicts, one per row:

import csv

with open("offers.csv") as f:
    rows = list(csv.DictReader(f))

rows[0]                                  # {'company': 'Shopify', 'salary_usd': '95000'}

Two things to remember:

1. Everything is text. '95000', not 95000. Convert with int() or float() before math.
2. Delimiters vary. European exports often use ;. Pass delimiter=";".

Writing:

with open("out.csv", "w", newline="") as f:
    writer = csv.DictWriter(f, fieldnames=["company", "salary_usd"])
    writer.writeheader()
    writer.writerows(rows)

In the playground there is no disk, so the exercises read CSV from a string with io.StringIO(text). It behaves exactly like an open file.

The typical pipeline shape

response = requests.get(url)             # 1. fetch
data = response.json()                   # 2. parse JSON
rows = [clean(r) for r in data["items"]] # 3. transform
save_to_database(rows)                   # 4. load

Fetch, parse, transform, load. Almost every Python data job is a variation of these four lines. requests is a package you install; the rest you already know.

Parquet: the format you will meet next

CSV is text and slow. Parquet stores columns in a compressed binary format. It is the default in data lakes, Spark, Snowflake, and BigQuery. You do not read it with the standard library. pandas or PyArrow do it in one line:

df = pd.read_parquet("offers.parquet")
df.to_parquet("out.parquet")

Format	Human-readable	Typed	Size	Where
CSV	yes	no, all text	large	exports, spreadsheets
JSON	yes	mostly	large	APIs, configs
Parquet	no	yes	small	data lakes, warehouses