Free Handbook · Runs in your browser

Files & Data

Reading and writing files with the with statement, walking directories with pathlib, CSV in and out, JSON in and out, and the two errors every data script hits — FileNotFoundError and the encoding crash. The browser here has a real filesystem, so every example actually writes and reads a file.

0 / 108 lessons🔥 0 day streak
ShareXLinkedIn

Module 06 · what you'll be able to do

  • Open, read, write and append files safely with with
  • Use pathlib.Path instead of string paths
  • Read and write CSV with the csv module, including DictReader
  • Load and dump JSON, and know what does not survive the trip
  • Diagnose FileNotFoundError and UnicodeDecodeError from the traceback
01

Reading and writing files

open() returns a file object; with guarantees it is closed when the block ends, even if an error is raised inside. Never call open() without with — an unclosed file is a leaked handle and, on Windows, a locked file. Modes: "r" read (default), "w" write (truncates!), "a" append, "x" create-or-fail; add "b" for bytes.

pythonEdit it. ⌘/Ctrl + Enter runs.
You should see
'first line\nsecond line\nthird line\n'
1 first line
2 second line
3 third line
['first line\n', 'second line\n', 'third line\n']
Your turn
Count the words in notes.txt without reading the whole file into memory (loop over lines, split each).
Always pass encoding="utf-8"
The default encoding depends on the operating system's locale. The same script that works on your Mac will crash on a Windows server the first time it meets an accented character. Say utf-8 every time and the problem never exists.
Error you will hit

FileNotFoundError: [Errno 2] No such file or directory: 'data/orders.csv'

python
with open("data/orders.csv") as f:
    print(f.read())
Traceback (most recent call last):
  File "your code", line 1, in <module>
    with open("data/orders.csv") as f:
FileNotFoundError: [Errno 2] No such file or directory: 'data/orders.csv'
Why the interpreter said that

A relative path is resolved against the current working directory — the directory you ran python from, not the directory the script lives in. Run the same script from a different folder and the file "disappears". The other causes: a typo, or writing to a directory that does not exist yet (open creates files, never directories).

The fix

Build paths from the script's own location, and create parent directories before writing.

python
from pathlib import Path

HERE = Path(__file__).parent if "__file__" in globals() else Path(".")
data = HERE / "data" / "orders.csv"
data.parent.mkdir(parents=True, exist_ok=True)
data.write_text("id,total\n1,9.99\n", encoding="utf-8")
print(data.read_text(encoding="utf-8"))
02

Paths and directories with pathlib

A path is not a string — it has a parent, a name, a suffix, and it joins with / the right way on every OS. pathlib.Path replaced os.path string-juggling in 2014 and every new codebase uses it. It also carries the quick file operations: read_text, write_text, exists, mkdir, glob.

pythonEdit it. ⌘/Ctrl + Enter runs.
You should see
project/logs/app.log | app.log | app | .log | project/logs
True True False
project/logs/app.log → boot ok
project/logs/db.log → connected
['README.md', 'logs']
project/logs/app.txt
c
Your turn
List every .md file under project with its size in bytes (p.stat().st_size).
TaskpathlibThe old way
Joinbase / "sub" / "f.txt"os.path.join(base, "sub", "f.txt")
Extensionp.suffixos.path.splitext(p)[1]
Exists?p.exists()os.path.exists(p)
Read allp.read_text()open(p).read()
Find filesp.glob("*.csv")glob.glob(os.path.join(p, "*.csv"))
Home dirPath.home()os.path.expanduser("~")
03

CSV — reading and writing

CSV looks simple and is not: fields can contain commas, quotes and newlines. Never split(",") a CSV line — use the csv module, which handles quoting. DictReader gives you each row as a dict keyed by the header, which is almost always what you want.

pythonEdit it. ⌘/Ctrl + Enter runs.
You should see
id,customer,total
1,"Ada, Inc.",19.99
2,Linus,5.0

{'id': '1', 'customer': 'Ada, Inc.', 'total': '19.99'}
{'id': '2', 'customer': 'Linus', 'total': '5.0'}
sum: 24.99
Your turn
Write a second CSV containing only the orders over 10, with the same columns.
newline="" and pandas
Pass newline="" when opening a CSV for the csv module, or Windows writes blank lines between rows. And once a file is bigger than a screen or you need types, groups and joins, reach for pandas (pd.read_csv) — Module 14.
04

JSON

JSON is the language of APIs and config files, and it maps almost one-to-one onto Python: objects ↔ dicts, arrays ↔ lists, strings, numbers, true/false/nullTrue/False/None. loads/dumps work on strings; load/dump work on files. What does not survive: tuples become lists, dict keys become strings, and dates, sets and your own objects need converting.

pythonEdit it. ⌘/Ctrl + Enter runs.
You should see
{
  "name": "Ada",
  "langs": [
    "python",
    "sql"
  ],
  "active": true,
  "manager": null,
  "42": "int key"
}
['python', 'sql'] list
42 is now a str
Ada
{"when": "2026-01-01"}
Your turn
Parse '{"items": [{"sku": "A1", "qty": 2}, {"sku": "B2", "qty": 5}]}' and print the total quantity.
Error you will hit

json.decoder.JSONDecodeError: Expecting property name enclosed in double quotes

python
import json
json.loads("{'name': 'Ada'}")
Traceback (most recent call last):
  File "your code", line 2, in <module>
    json.loads("{'name': 'Ada'}")
  ...
json.decoder.JSONDecodeError: Expecting property name enclosed in double quotes: line 1 column 2 (char 1)
Why the interpreter said that

JSON is stricter than Python: keys and strings must use double quotes, there are no trailing commas, and True/None are spelled true/null. This error almost always means you printed a Python dict with str() and tried to parse it back — or an API returned an error page (HTML) instead of JSON.

The fix

Produce JSON with json.dumps, never with str(dict). When parsing untrusted input, catch the error and report the offending text (Module 07).

python
import json
print(json.loads('{"name": "Ada"}')["name"])
05

Text, bytes and encodings

A file on disk is bytes. Text is what you get after decoding those bytes with an encoding — almost always UTF-8 today, but Windows exports and old systems still produce Latin-1 and Windows-1252. Python 3 keeps str (text) and bytes strictly apart, so the mistake shows up as an error instead of as garbled text later.

pythonEdit it. ⌘/Ctrl + Enter runs.
You should see
b'caf\xc3\xa9 \xe2\x82\xac5' 7 10
café €5
b'caf\xe9 ?5'
UnicodeDecodeError: invalid continuation byte at byte 1
résumé
Error you will hit

UnicodeDecodeError: 'utf-8' codec can't decode byte 0xe9 in position 1: invalid continuation byte

python
with open("legacy.txt", encoding="utf-8") as f:
    print(f.read())
Traceback (most recent call last):
  File "your code", line 2, in <module>
    print(f.read())
UnicodeDecodeError: 'utf-8' codec can't decode byte 0xe9 in position 1: invalid continuation byte
Why the interpreter said that

Byte 0xe9 is é in Latin-1 but, in UTF-8, it announces a multi-byte character and the byte after it does not fit — so the decoder stops. The file is not corrupt; it was written with a different encoding than you are reading it with.

The fix

Find out the real encoding (ask the source; try latin-1 or cp1252) and pass it. As a last resort, errors="replace" substitutes � rather than crashing — acceptable for logs, not for data you keep.

python
with open("legacy.txt", encoding="cp1252") as f:
    print(f.read())

Finish the Python handbook, then get hired

Sit the exam for your certificate, run your resume through the ATS checker, and see the jobs that ask for exactly this.

Check my resume
Found this course useful? Share it.
ShareXLinkedIn

Comments

0

Join the conversation. Sign in to leave a comment — we'd love to hear your thoughts.