Free Interactive Course · Apache Iceberg

Snapshots & Time Travel

Every write to an Iceberg table creates an immutable snapshot — a complete, named version of the table. That single idea gives you free time travel: query the table exactly as it was minutes, hours, or commits ago, roll back a bad load, and audit what changed. Here's how snapshots work and every way to read an old one.

The tables

We track the same db.orders Iceberg table as it's built up over two commits. The 6-row orders you see is the current (latest) snapshot; an earlier snapshot held only the first 5 rows.

customerscustomer_id, name, country, tier, signup_date
ordersorder_id, customer_id, product, category, amount, qty, status, order_date
2.0

Every write is a snapshot

When you INSERT, MERGE, DELETE, or OVERWRITE an Iceberg table, the engine doesn't edit files in place. It writes new files and commits a new snapshot — a full, immutable picture of which data files make up the table at that instant — then atomically moves the table's pointer to it. Old snapshots stay valid until you explicitly expire them.

So the history of db.orders below isn't a log you have to replay — each snapshot is a queryable version of the whole table. Snapshot s1 was the initial load of 5 orders; s2 appended the 6th. The current table is s2:

Spark SQL
-- The current (latest snapshot) state of the table
SELECT order_id, product, status, order_date
FROM db.orders
ORDER BY order_id;
Run it live in the browser — edit the SQL and press Run
SQL
⌘/Ctrl + Enter
Because a commit is a pointer swap over immutable files, snapshots are cheap — they share unchanged data files rather than copying them. An append that adds one file creates a new snapshot that references all the old files plus the one new file. That's why keeping thousands of snapshots costs metadata, not duplicated data (until you rewrite files in maintenance).
2.1

The .snapshots and .history tables

Iceberg exposes the version log as ordinary tables you can query — db.orders.snapshots and db.orders.history. This is where you find the snapshot_id values you'll time-travel to. (These are Iceberg metadata tables, so they're shown here as baked output — the browser engine can't materialise them.)

Spark SQL
SELECT committed_at, snapshot_id, parent_id, operation
FROM db.orders.snapshots
ORDER BY committed_at;
db.orders.snapshots — two commits; s2's parent is s1
committed_atsnapshot_idparent_idoperationsummary.added-records
2025-07-20 09:14:07.5122937501846267394851NULLappend5
2025-07-24 16:42:33.90751638472901837465222937501846267394851append1
baked output — Iceberg-only surface (snapshot / metadata state), not runnable in the browser engine
.history is similar but tracks the sequence of current snapshots (including rollbacks), while .snapshots lists every snapshot with its operation and a summary map (added/deleted records and files). Reach for .snapshots to pick a version, and the richer metadata tables (Module 8) to debug file counts and sizes.
2.2

Time travel by snapshot — VERSION AS OF

The headline feature. Append VERSION AS OF <snapshot_id> to a query and you read the table exactly as it was at that commit — no restore, no copy, just a different pointer.

Syntax
SELECT * FROM db.table VERSION AS OF <snapshot_id>;
SELECT * FROM db.table VERSION AS OF '<branch-or-tag-name>';
-- also spelled: FOR SYSTEM_VERSION AS OF <snapshot_id>

Query the table at the earlier snapshot s1 — before the USB-C Cable order existed. Same table, older version, five rows:

Spark SQL
SELECT order_id, product, status
FROM db.orders VERSION AS OF 2937501846267394851;   -- s1
db.orders VERSION AS OF s1 — the table before the 6th order was appended (5 rows)
order_idproductstatus
1Wireless Mousedelivered
2Notebookshipped
3Mechanical Keyboarddelivered
4Desk Lamppending
5Coffee Mugcancelled
baked output — Iceberg-only surface (snapshot / metadata state), not runnable in the browser engine
This is how you recover from a bad write instantly: something corrupted the table at 16:42? Read VERSION AS OF the snapshot just before it to see the good data, then roll back to it. No backup restore, no downtime — the good version never went anywhere.
2.3

Time travel by clock — TIMESTAMP AS OF

Don't have a snapshot ID handy? Travel by wall-clock time instead. TIMESTAMP AS OF resolves to the snapshot that was current at that moment.

Spark SQL
-- "How did orders look last Monday evening?"
SELECT count(*) AS n_orders
FROM db.orders TIMESTAMP AS OF '2025-07-21 00:00:00';
At 2025-07-21, snapshot s1 was current → 5 orders (s2 was committed on the 24th)
n_orders
5
baked output — Iceberg-only surface (snapshot / metadata state), not runnable in the browser engine
The timestamp is matched against each snapshot's committed_at; Iceberg picks the most recent snapshot at or before your timestamp. Ask for a time before the table's first commit and you get an error, not an empty table — there was no version to read yet.
2.4

Time travel from the DataFrame API

The same two dimensions — snapshot ID and timestamp — are read options on the DataFrame reader, handy inside a Spark/PySpark job:

Spark SQL
// by snapshot id
val past = spark.read
  .option("snapshot-id", 2937501846267394851L)
  .table("db.orders")

// by timestamp (epoch millis)
val asOf = spark.read
  .option("as-of-timestamp", 1753056000000L)
  .table("db.orders")
Use snapshot-id / as-of-timestamp for a fixed point read, and the SQL VERSION AS OF / TIMESTAMP AS OF for ad-hoc queries. Both hit the exact same snapshot machinery — pick whichever fits where you're writing the code.
2.5

Incremental reads — just what changed

Because snapshots form a chain, you can ask Iceberg for only the rows appended between two snapshots — the foundation of incremental pipelines (process new data without rescanning the whole table).

Spark SQL
// rows added strictly after s1, up to and including s2
val newRows = spark.read
  .option("start-snapshot-id", 2937501846267394851L)  // s1 (exclusive)
  .option("end-snapshot-id",   5163847290183746522L)  // s2 (inclusive)
  .table("db.orders")

newRows.show()
Incremental read s1→s2 — only the 1 row appended in s2
order_idproductcategorystatus
6USB-C CableElectronicsshipped
baked output — Iceberg-only surface (snapshot / metadata state), not runnable in the browser engine
Incremental reads currently surface appends between snapshots (not the effect of deletes/overwrites), which is exactly what you want for append-only ingestion. Pair it with a checkpoint of the last-processed snapshot_id and each run picks up only the new data.
2.6

Mission: find the newest orders

M
MiaAnalyst · Growthnow
For the freshness check on the orders table, pull me the 3 most recent orders — just order_id and order_date, newest first. This is the tail of the latest snapshot, so it's what our incremental job should have just picked up. 📈
🎯 Your mission From orders, return order_id and order_date for the 3 most recent orders by order_date, newest first.
SQL
⌘/Ctrl + Enter

Frequently asked questions

What is a snapshot in Apache Iceberg?
A snapshot is an immutable, complete version of an Iceberg table at the instant of a commit — it records exactly which data files make up the table then. Every write (INSERT, MERGE, DELETE, OVERWRITE) produces a new snapshot and atomically moves the table pointer to it, while older snapshots remain queryable until they are expired. Snapshots share unchanged data files, so keeping many of them costs metadata rather than duplicated data.
How does time travel work in Iceberg?
Because every version of the table is kept as a snapshot, reading an old version is just following an older pointer. In Spark SQL you append VERSION AS OF or TIMESTAMP AS OF '' to a query, or set the snapshot-id / as-of-timestamp read options in the DataFrame API. There's no restore or copy — you query the historical files directly.
What is the difference between VERSION AS OF and TIMESTAMP AS OF?
VERSION AS OF reads a specific snapshot by its ID, which you get from the table's .snapshots metadata table. TIMESTAMP AS OF '' reads whichever snapshot was current at that wall-clock time — Iceberg picks the most recent snapshot committed at or before the timestamp. Use VERSION AS OF when you know the exact commit, TIMESTAMP AS OF when you only know roughly when.
How do I see the history of an Iceberg table?
Query the table's metadata tables: db.orders.snapshots lists every snapshot with its snapshot_id, parent_id, operation and a summary of added/deleted records, and db.orders.history tracks the sequence of current snapshots including rollbacks. These give you the snapshot IDs to use for time travel and rollback.
How do I do an incremental read in Iceberg?
Set start-snapshot-id (exclusive) and end-snapshot-id (inclusive) read options on the DataFrame reader to get only the rows appended between those two snapshots. Combined with a checkpoint of the last-processed snapshot_id, this lets an incremental pipeline process only new data without rescanning the whole table. Incremental reads surface appends between snapshots.

Ready for the full data stack?

Iceberg is the table format at the heart of the lakehouse. When you want the pipelines, Spark, Kafka, and orchestration around it — plus a full end-to-end project — the Data Engineering course is free too.

Explore Data Engineering →

Comments

0

Join the conversation. Sign in to leave a comment — questions and feedback welcome.