release-diff — what changed between two renderings of the same site

One file, standard library only — no crawler to install, no headless browser, no account. Python 3.8 or newer; it was written and run here on Python 3.12, which is the only version it has been executed on.

It honours robots.txt on both hosts and skips any path either side disallows, reporting ROBOTS rather than guessing. It sends no cookies, runs no JavaScript, and writes nothing anywhere. Exit status is 0 when nothing moved that was not declared and 3 when something did, so it drops into a pipeline without a wrapper.

The run below is not a fixture. It is the release that published this page, run before it went out: live plantroomlabs.com as --old, the build waiting on disk as --new, over nine paths. Three of them moved. The home page and the tools hub both gained a link to this page, the hub's heading and description changed because thirteen tools became fourteen, and how this site is built restated its own two figures — 23,004 assertions to 23,346, and 268 deliberate sabotages to 271:

That is the tool doing its job on its author, and it is worth being plain about what it means: every one of those five changes was deliberate, and the gate failed the release anyway, because at that point none of them had been written down. Declaring them is five lines. The comment above each one is for the human reading the file in three months, and writing it is the part that catches the change nobody meant:

The same run, with that file passed as --expect:

Note what the second run still prints. Every changed field is still reported, with both values, and all three pages are still marked CHANGED — the fields read declared rather than UNDECLARED, and that is the only difference. A gate that hides a declared change teaches you to declare things to make it quiet.

The list of fields above is not ours. Asked what actually gets an outsourced developer dropped, an agency owner who buys development this way answered that it is not bad code: it is silent changes. A URL pattern that moved. A dropped canonical. An h1 turned into a styled div because it sat better in the layout. A noindex that never came off. A redirect tested on staging and never checked against the live box. An analytics tag quietly not firing. The site works perfectly afterwards; rankings slide three weeks later and by then nobody connects it to the release. It costs a client relationship rather than a bug ticket.

The same person named why a launch checklist does not fix it, and the reasoning is the whole design of this tool: a checklist only catches what somebody already thought to write down, and nobody writes down the failure they have not yet suspected. A diff has no such limit. It does not need anyone to decide in advance what matters — it reports everything that moved and asks you to account for it, which is the opposite direction of work.

That is also why this is eleven fields and not five hundred. Each one is a thing a release has been observed to break while looking fine, and the set is small enough that a report you cannot explain is worth reading rather than worth muting.

A correctly configured staging site points its canonical at production, and writes its internal links in its own host. Compare those two raw and you get a difference on every page, every time — at which point the gate is noise, and a noisy gate is a gate somebody turns off. So both base hosts are folded to a single token before anything is compared, and neither base host can itself be a finding.

A canonical pointing at a third host still is. That distinction is the difference between a tool that works on a real staging setup and one that only works on a copy of production, and it is the piece most likely to be removed by someone simplifying the comparison.

A redirect that names its own URL in Location is not a redirect. The browser follows it until it hits its own limit and then shows the visitor nothing, and no amount of comparing will surface it, because it is the same nothing on both sides. Every field matches, and the path reads SAME — the one shape a diff is blind to by construction. This is not hypothetical: reading every URL one UK manufacturer's sitemap submits, on 6 October 2026, found eleven of fifty-four answering 301 with their own address, one of them the privacy policy their home page links twice.

So it is reported absolutely rather than diffed. A path whose chain comes back to a URL it already requested is BROKEN, named per side, and it fails the run on its own — --expect cannot declare one, because no release is meant to contain one. The run below reads two paths on one host, where the second answers 301 to itself:

Both sides are the same host there on purpose. Had the loop been on one side only, the status field would have moved and an ordinary diff would have caught it; it is the symmetrical case that needs its own reading.

It reads the HTML the server sends. A change made by JavaScript after load is invisible to it, so a measurement tag that is present in the markup and fails to fire still reads as present: the field is the snippet is on the page, not the measurement arrived. If what you need is the second thing, this is the wrong instrument and a real browser is the right one.

Two renderings taken minutes apart can also differ for honest reasons — a date, a counter, a rotating testimonial. That is what --expect is for, and a field that always moves belongs in the declared file with a comment saying why, rather than removed from the tool.

It is not a crawler. It reads the paths you give it and discovers nothing, because the question at release time is whether these pages changed, and a crawl that finds different pages on the two sides cannot answer it.

A diff run over two identical trees reports nothing, and proves nothing, because the trees are identical. So the proof is deliberate sabotage, which is the same reading the mutation catalogue for this site rests on. The test below serves a copy of this site's released tree twice over loopback, applies each of the seven failures named above to one side on its own, and asserts the tool names the field that moved — plus the loop below, plus a control where the tree is compared against itself:

And the parsing underneath it has its own suite, over held bytes rather than over a network, covering the cases that are easy to get subtly wrong: an h1 read through inline markup, the same canonical written in two different hosts, a GA4 id inside a script body, a link to a third host not counted as internal, a link written by JavaScript not counted at all, an X-Robots-Tag read as robots, and the four readings of whether a chain of redirects has come back to a URL it already asked for.

Both of those are in the repository, not described in it. The seven-failure test needs a copy of a built site to break, so it ships with the instructions to point it at your own.

Published here so the download is checkable rather than trusted. Both figures are read off the file served at /tools/release-diff.py when this page is built, so they cannot disagree with it.

To check it, on Linux sha256sum release-diff.py, on macOS shasum -a 256 release-diff.py, on Windows certutil -hashfile release-diff.py SHA256. A different digest means a different file — not necessarily a hostile one, but not this one.

The repository holds the same file, byte for byte, together with everything needed to re-run the checks this page's claims rest on — so they can be run rather than read about. Issues and pull requests there are read.