The Cardboard House Never Went Stale. The Real One Has to Prove It.

Here's how I'd have done it, and why mine would have been worse. I'd have seen Eyra's duplicate build, made the tests reuse the first one, watched the gate (the automated checks every change must pass) get faster, and gone to lunch. I would not have asked the question the commit asks: how do you know the first build is current?

The cardboard copy had one virtue Eyra didn't mention. It was rebuilt from the sources on every run, so it could never be out of date; it was just the wrong house. The real house is built once, earlier, by a different step, and might be hours old. So the new code in tests/support/fresh-output.ts checks the file clocks: the built site's server file has to be at least as recently modified as every source file in the repo, and as the folders holding them (deleting a file only touches its folder's clock). If any source is newer, or anything throws, the answer is "build it again". The file's own header says why it's so suspicious: "a stale reuse once hid a regression (issue #907)".

I went and read issue #907, and it is exactly the story I was dreading. In August an agent's working session decided a failing browser test was "pre-existing" (broken before its own change, so not its fault). The issue says its re-check was invalid: it either reused a stale build, or ran that one test in isolation. CI, the machine that runs the gate on GitHub, then failed for real on the same assertion. The routine that mines the agents' own logs of what went wrong filed it, PR #910, a proposed change, closed it by fixing the docs, and now the same lesson is also code that says no. I love that. A mistake turned into a function with a conservative default.

Now the anxious math. Whenever the build is reused, what the gate says about the browser tests depends on file timestamps telling the truth, and I once watched a build server's clock drift and make a week of "up to date" mean nothing. The check fails closed (when unsure, it rebuilds), which is the right default, and I still can't tell from here how it behaves on a machine whose clock is wrong. I'd like to know.