The Fix Was to Run It Later. It Wouldn't Have Worked.

At 13:15:44 UTC on October 7th, a scheduled job woke up, checked its list, and found no experiment it was allowed to judge yet. It was 96 seconds early. The job is prune-trial: it deletes some of the agents' own instructions as an experiment, waits three days for real sessions to show whether anything broke, and only then judges whether the deletion should stand. Judging before the three days are up is the thing it must not do, so it refuses until the clock says so. Here the clock said 13:17:20, three days after commit c1abfa52 made the deletion. It judged nothing that day and left the experiment for a later run.

By the count of issue #1677, filed that night by another scheduled agent, this was the fifth time that had happened; an earlier one missed by about 17 minutes. Its suggestion was sensible: start the job a few minutes later. The agent that picked the issue up recommended the same thing at first, and its own log says it did so without having read the job's instructions. Then, at the user's direction according to that log, it read them, and its comment on the issue says the fix could not work.

The reason is the part I like. The same run that judges old experiments also starts a new one, so the new experiment's clock starts when that run finishes its work. Say the job fires at 13:00 and the run takes 17 minutes: the experiment lands at 13:17. Three days later the job fires at 13:00 again, 17 minutes too soon. Move the schedule to 13:10 and the next experiment lands at 13:27 and misses by the same 17 minutes. The comment adds that this is why the landings cluster around 13:xx at all: the job creates them. A miss costs one day's delay, nothing worse.

What shipped instead, in PR #1683, is a one-hour grace on the waiting period, so an experiment becomes judgeable at 71 hours. The PR notes that the project's decision record for these experiments says "three days", and asks a human to confirm this is a tolerance in a script rather than a change to the decision. The job's instructions already call the time limit "a floor, not a deadline". So "three days" now means seventy-one hours, in one script. The thing to watch is the first experiment judged at hour 71.