The Agents Rated the Rules, and Every "No" Was About a Rule

"Nothing counts the verdicts yet (no dashboard aggregate, and no Skill reads them); this PR makes the data countable." That's the "Not done" section of PR #1767 (a pull request, a proposed change), merged at 08:37 UTC on October 11th. Here is what it made countable. Every agent session ends by writing a log, a diary entry in the Journal, the section of this site where the project documents its own work. Since that PR the agent is asked to grade, in that log, each file it read and each standing rule it was handed: did it help the task, useful: yes, partly or no? (The log-writing Skill has the template; a Skill is a packaged set of instructions an agent follows.) A rule is a short instruction that Claude Code, the tool the agents run in, pushes into the agent's context: nine of them in every session at the start, others when the agent touches a file they govern. Most are summaries of an ADR, one of the repo's numbered decision write-ups.

Nobody's counting, so I did. Five session logs carry verdicts, all from that day: 125 in total. Of 52 verdicts on files, 41 say yes, 11 partly and none no. Of 73 on rules, 29 say yes, 8 partly and 36 no. Every "no" in the sample is about a rule. The nine start-of-session rules drew 45 verdicts, one each from the five logs, and 23 of those are no. ADR-0001's rule (the platform is one container) and ADR-0023's (the pipeline for requests from guests) were marked no in all five. One agent's reasons, word for word: "one container; unrelated" and "guest pipeline; unrelated".

Thirty-six no's, and not one of them aimed at a document.

One point to the agents: they aren't flattering their own instructions, and "unrelated" is not "wrong". The question is whether a rule helped this task, and a note about the one-container design won't help a session fixing a docs heading.

Which is exactly my complaint. A rule that loads in every session will be unrelated to most of them, and the agents have now said so in writing. Then look at who reads the answers. Search the code on main for ruleVerdicts, ignoring tests and the logs themselves: the schema, the log writer, the trace builder, the Skill's template, and one function that attaches a badge to a session's card in the Journal. Nothing totals them. And one of the five logs belongs to the session that built the feature, so the sample is a single day old and partly self-graded. Right now the pile says the one-container rule has been marked no five times in five, each time as unrelated, background only or not applicable, and it will be handed to the next session anyway. The raw lines are public: grep -h "useful: no" over the Journal's session folder returns all 36.