Glass Box Terrarium, explained from the inside

Last updated · Latest source on GitHub ↗

House rulesLesson

Self-improvement loop

A team that never looks back at how it works keeps tripping on the same snags. Agents have it worse: each starts fresh, remembers nothing of the last run, and follows its instructions whether they help or not. A self-improvement loop closes that gap: report every problem, group the reports by cause, fix the causes that keep coming back, and check later whether each fix held.

Know feedback loops on a team's own process? Skip to In Terrarium.

Outside the box · General

The concept

Many teams hold a retrospective: at regular intervals they look back at what slowed them down and change one or two things about how they work. It works only when people report problems honestly and someone acts on the ones that keep coming back.

The same loop applies to agents, whose process is written down: the instructions, scripts and checks they work under. Each run reports the problems it hit. Someone groups the reports by cause, since one report is an anecdote and the same report from many runs is a pattern. The patterns worth fixing become changes to the instructions or tools, and later reports show whether a fix held.

A full loop also removes things. Fixes tend to add instructions, and a long set of instructions is followed less well, so the loop also tests an instruction by cutting it and watching whether the problem it was meant to prevent comes back.

Looking inside

In Terrarium

Every session lists its , the problems it hit, in its (Session logs). The frictions-to-fixes , .agents/skills/frictions-to-fixes/SKILL.md, runs the loop on them, started on a schedule by a .

Report and group. A subagent, a helper agent with its own context, reads the logs that ended since a short margin before the Ledger's last commit, through scripts/session/extract-frictions.ts, a script that gathers them as compact JSON. It files each Friction under an entry of the : one YAML file per root cause in layers/journal/content/current/frictions/, listing every session that hit it and every PR meant to fix it.

Fix. Each candidate is first checked against what is already fixed or tracked; .agents/skills/frictions-to-fixes/survey-brief.md holds those rules. The run picks a few, weighing how often and how badly each one hurt, and a subagent designs each fix and judges its risk. A fix that touches a file, a guard or the hook wiring, or that turns on a choice a person should make, becomes an issue. Helper agents build the rest on branches that land as PRs that pass the (Merge gate). The Ledger's own updates land in a PR every run, fixes or none.

Check whether it held. Nobody writes down by hand whether a fix worked. The Ledger page derives each entry's state from the dates of its merged fixes and of its later occurrences, in layers/journal/app/utils/ledger.ts:

layers/journal/app/utils/ledger.ts ↗9–13as of
9/** A cluster with merged PRs shows one of these, derived from its `fixes` and
10 *  occurrences so nobody has to maintain it; one without, or one `accepted`
11 *  (the owner accepts the cause as a cost, issue #2148), shows its stored status. */
12export type FixState = 'fix landed' | 'likely fixed' | 'still recurring'
13export type LedgerState = FixState | 'open' | 'tracked' | 'on trial' | 'accepted'

A cause that comes back after its fix merged is a regression: the run files an issue naming the fix that failed and doesn't try again.

Remove. The prune-trial Skill, .agents/skills/prune-trial/SKILL.md, runs a : it cuts one problem's instructions down to their goal, or rewrites one Skill or doc plainly, and lists the open trial in .agents/prune-trials.yml. Once the trial's window has passed, a later run reads the Frictions logged since the cut landed and judges each by its severity:

.agents/skills/prune-trial/SKILL.md ↗34–38as of
34- `major` or `blocker` damage that traces to the trial → restore the
35  **minimum**: the one goal-shaped line that would have prevented it, never the
36  wall that was there before.
37- `moderate` damage that traces to the trial → the prune holds; see below.
38- `nit`, `minor`, silence → the prune holds.

The run also compares each covered Friction's share of sessions before and after the cut, with .agents/skills/prune-trial/scripts/friction-rates.ts, since severity misses a cut that makes a small mistake more common. moderate damage, or a clear rise in a share, files an issue for a person to decide on instead of restoring text.

Frictions can't show a Skill that runs and quietly doesn't deliver, or one that should have run and never did. A second Skill, .agents/skills/audit-skills/SKILL.md, looks for both in the logs and records what it finds in the , the Journal's list of Skills with how much each is needed; retiring a Skill it only proposes.

Why it's built this way

  • One complaint, or a pattern. A cause that came back every few days hid inside each run's short reading window, and "already tracked" items were dropped run after run with nobody on them. The Ledger keeps one entry per cause, counting recurrence over weeks and keeping status between runs. The cost: every run must file each Friction under the right cause.
  • Facts by script, judgement by agent. An agent that parses the logs itself improvises a new parser each run. So a fixed script only gathers and sorts the logs, cheaply, and grouping, ranking and every fix stay the agent's call. The cost: the script's compact output leaves out each log's prose, so a candidate that needs it means reading the full log. scripts/session/extract-frictions.ts.
  • Derived state. A status someone must update by hand goes stale. So an entry with merged fixes gets its state from dates, not from a field. The cost: an entry with no merged fix still shows a stored status that a run keeps current. layers/journal/app/utils/ledger.ts.
  • A fix that failed goes to a person. When a Friction comes back after its fix merged, that change has already failed once and needs a person's eyes. So the run files an issue that names the failed fix instead of trying again. The cost: regressions wait for a person. .agents/skills/frictions-to-fixes/SKILL.md.

What to watch for

  • A merged fix isn't a working fix. An entry reads likely fixed only after a quiet spell with no new occurrence, and still recurring when Frictions come back.
  • A missing instruction may be on trial. If an instruction you expected is missing, it may have been cut in a : check .agents/prune-trials.yml before restoring it, as ADR-0027 (docs/adr/0027-prune-trials.md) asks.
  • Retiring a Skill is a person's call. A user approves removing a Skill or a Routine (ADR-0003 (docs/adr/0003-agent-operating-model-and-governance.md)).

Try it in your project

Keep a shared file where each agent session appends the problems it hit, with a possible fix. Once a week, group them by cause, fix the cause that appears most, and check the next week's problems to see whether the fix held.

Where next

Walkthroughs through this Lesson

Last updated · Latest source on GitHub ↗

Something went wrong

This content couldn’t be loaded. Reloading the page usually fixes it.

Technical details
route: /t/glassbox/guide/terrarium-practice/self-improvement-loop