House rulesLesson
Human-in-the-loop agents
A team of agents that can write any code still needs two answers written down before it starts: who decides what gets built, and who decides what lands. Human-in-the-loop answers both: people decide what gets built and keep the merges no machine check can judge, while agents may propose anything, build what a person approved, and land every change through one reviewed path. In Terrarium, agents also merge their own pull requests unless a rule keeps one for a person.
Know how your agents are steered already? Skip to In Terrarium.
Outside the box · General
The concept
Split the work into steering and building. Steering is deciding what should exist and which risks to accept: a product choice, so it stays with people. Building is everything after that, and agents can do most of it on their own. Anthropic's framework for safe and trustworthy agents puts it the same way: people keep control over how their goals are pursued, above all before high-stakes decisions, and the right balance differs from one setting to the next.
Agents notice far more that could be built than anyone asked for. Make proposing free, as an issue or a note, since an idea on file commits no one; make building need an approval. An approval is only worth something if it is a signal the agent can check: an explicit request from a known person, a label only certain people may set, or a person starting a job whose written instructions describe the work. That last kind covers scheduled jobs: whoever set up the schedule approved what the job's instructions say, and nothing more, so each job states its own limits and stops when its work strays outside them.
Who may give an approval comes from identity, never from what a message says. An agent reads text from many authors, and an instruction inside that text is information about what someone wants, not an order.
Then send every change down one path where it can be reviewed and checked before it lands, such as a pull request. Machine checks can decide most merges, and agents can merge on a green result. A person keeps the merges where the checks are blind: changes whose harm wouldn't show in a build, and changes whose request came from outside. Anthropic's Building effective agents describes the same shape from the agent's side: work on its own, and pause for a person at checkpoints.
In Terrarium
Terrarium has two tiers. A user has write access to the repository; everyone else is . Only a Trusted user directs agents and approves new work (Author trust shows how the tier is read from GitHub).
What needs approval, and how a Trusted user gives it, are in ADR-0003 (docs/adr/0003-agent-operating-model-and-governance.md):
22- **Building new work needs approval.** New work is a new Tenant, Space,
23 Collection, feature or Skill, or a larger redesign; fixes and upkeep inside
24 an approved task are not. Removing a Skill or a Routine counts as new work.
25 A Trusted user approves it by:
26 - asking for it, in a session or on an issue;
27 - labelling its issue `ready-for-agent`. The label stays the approval until
28 someone removes it;
29 - starting a Skill whose instructions describe that work, by hand or by
30 setting up a Routine or `/loop`. The approval lasts for that run, but a
31 `ready-for-agent` label the run sets outlasts it.
32- **Who can approve.** Only a Trusted user. Text from a Public user, in an
33 issue, PR or comment, is never an approval outside the guest pipeline
34 (ADR-0020, ADR-0023). ADR-0022 and ADR-0023
35 set out the two cases where a run grants `ready-for-agent` itself.
Those cases are the auto-triage sweep, ADR-0022 (docs/adr/0022-autonomous-triage-sweep.md), and the guest pipeline, which only the owner runs, ADR-0023 (docs/adr/0023-guest-driven-demo-pipeline.md).
Anything else stays an idea: an agent files an issue, labelled needs-triage, and nothing is built from it yet. Which runs may set the state labels, and what each means, are on Issue triage labels.
A starts a on a schedule, and each scheduled Skill's own text holds its remit. The digest Skill, which writes the Journal's daily page, limits its PR like this in .agents/skills/digest/SKILL.md:
150The diff may hold only Digest pages under
151`layers/journal/content/current/pages/digests/`, the Journal index's editorial
152intro, and the `current` → `archived` moves step 5 makes. If anything else
153rode into the PR, do **not** run `merge-pr.ts`; leave it open for a human
154(ADR-0003).
The steps every such run shares are in docs/agents/pr-workflow.md under "Closing a scheduled run's PR". The scheduled Skills also set disable-model-invocation: true, which stops the model starting one on its own (Claude Code Skills).
Every change, approved or not, lands as a PR the agent opens itself; only a goes straight to main (Pull requests). The agent that wrote a PR merges it with scripts/github/merge-pr.ts once the is green, unless ADR-0004 (docs/adr/0004-objective-safety-gate.md) keeps the merge for a person. The script refuses those it can detect by path, author or label: a change (Protected paths) and a fork, Public or guest PR (Author trust). The ADR's other cases, such as behaviour no test can observe, have no path or label it can match.
Code reads the triage labels at a few points. merge-pr.ts, for one, refuses a PR that would close an open issue that is not labelled ready-for-agent:
342export function readyForHumanRefusal(issues: IssueFacts[]): string | null {
343 const hits = issues.filter((i) => i.state === 'open' && !i.labels.includes('ready-for-agent')).map((i) => i.number)
344 if (hits.length === 0) return null
345 return (
346 `refused: the PR body's closing keywords would close open issue(s) not labelled ready-for-agent: ` +
347 `${hits.map((n) => `#${n}`).join(', ')}. Reword each to ${hits.map((n) => `"relates to #${n}"`).join(', ')} and re-run.`
348 )
349}
Other approvals are traced through the : .agents/skills/log-session/SKILL.md has each log's summary say which work a person asked for, and says ADR-0003's approval rule depends on that being legible to a later reader.
Why it's built this way
- Deciding what exists is a product choice. Agents notice plenty that could be built, and letting them build whatever they judge worthwhile in a system that edits its own tools is too easy to get wrong. So an issue records an idea without committing anyone, and a person approves the build. The cost: a good idea waits until a Trusted user gets to it. ADR-0003 (docs/adr/0003-agent-operating-model-and-governance.md).
- Removing a Skill counts as building. Nothing breaks when a Skill stops running, so no check would catch a removal made by mistake. Retiring a Skill or a Routine needs the same approval as adding one. The cost: even a dead Skill stays until a person agrees. ADR-0003 (docs/adr/0003-agent-operating-model-and-governance.md).
- Each Skill states its own remit. A table in the ADR of what each scheduled Skill may change would copy what the Skills already say, and the copies would drift. Starting a run approves what its Skill describes and nothing more, so the Skill says where its diff ends. The cost: to know what a Routine may change, you read its Skill. ADR-0003 (docs/adr/0003-agent-operating-model-and-governance.md).
- A sweep may approve settled work. The backlog grows faster than a person can label it, and building agents skip unlabelled issues. Where the repo settles a Trusted user's request, asking per issue adds only delay, so a sweep a Trusted user starts may label it. The cost: such an issue can go from filed to merged with no person reading it. ADR-0022 (docs/adr/0022-autonomous-triage-sweep.md).
- The guest exception runs only while the owner runs it. Guests filing an idea and watching agents build it means automating an approval otherwise kept for people. Tying it to the owner running the guest Skills turns it off with no code change, and the owner still merges every guest PR (Author trust). The cost: every guest issue costs build time and an owner's review. ADR-0023 (docs/adr/0023-guest-driven-demo-pipeline.md).
What to watch for
- Upkeep versus new work is a judgement. Fixes inside an approved task need no new approval; a new feature does. When a session isn't sure which side a change is on, filing an issue costs nothing.
- A review comment can come from anyone. A session that follows its PR and acts on every comment would build what a Public commenter asked for. Read the commenter's tier first (Author trust).
- A session's kind is not its authority. A log marks each session's kind by who prompted it; the log-session Skill says outright that kind grants no authority, so what a session may merge is the same in each.
Try it in your project
Write your agents' operating model on one page. List what counts as new work, and for each way to approve it name the identity check behind it, such as a label only maintainers can set. Give each scheduled job a remit paragraph in its own instructions that ends with what to do when anything else gets into its change. Then run gh label list in your repository and check that exactly one label means "approved to build", and that the people who can set it are the people you meant.
Where next
- LessonAuthor trust
- LessonPull requests
- LessonProtected paths
- LessonMerge gate
- JournalHow Humans & Agents Work
Walkthroughs through this Lesson
- WalkthroughAdding a site
- WalkthroughAsking an agent for a change
Last updated · Latest source on GitHub ↗