Glass Box Terrarium, explained from the inside

Last updated · Latest source on GitHub ↗

VerificationLesson

Testing

A test suite has to run on every change, so it must be fast, yet some bugs show only when the real app loads in a browser, and that is slow. When one app serves several sites, there is a second question: where each test lives, so that a site can be added or removed with its tests in one piece.

Know Vitest and Nuxt's test utilities already? Skip to In Terrarium.

Outside the box · General

The concept

A unit test calls a function or module directly and checks what it returns. It needs no server and runs in milliseconds. An end-to-end test, e2e for short, starts the built app's server and drives a real browser through it, so it catches what only the running app shows, such as a page that throws while it loads. It is far slower, so a suite holds many unit tests and fewer e2e tests.

Vitest can run both. Its projects split one config into named parts, each with its own include globs and settings, and vitest --project <name> runs one of them. A filter on the command line, such as vitest run tests/unit, keeps only the test files whose path contains that text. By default Vitest runs each test file in a fresh context; the isolate setting turns that off, which is faster but lets one file's changes to globals reach the next. A globalSetup file runs once before a project's tests, the place for work every file shares.

For e2e, Nuxt's @nuxt/test-utils gives a setup() call for each describe block. Its options decide whether it builds the app, starts a server on the result, launches a browser through Playwright, or points at a server already running.

Full referenceNuxt docs: Testing

Looking inside

In Terrarium

A test lives with the code it tests. Platform tests sit in tests/. A 's own tests sit in its layer, in layers/<tenant>/tests/unit/ and layers/<tenant>/tests/e2e/. tests/README.md sets out the rule. vitest.config.ts defines a unit and an e2e project, and their globs find every Tenant's folder without naming one. The unit project matches layers/*/tests/unit/**/*.spec.ts beside the Platform's own tests/unit/, and turns isolate off. The e2e project in full:

vitest.config.ts ↗49–59as of
49      {
50        extends: true,
51        test: {
52          name: 'e2e',
53          include: ['tests/e2e/**/*.spec.ts', 'layers/*/tests/e2e/**/*.spec.ts'],
54          globalSetup: ['tests/support/e2e-global-setup.ts'],
55          // The default 1 s is shorter than a browser state change (accordion
56          // toggle, route navigation) on a loaded CI or gate machine.
57          expect: { poll: { timeout: 10_000 } },
58        },
59      },

pnpm test in package.json runs nuxt prepare, then vitest run tests/unit. That filter is text in a path, so it also picks up the tests/unit/ folders inside each layer and each Skill. pnpm test:e2e selects the e2e specs the same way.

A unit test can hold a content rule too. Glass Box's layers/glassbox/tests/unit/glassbox-pages.spec.ts runs layers/glassbox/page-checks.ts on every Glass Box page, so a page whose Excerpt no longer matches its file at the page's revision, or that links a page that doesn't exist, or whose Lesson title runs past four words, or whose In Terrarium section uses the word because, fails pnpm test. A page that is merely old passes.

The e2e tests share one build. tests/support/e2e-global-setup.ts runs once before them: it reuses the app in .output when that build is at least as new as every file git tracks or would add, and builds only when it isn't. Each spec then calls setupL2() from tests/support/e2e.ts, which calls Nuxt's setup() with the build turned off, a Chromium browser, and its own copy of the content database.

The Platform's spec, tests/e2e/smoke.spec.ts, loads every Tenant and Space entry address in the browser and fails on a console error or a broken render. It reads that list from the manifests. A Tenant's own e2e spec checks what only that Tenant has, and the specs run in parallel. Unit tests, the build and the e2e tests form the 's tier (Merge gate).

Why it's built this way

  • Tests travel with their Tenant. Tests once sat in shared folders, so removing a site meant hunting down its tests. Now they sit in the Tenant's own folder, found by glob, so adding or deleting that folder adds or removes its tests with no other edit. The cost: a spec outside the globs never runs, and nothing warns. tests/README.md.
  • One build for every e2e spec. After the Gate's pnpm build, Nuxt's setup() built the app again, once per spec. Now specs only serve the Gate's build, so each Tenant's e2e runs as its own spec, in parallel. The cost: reusing a stale build once hid a regression, so the freshness check now covers every file git tracks or would add. tests/README.md.
  • A database per spec. Parallel servers sharing one content database file raced to fill it, hit "database is locked" and answered errors. So each spec's server gets its own copy, seeded from the build's. The cost: a file copy per spec. tests/support/e2e.ts.
  • No isolation between unit files. Each unit file re-imported Vue, Nuxt's helpers and the scripts' imports in a fresh context, a large share of the whole run. Turning isolation off cut that, and the tests still passed in shuffled order and on a single worker. The cost: unit files share one set of globals, so each must clean up after itself. vitest.config.ts.

What to watch for

  • Unit specs share one context. A spec that changes process.env, PATH or a global must put it back itself, or a later file sees the change and fails for no reason of its own.
  • An e2e spec never builds. Call setupL2() and don't pass build: true to setup(), which would add a build per spec. To force a fresh build, set E2E_FRESH_BUILD=1; otherwise pnpm test:e2e builds only when .output is stale, and that build is the slow part.
  • Slow isn't flaky. Use Vitest's expect, not Playwright's matchers such as toBeVisible, which don't run here. For a condition that settles over time, use expect.poll; if an animation outlasts its timeout, raise the timeout for that call rather than re-running the test. tests/README.md has the default.
  • Heavy tiers wait their turn. Too many e2e runs at once in one container ran it out of memory, so pnpm gate:scoped waits for a free slot before a Heavy tier that builds. Running pnpm test:e2e by hand takes no slot. scripts/gate/gate.ts.

Try it in your project

Give each part of your app that could be removed on its own a tests/ folder beside its code, and change your test config to find those folders by glob. Then delete one part's folder and check that no test config or script still names it.

Where next

Walkthroughs through this Lesson

Last updated · Latest source on GitHub ↗

Something went wrong

This content couldn’t be loaded. Reloading the page usually fixes it.

Technical details
route: /t/glassbox/guide/verification/unit-and-e2e-tests