test(e2e): per-test capture + custom HTML result index

.gitignore

@@ -147,6 +147,7 @@ build/
 # Playwright
 test-results/
 playwright-report/
+e2e-results/
 playwright/.cache/
 e2e/.auth/
 

docs/superpowers/specs/2026-06-11-e2e-result-capture-index-design.md

@@ -0,0 +1,153 @@
+# E2E Per-Test Capture + Custom Result Index — Design
+
+**Date:** 2026-06-11
+**Status:** Approved (design); pending implementation plan
+**Scope:** Local-only tooling. No CI, backend, or app-code changes.
+
+## Problem
+
+The Playwright suite (182 tests) currently captures artifacts **only on failure**
+(`screenshot: 'only-on-failure'`, `video: 'retain-on-failure'`, `trace: 'on-first-retry'`).
+We want a visual record of **every** test case — passing and failing — usable for
+debugging, as a shareable deliverable, as living documentation, and as run evidence.
+
+## Goals
+
+- Capture video + screenshot + trace for **every** test (max fidelity, accepted cost).
+- Produce a **custom index page** that lists every test grouped by spec file, with
+  status, duration, an inline thumbnail, an inline playable video, and access to the trace.
+- Keep the output **self-contained and portable** (a folder that can be zipped and handed off).
+- Local only — runs from `bun run`/`npm run`; no CI wiring, no hosting.
+- **Zero new dependencies**; plain Node ESM script matching the repo's `scripts/*.mjs` convention.
+
+## Non-Goals
+
+- No CI workflow changes, no GitHub Pages / S3 publishing, no remote hosting.
+- No new runtime libraries, no template engine, no thumbnail-from-video (ffmpeg) step.
+- No changes to test specs, page objects, app code, or backend.
+
+## Chosen Approach
+
+**JSON reporter + standalone generator script** (decoupled, two-step).
+
+Playwright emits a JSON results file each run; a separate Node script reads it and
+builds the index. Decoupling means the index can be regenerated from the last run
+**without re-running the 182-test suite**, and the index page can be iterated cheaply.
+
+Rejected alternatives:
+- *Custom Playwright reporter class* — single-step but cannot regenerate without re-running
+  tests, and is coupled to the reporter API.
+- *Capture config only (built-in HTML report)* — under-delivers; the custom index was an
+  explicit requirement.
+
+## Design
+
+### 1. Capture config — `playwright.config.ts`
+
+Under `use`, change capture from failure-only to always-on; add a JSON reporter as the
+generator's data source.
+
+```ts
+reporter: [
+  ['list'],
+  ['html', { open: 'never' }],
+  ['json', { outputFile: 'e2e-results/results.json' }],   // NEW
+],
+use: {
+  // ...existing...
+  screenshot: 'on',   // was 'only-on-failure'  → end-of-test still used as thumbnail
+  video: 'on',        // was 'retain-on-failure' → .webm per test
+  trace: 'on',        // was 'on-first-retry'    → trace.zip per test
+},
+```
+
+- Local `retries: 0` is unchanged, so exactly one result/artifact set per test.
+- CI behavior (`retries: 2`, `workers: 1`) is unchanged; the generator picks the **last**
+  result per test, so retries degrade gracefully if ever run in CI.
+- **Cost (accepted):** video + trace across 182 tests materially increases runtime and disk
+  (expect hundreds of MB to ~1–2 GB per run).
+
+### 2. Generator script — `scripts/generate-e2e-index.mjs`
+
+Plain Node ESM, no dependencies. Behavior:
+
+1. Read `e2e-results/results.json`. If missing, exit with a clear message ("run the suite first").
+2. Walk `suites → (nested suites) → specs → tests → results → attachments`.
+   Take the **last** result per test.
+3. For each test, collect: spec file path, full test title, status
+   (`passed | failed | skipped | timedOut | interrupted`), duration, and attachment paths
+   by `name` (`video`, `screenshot`, `trace`).
+4. Copy attachments into `e2e-results/assets/<safe-test-id>/` with friendly names
+   (`thumb.png`, `video.webm`, `trace.zip`). `<safe-test-id>` is the Playwright test `id`
+   (or a slugified title fallback) sanitized to a filesystem-safe string. Copying makes
+   `e2e-results/` self-contained and zippable.
+5. Emit `e2e-results/index.html`.
+
+Edge cases:
+- **Skipped tests:** no media; render the row with a "skipped" badge and dashes for media.
+- **Missing attachment** (guard even though `'on'` should always produce them): omit that
+  cell gracefully; never crash.
+- **Title escaping:** HTML-escape all test titles / file paths.
+- **Stale assets:** clear/overwrite `e2e-results/assets/` and `index.html` at the start of
+  generation so the folder reflects only the latest run.
+
+### 3. Index page — `e2e-results/index.html`
+
+Single self-contained HTML file with inline CSS (no framework, no external assets beyond
+its own `assets/`). Lightly echoes the app's clean blue look; not theme-coupled.
+
+- **Header summary:** run timestamp, totals (total / passed / failed / skipped), total duration.
+- **Grouping:** sections per spec file (e.g. `cluster/ClusterManagement.spec.ts`), mirroring
+  the `e2e/tests/` layout.
+- **Per-test row:**
+  - status badge (pass = green/success, fail = destructive, skip = muted),
+  - full test title,
+  - duration,
+  - **thumbnail** — inline `<img src="assets/<id>/thumb.png">`,
+  - **video** — inline `<video controls src="assets/<id>/video.webm">`,
+  - **trace** access (see below).
+- **Trace handling** (`trace.zip` is not browser-viewable):
+  - (a) a copy-able command: `npx playwright show-trace e2e-results/assets/<id>/trace.zip`, and
+  - (b) a deep-link into the existing Playwright HTML report:
+    `../playwright-report/index.html#?testId=<id>` (its built-in trace viewer also shows
+    video + screenshots). If the report's testId differs, the link degrades to the report root.
+  - This keeps inline video/thumbnail shareable while preserving full step-level debugging.
+
+### 4. Wiring — `package.json` + `.gitignore`
+
+`package.json` scripts:
+```jsonc
+"test:e2e:index": "node scripts/generate-e2e-index.mjs",
+"test:e2e:full":  "npx playwright test; node scripts/generate-e2e-index.mjs"
+```
+- `test:e2e:full` uses `;` (not `&&`) so the index is generated **even when tests fail**.
+- `test:e2e:index` regenerates the page from the last run with no re-test.
+
+`.gitignore`: add `e2e-results/` (alongside existing `test-results/` and `playwright-report/`).
+
+## Verification
+
+1. Run the new config against **one small spec** (not the full suite).
+2. Run `node scripts/generate-e2e-index.mjs`.
+3. Open `e2e-results/index.html` and confirm:
+   - summary counts are correct,
+   - thumbnail renders,
+   - video plays inline,
+   - the trace command and report deep-link resolve,
+   - a skipped/failed row (if present) renders correctly.
+4. Then run the full suite once to confirm scale (grouping, disk, runtime) is acceptable.
+
+## Files Touched
+
+| File | Change |
+|------|--------|
+| `playwright.config.ts` | capture settings → `on`; add `json` reporter |
+| `scripts/generate-e2e-index.mjs` | **new** — generator |
+| `package.json` | add `test:e2e:index`, `test:e2e:full` scripts |
+| `.gitignore` | add `e2e-results/` |
+
+## Open Questions / Future (out of scope)
+
+- Optional CI publishing (GitHub Pages / S3) — deferred; design is local-only by choice.
+- Optional flag to reference `test-results/` in place instead of copying, to halve disk — not
+  built now; copying chosen for portability.

package.json

@@ -51,7 +51,9 @@
     "test:e2e:ui": "npx playwright test --ui",
     "test:e2e:headed": "npx playwright test --headed",
     "test:e2e:debug": "npx playwright test --debug",
-    "test:e2e:report": "npx playwright show-report"
+    "test:e2e:report": "npx playwright show-report",
+    "test:e2e:index": "node scripts/generate-e2e-index.mjs",
+    "test:e2e:full": "npx playwright test; node scripts/generate-e2e-index.mjs"
   },
   "eslintConfig": {
     "parser": "@typescript-eslint/parser",

playwright.config.ts

@@ -12,14 +12,15 @@ export default defineConfig({
   reporter: [
     ['list'],
     ['html', { open: 'never' }],
+    ['json', { outputFile: 'e2e-results/results.json' }],
   ],
   use: {
     baseURL: process.env.BASE_URL || 'http://localhost:3100',
     headless: !!process.env.CI,
     ignoreHTTPSErrors: true,
-    screenshot: 'only-on-failure',
-    trace: 'on-first-retry',
-    video: 'retain-on-failure',
+    screenshot: 'on',
+    trace: 'on',
+    video: 'on',
     storageState: 'e2e/.auth/user.json',
   },
   projects: [

scripts/generate-e2e-index.mjs

@@ -0,0 +1,68 @@
+import {
+  readFileSync,
+  writeFileSync,
+  mkdirSync,
+  rmSync,
+  copyFileSync,
+  existsSync,
+} from 'node:fs';
+import { fileURLToPath } from 'node:url';
+import { dirname, join, extname } from 'node:path';
+import {
+  parseResults,
+  summarize,
+  renderIndexHtml,
+  safeId,
+} from './lib/e2e-index-format.mjs';
+
+const root = join(dirname(fileURLToPath(import.meta.url)), '..');
+const outDir = join(root, 'e2e-results');
+const resultsPath = join(outDir, 'results.json');
+const assetsDir = join(outDir, 'assets');
+
+if (!existsSync(resultsPath)) {
+  console.error(
+    `No results at ${resultsPath}. Run the e2e suite first (e.g. bun run test:e2e:full).`
+  );
+  process.exit(1);
+}
+
+const report = JSON.parse(readFileSync(resultsPath, 'utf8'));
+const tests = parseResults(report);
+
+// Rebuild assets from scratch so the folder reflects only the latest run.
+rmSync(assetsDir, { recursive: true, force: true });
+mkdirSync(assetsDir, { recursive: true });
+
+// Playwright attachment name -> friendly basename in the index folder.
+const NAMES = { screenshot: 'thumb', video: 'video', trace: 'trace' };
+
+for (const test of tests) {
+  const id = safeId(test.id);
+  const dir = join(assetsDir, id);
+  test.assets = {};
+  let made = false;
+  for (const [attName, friendly] of Object.entries(NAMES)) {
+    const src = test.attachments[attName];
+    if (!src || !existsSync(src)) continue;
+    if (!made) {
+      mkdirSync(dir, { recursive: true });
+      made = true;
+    }
+    const ext = extname(src);
+    copyFileSync(src, join(dir, friendly + ext));
+    test.assets[friendly] = `assets/${id}/${friendly}${ext}`;
+  }
+}
+
+const summary = summarize(tests);
+const generatedAt = new Date().toISOString().replace('T', ' ').slice(0, 19);
+writeFileSync(
+  join(outDir, 'index.html'),
+  renderIndexHtml({ tests, summary, generatedAt })
+);
+
+console.log(
+  `Generated e2e-results/index.html — ${summary.total} test(s): ` +
+    `${summary.passed} passed, ${summary.failed} failed, ${summary.skipped} skipped.`
+);