feat(generator): @humanjs/generator v0.1 — visual recorder + editor#59
Merged
Conversation
…aywright Milestone 1 of the visual recorder v0.1: package shell (mirrors @humanjs/mcp bin-package template, version 0.0.0) with the CLI entry, README, and ROADMAP. Also surfaces generateHumanJS / generatePlaywrightTest from the @humanjs/playwright root so the generator can turn a curated Timeline into code without reimplementing codegen.
Milestone 2: `npx @humanjs/generator <url>` now starts a loopback-only dashboard server (HTTP serves the built SPA when present, else an embedded placeholder; WebSocket carries the live channel), launches a real headed Chromium at the target URL, opens the dashboard in the default browser, and ties browser-close / SIGINT to a clean shutdown. URL normalization defaults bare hosts to https (loopback to http). Transport covered by integration tests (placeholder serving, 404, WS broadcast).
Milestone 3: an injected IIFE recorder (bundled from src/injected via a second tsup target) captures click / rightClick / type / check / uncheck / selectOption / press / scroll in the page and posts each through the __humanjsEmit binding; the Node side adds main-frame navigations as goto steps, stamps tMs, and streams the live timeline to the dashboard over WS. Selector inference ranks ARIA role+name, aria-label, visible text, test id, unique #id, CSS path, then XPath — pure and unit-tested via happy-dom. The placeholder dashboard now renders the live steps.
…esses Field feedback: typing wasn't recorded (keyed off the unreliable change event) and Backspace/Delete inside a field spammed press steps. Text capture now keys off input (debounced, flushed on blur and before any following action so type-then-click order holds); password values stay masked. Editing keys (Backspace/Delete/arrows) no longer emit press inside fields — only Enter/Tab/Escape do. Adds drag capture: native HTML5 DnD plus pointer drags (movement threshold + text-selection guard).
generatePlaywrightTest now maps assert timeline events to @playwright/test assertions — visible -> toBeVisible, text -> toHaveText, url -> toHaveURL — interleaved in recorded order, pulling page + expect into the test. The standalone generateHumanJS script export filters them out. Lets timeline- producing tooling (the generator) emit intentional assertions.
Milestone 4: generate a .spec.ts / .ts from the captured timeline via the shared codegen. New export.ts wraps the timeline with the chosen personality/seed/speed and substitutes process.env.* for secret-flagged type/paste steps (sentinel round-trip, works even when the value was masked); formatFromFilename maps .spec/.test -> spec. The CLI streams a live code preview over WS and writes humanjs-recording.spec.ts / .ts on an export command; the placeholder dashboard gained a preview pane and Export buttons. Covered by unit tests (format, personality, secrets, assertions) and verified end-to-end against a real captured flow.
Milestone 5a: the CLI now owns a canonical, id-stamped TimelineStore (append / delete / move / update / addAssert), and the WS protocol carries editor commands (delete, move, update, addAssert, setPersonality, export) with the CLI broadcasting a full state snapshot (steps + personality + live code) after every capture or edit. Store logic is unit-tested. The placeholder dashboard now renders the state snapshot read-only; the React editor UI replaces it next.
…crets Milestone 5b: a Vite + React dashboard built into dist/dashboard and served by the CLI (placeholder remains the fallback). Renders the live timeline with delete, drag-reorder, relabel, captured-value editing, a per-step selector picker over the ranked candidates, point-and-add assertions, a secret toggle (emit process.env.*), a personality switcher, and a live code preview that refreshes on every edit. Dedicated vitest.config keeps tests at the package root (vite.config sets the dashboard root); the server gains a testable dashboardDir, with tests for real asset serving and traversal rejection.
Editor polish: replace native <select> with a themed, animated, keyboard- accessible Select (button-based options); the live preview is now syntax-highlighted via a small dependency-free TS tokenizer (HTML-escaped, safe for captured values); and reordering uses framer-motion's Reorder so steps slide out of the way live as you drag a handle, with the CLI applying a new full-order 'reorder' command. Aligns the dashboard to React 19 + framer-motion to match the landing.
Editor refinements: the custom select panel now sizes to its content (short options like 'visible' no longer wrap) with an aligned check column, and opens upward when there isn't room below. The live preview renders line-by-line with framer-motion layout animation, so lines slide to their new positions when the timeline is reordered (new lines fade in, removed fade out).
…ollbars Three editor fixes from testing: (1) a pointer-drag on a carousel/slider that fires a trailing click on release no longer records a stray click step — the recorder swallows the click that immediately follows a captured drag; (2) the step list smooth-scrolls to the bottom as new steps stream in, but only when the user is already at the bottom (scrolled-up inspection is left alone); and (3) themed thin scrollbars across the editor, preview, and dropdowns.
Long role=...[name="…"] selectors gave the preview's no-wrap code lines a huge min-content width; CSS grid's default min-width:auto then let the preview track grow and squeeze the editor. Set min-width:0 on both columns so they hold their 1fr split and long lines scroll horizontally inside the preview.
Autoscroll now defers the scroll to the next animation frame so it measures scrollHeight after the new row lays out (framer-motion was mid-animation), so it reaches the actual bottom instead of stopping short. The live preview soft-wraps long lines (with a hanging indent for continuations) instead of scrolling horizontally — a read-only preview reads better top-to-bottom, especially with the very long selector names some pages produce.
…eset Milestone 6: rewrite the README for the v0.1 launch (how-it-works, editor capabilities, sample output, requirements, screenshot slot), update the docs/DESIGN generator section to reflect it's shipping with a cross-link, and add the minor changeset that publishes @humanjs/generator@0.1.0.
Re-theme the dashboard from the placeholder emerald to the landing's actual palette (apps/web/app/globals.css): warm dark surfaces (#060604 / #0c0b0a / #110f0d), warm text (#f0ece5 / #8a857c), and the orange accent (#f5a55c, bright #ffb87a). Warmed the field/dropdown/scrollbar neutrals to suit. Syntax-highlight token colors are intentionally left as their own balanced code palette (an all-orange code pane would be unreadable). Fallback placeholder updated too.
createHuman now calls installMouseHelper on its page by default, so humanized motion is visible in headed runs and recordings without a manual call — the generator's / Recording.toHumanJS() exported scripts now show the cursor when run. Opt out with cursor: false (for speed: 'instant' / CI, where there's no motion and the injected cursor would pollute test DOM/screenshots); pass an options object to style it. The test fixture opts out automatically in CI. The install is page-scoped, idempotent, and skipped on page objects lacking addInitScript (unit-test mocks).
…ded gesture Clicking a link (or submitting a form, or search-as-you-type) was captured as both the action AND a goto for the resulting navigation, so replays navigated twice. The recorder now pings a throttled gesture sentinel on pointerdown / keydown, and the CLI skips a main-frame goto that lands within 2.5s of the last gesture — that navigation is reproduced by replaying the gesture. Navigations with no recent gesture (initial load, address-bar nav) are still captured. Verified end-to-end: a link click records the click only, no trailing goto.
SPA routers reset scroll to (0,0) on client-side navigation, firing a scroll
event the recorder captured as a no-op scroll({to:0}) step. The CLI now tracks
the last main-frame navigation and drops a scroll-to-top (offset <= 3) that
lands within 1.5s of one — it's scroll-restoration, not a user scroll. Gated on
navigation timing (not gesture) so genuine keyboard scrolls (PageUp/Home) are
kept. Verified end-to-end: a real scroll-down is recorded, the post-pushState
scroll-to-0 is dropped.
Generalizes the navigation scroll-to-top fix: the recorder now records a scroll only when it was physically driven by the user (a wheel / trackpad / touch input). Programmatic scrolls — SPA scroll-restoration on navigation, a gallery resetting page scroll on arrow keys, anchor jumps, scroll-to-top buttons — have no scroll input and are already reproduced by the action that caused them, so they're no longer captured as no-op scroll steps. Keyboard scrolling stays covered by the recorded key press. Replaces the narrower CLI-side nav-gated scroll-to-0 suppression. Verified: a wheel scroll is recorded; programmatic scrollTo calls are not.
…ic/keyboard scrolls Flips the scroll model so it records every real user scroll — wheel, trackpad, touch, AND dragging the scrollbar (which fires no wheel and, with overlay scrollbars, no pointer event either) — and drops only scrolls it can prove are side-effects: programmatic ones (the recorder patches window/Element scroll/scrollTo/scrollBy/scrollIntoView and the scrollTop/scrollLeft setters to timestamp themselves) or keyboard ones (already reproduced by the recorded press). Replaces the wheel-only allowlist, which missed scrollbar drags. Verified: wheel kept; scrollTo and scrollTop= suppressed; PageDown → press.
Dragging the scrollbar was caught by the pointer-drag detector and emitted a
bogus drag('xpath=//','xpath=//') (the gesture's element is the root <html>,
which has no real selector) alongside the correct scroll. The pointer-drag
detector now skips a gesture that scrolled the page during the press (scrollbar
drag / drag-to-edge autoscroll — a scroll, not a content drag) or that starts
on the root/body (no meaningful source, degenerate selector). Verified: a real
content drag still records; a scrollbar-style gesture records only the scroll.
|
The latest updates on your projects. Learn more about Vercel for GitHub.
|
# Conflicts: # pnpm-lock.yaml
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Ships
@humanjs/generator—npx @humanjs/generator <url>launches a real Chromium window and a local (loopback-only) dashboard, captures your clicks/typing/scrolls/navigation with role-first selectors, and exports a clean, humanized Playwright test from a full in-browser editor.What's in it
@humanjs/generator(new package, →0.1.0)addInitScript) with ranked selector inference: ARIA role + accessible name → label → text → test id →#id→ CSS → XPath.toBeVisible/toHaveText/toHaveURL), a secret toggle (→process.env.X), a personality switcher, and a live, syntax-highlighted code preview. Themed to matchhumanjs.dev(warm/orange).@humanjs/playwright/testspec (.spec.ts) or a standalone HumanJS script (.ts); the curated timeline runs through@humanjs/playwright's codegen, so generated specs stay in lockstep with the library.@humanjs/playwright(→ minor)generateHumanJS/generatePlaywrightTestfrom the package root (the codegen the generator builds on).generatePlaywrightTestrenders explicitasserttimeline events.createHumaninstalls the visual cursor overlay by default (opt out withcursor: false; the test fixture opts out in CI).Capture-quality fixes (from real-world testing)
goto.drag(xpath=//).Tests
@humanjs/generator: 40 tests (timeline store edit ops, export/codegen integration, dashboard server incl. asset serving + path-traversal rejection, URL normalization). Capture pipeline verified end-to-end against local fixtures in a real browser.@humanjs/playwright: full suite green (incl. the new assert codegen + cursor default).Release
Changesets queue:
@humanjs/playwrightminor (codegen exports, assert events, cursor default),@humanjs/generator0.1.0initial, plus dependency-bump patches for mcp/recorder.Note
The README editor screenshot/GIF is intentionally omitted for now — to be added later.