Skip to content

Automatic screenshot test suite #53

@gwern

Description

@gwern

Inspired by https://interblah.net/self-updating-screenshots https://news.ycombinator.com/item?id=47908051 : it would be nice to have some sort of automatic screenshotting service or tool which would record a screenshot of specified pages (eg. /lorem page stress tests) and commit the files, so we had a history of Gwern.net appearance over time and can check for visual regressions or shift. Right now, we don't know how Gwernnet looked at any given point; we have just a handful of scattered screenshots I happened to manually take and save as permanent files rather than /doc/www/misc/ working files.

Chrome, for example, has a built-in PDF/PNG screenshot utility, which we used a long time ago to generate crude link previews: https://gwern.net/design-graveyard#link-screenshot-previews / linkScreenshot.sh (added e67a9bb , removed 4189ec7 ). Old reference invocation:

# Remote HTML, which is actually HTML:
timeout chromium-browser --disable-background-networking --disable-background-timer-throttling --disable-breakpad --disable-client-side-phishing-detection --disable-default-apps --disable-dev-shm-usage --disable-extensions --disable-features=site-per-process --disable-hang-monitor --disable-popup-blocking --disable-prompt-on-repost --disable-sync --disable-translate --metrics-recording-only --no-first-run --safebrowsing-disable-auto-update --enable-automation --password-store=basic --use-mock-keychain --hide-scrollbars --mute-audio --headless --disable-gpu --hide-scrollbars \
  --screenshot=/tmp/"$HASH" \
  --window-size=768,768 "$@"

We probably want to upgrade to something more intended for this purpose, like chrome-headless-shell / Playwright. (Something like npx playwright screenshot --full-page --wait-for-load-state=networkidle <URL> out.png? We want to capture light vs dark-mode and desktop vs mobile-mode as well, so ideally we need 2x2 = 4 screenshots per target.)

So perhaps inside sync.sh, something like everyNDays 7 && linkScreenshot [...] which saves the targets to /doc/meta/screenshot/$YYYY/MM-DD-screenshot-$URL-$DEVICE-$THEME.png and compressPNG foo.png && git add && git commit? This directory structure means I can easily view a URL as simply as feh ~/wiki/doc/meta/screenshot/*/*$URL-slug*

Since we'd expect most screenshots to be redundant (as Gwern.net doesn't have much dynamic content, and rare instances like holiday theme modes are desirable to screenshot when they fire), and the size of PNGs could add up quickly without deduplication, we could add a pixel distance step using ImageMagick: keep the new one only if it is X pixels different somehow from the previous one of the same URL target? ImageMagick's -define compare:similarity-threshold might work here.

And we can provide a canonical 'latest' screenshot URL for each target by just having a /doc/metadata/screenshots/latest/$URL.png symlink, which could be version-controlled, although I'm not sure there's any point to that (since the 'latest' will always be well-defined and recoverable from the regular version-controlled PNGs).

Metadata

Metadata

Assignees

Labels

AIIssue which can probably be resolved by agentic LLM AI coding, rather than scarce humans.FrontendApplies to in-browser experience; primarily CSS/JS, but not the raw HTML, nginx, or backend/content.documentationImprovements or additions to documentationenhancementNew feature or request

Projects

No projects

Milestone

No milestone

Relationships

None yet

Development

No branches or pull requests

Issue actions