Skip to content

About

Automated health checks for externally-hosted Rize course pages. Opens every page in a real browser weekly and before each term.

Resources

Stars

0 stars

Watchers

0 watching

Forks

Repository files navigation

Interactive health

A robot opens every externally-hosted course page and looks at it, on a schedule. If anything is broken, it says so before the semester starts.

This is the answer to Greg's condition (2026-09-18) on allowing externally-hosted Resource pages: "yes but only if we have an early automated way of checking if they're broken ahead of upcoming semesters" and "would rather not over-engineer a bespoke solution... find an elegant way of replicating without building something that we have to maintain."

There is no server, no database, no dashboard app, and nothing for the product team to own. GitHub runs it for free, in one repo, on a schedule.

Why a link checker is not enough

Every failure below returns HTTP 200. A link checker calls all of them healthy:

  • the graphing library moved, so the plot area is blank
  • an API key expired, so the widget never mounts
  • a JavaScript error on load silently drops every interactive on the page
  • an answer key was edited into the wrong shape, so every answer grades wrong
  • the page is fine, but the third-party PDF it sends students to is gone

A student sees text with a hole in it. Nobody files a ticket, because nothing looks broken. It looks sparse. So the check opens the page in a real browser instead.

What it checks

Check Severity
The page loads (not a 404, not a timeout) FAIL
Its scripts and stylesheets load FAIL
It rendered a real amount of visible text, not an empty shell FAIL
No uncaught JavaScript error on load FAIL
The page's own window.__selfCheck() passes FAIL
Images, fonts, and other subresources load WARN
Outbound links to third-party material still resolve WARN
No dependency on a floating CDN version (mathjax@3, @latest) WARN

WARNs never fail the run. A check that cries wolf gets ignored, and then it is worth nothing.

The interesting half is __selfCheck(): each page declares what "working" means for itself, in about ten lines, when it is built. The checker never learns anything about calculus or about any particular widget, so it never has to change. That is what keeps this from becoming a thing somebody maintains. See SELFCHECK.md.

What people see

A single status board, regenerated by every run, listing every page green / amber / red with the date it was last checked. Link it from the instructor briefing folder with one line: if anything here is red, or a page looks wrong while you are teaching, file a helpdesk ticket. Instructors are the backstop for what the robot cannot see, not the primary detector.

Failures also fail the scheduled GitHub run, which emails the repo owner, and post to Slack if a SLACK_WEBHOOK secret is set.

Status

Live since 2026-09-19. The workflow is active at .github/workflows/check.yml, Pages serves the status board from /docs, and Actions has write permission so each run commits the board back. Failure notification confirmed end to end with a deliberate 404 canary.

Setting this up fresh elsewhere: moving a file into .github/workflows/ needs a token with workflow scope, which the default gh token does not carry. One command, opens a browser once: gh auth refresh -h github.com -s workflow.

When it runs

  • on every push to the pages repo
  • every Monday, year round
  • daily through December, January, July and August, so a break surfaces weeks before students arrive rather than during week one
  • on demand, from the Actions tab

When a run fails

Read TRIAGE.md. Every run writes a summary naming the failure class: a content failure (a page students use is broken, fix the page) or an infrastructure failure (the pages are fine, the run itself broke, usually just re-run). They need opposite responses, so the summary says which before you start investigating.

Layout

TRIAGE.md               what to do when a run fails; read this first
manifest.json           what gets checked, and which course each page serves
scripts/check.mjs       the checker (Playwright; ~230 lines, the whole engine)
scripts/render_status.mjs   builds the status board from results.json
scripts/sanity_rows.py      reads every Rize-hosted link from both Sanity datasets (each run)
scripts/psyque_rows.py      the same for courses that live in Psyque (needs PSYQUE_TOKEN)
scripts/build_manifest.py   merges those rows into the manifest (keeps manual entries)
scripts/sanity_urls.groq    the older MCP query, for a by-hand inventory
template/interactive-page-template.html   start every new page from this
workflow/check.yml      the GitHub Actions workflow (the entire "system")
SELFCHECK.md            the page-author contract
FOR-CLAUDES.md          runbook for another Rize Claude setting this up

Running it locally

npm install && npx playwright install chromium
node scripts/check.mjs                 # writes results.json, exits 1 if anything is broken
node scripts/render_status.mjs         # writes docs/index.html
node scripts/check.mjs --only calc      # just the calculus pages
node scripts/check.mjs --no-links       # skip the outbound-link pass (much faster)

Refreshing what gets checked

manifest.json is generated from Sanity, so the list tracks what courses actually link to rather than a list someone remembers to update. Sanity is read by a Claude through the MCP connector, never by CI, so no Sanity credentials ever live in GitHub. See FOR-CLAUDES.md.

Two things worth doing alongside this

  1. Move the pages repos into the Rize GitHub org. They currently live under a personal account. In the org, the eventual handoff to a product-team repo is a transfer rather than a rebuild, and the bus-factor objection never gets raised.
  2. Vendor every library. A page with no external runtime dependency can only break when somebody edits it, and the checker catches that on the next push.

About

Automated health checks for externally-hosted Rize course pages. Opens every page in a real browser weekly and before each term.

Resources

Stars

0 stars

Watchers

0 watching

Forks

Releases

Packages

Contributors

Languages