fix(run-local): supervise the --watch web client bundler (#971)

[ako]

Closes #971

Reproduction (Mendix 11.13 scratch app from mxcli new --version 11.13.0, run --local --watch --db-type hsqldb)

Machine load was high (load average ~60 on 10 cores), so the timings show order of magnitude only.

Sequence on main: add page P1, change P1 and add P2, add a microflow, add an entity and page P3 (runtime restart), change the microflow and P2.

build	change	main
#2	add page	applied via reload, 16.3s (bundler restarted for the new page)
#3	change + add page	8.4s
#4	microflow	3.3s
#5	entity + page (restart)	2m8s. /dist/index.js was not served after the restart (Gradle deletes dist), so a one-shot production bundle ran for 15.4s while the incremental watcher (still alive) was running. This is item 1.
#6	microflow + page, written during #5's apply	never built (separate bug, see below)
kill runner, then 2 page changes		web client rebuild failed: web client watcher exited twice, and neither change was applied. This is item 2.

I did not hit the 5-minute timeout (item 3). Nothing ran long enough. I made it configurable and gave it a diagnosis anyway.

Changes

Commit 1: supervise the bundler (#971)

• New bundlerSupervisor (webclient_supervisor.go) owns the incremental bundler for the whole watch loop:
  • EnsureAlive runs at the start of each change and restarts a bundler that has exited. It prints the dead bundler's log tail. After a failed start it backs off: 2s, 4s, … up to 60s.
  • AwaitRebuild waits for the incremental re-bundle. If that rebuild fails or the bundler exits during it, it retries once with a fresh bundler, whose first build is a full bundle of the current source. Measured on 11.13 after adding an entity: ENOTDIR … web/pages/<Page>.js/package.json, and the next change failed the same way. Only an error from the fresh bundler is reported as a failure.
  • Rebundle / Restart stop and reap the old bundler before starting a new one.
• ensureClientServed now takes the rebundle function. clientRebundler passes the supervisor's Rebundle under --watch, so a one-shot never runs beside the watcher. Without a bundler (11.14+, classic client, non-watch) it passes the one-shot.
• WebClientWatcher.Stop waits on a done channel instead of calling cmd.Wait a second time. I checked this: the old double Wait did block until exit on this Go version, so this is a cleanup, not the fix.
• Timeout: there is a new --web-client-timeout flag and a MXCLI_WEB_CLIENT_TIMEOUT env var, default 5m. The value is passed to every bundle build in run --local. On a timeout, BuildWebClient and the watcher's first build now say how to raise the limit and append the last 30 lines of deployment/log/web-client-build.log.
• After an apply that restarted the runtime, the output says runtime restarted for this change — browser sessions were dropped; log in again.
• Docs: run-local skill and docs-site/src/tools/run-local.md (flag and session note), CHANGELOG, and a finding.

Commit 2: an edit made while a change is being applied was lost (found while reproducing)
After each apply the loop set last = sourceMTime(...), under a comment saying it did the opposite. An edit written during the build was folded into the baseline and never built. The baseline now stays at the mtime the build settled on. During a live run, find -newer showed that the build writes nothing under the watched model or theme source, so this cannot cause a rebuild loop.

Test plan

• Unit tests in cmd/mxcli/docker/webclient_supervisor_test.go use fake bundlers that record how many run at once:
  • an exited bundler is restarted
  • backoff after a failed start (injected clock)
  • AwaitRebuild recovers from both a failed incremental build and an exited bundler; if the fresh bundler also fails, the error surfaces; a healthy rebuild does not restart anything
  • Rebundle never overlaps
  • ensureClientServed in watch mode, with a missing page: no one-shot while the bundler is live, and one replacement. Control: without a bundler, the one-shot runs.
  • Stop waits for a real process (a shell that traps SIGTERM)
  • timeout resolution (flag, then env, then default; an unparsable env value is ignored)
  • a BuildWebClient timeout with a fake node that sleeps: the error names the flag and env var and includes the log tail
  • sessionNotice
• Revert checks (each one fails with the symptom):
  • clientRebundler always one-shot → a one-shot bundle ran 1 time(s) while the incremental bundler was live
  • EnsureAlive ignores an exited bundler → EnsureAlive = (false, <nil>)
  • Restart starts before it stops → 2 bundlers ran at once
  • AwaitRebuild passes errors through → both recovery cases fail with ENOTDIR / watcher exited
  • old timeout message → missing flag, env var and log tail
  • old Stop → still passes. It is kept as a guard only, as noted above.
• Live run with the fixed binary, same 11.13 app: 7 consecutive changes, all applied.
  • After the restart, /dist/index.js recovery went through the supervisor; no one-shot ran.
  • After killing the runner: web client bundler exited unexpectedly; restarting it... / web client bundler restarted / build #7 applied. The next change was applied by the restarted bundler. dist/pages/MyFirstModule.P1.js contained the last edit.
  • The session notice was printed on both restarts.
• Lost-edit check, live: add an entity, then change a page 15s into that build. The fixed binary built it as the next build (build #10 applied). A binary with the refresh line restored (otherwise identical) built nothing within 45s.
• make build, make lint, make check-findings, make check-conformance, go test ./cmd/mxcli/, go test ./cmd/mxcli/docker/: all pass.

Not done / follow-up

• WaitForRebuild's 90s incremental limit is still hard-coded. The new timeout covers full bundle builds only.
• An incremental rebuild that fails now costs one fresh-bundler start (~5s) before the error is reported. That is deliberate, but a genuine build error is reported about 5s later than before.

🤖 Generated with Claude Code