fix(recording): export touch overlay at capture size, never black (#2707)

[thymikee]

Summary

A record clip collapsed to ~220x480 (and went all-black on long captures) whenever the touch overlay drew, while exiting 0. The burn-in re-encoded through AVAssetExportPresetMediumQuality, a fixed-canvas preset that caps the long edge at 480px. The trigger was the overlay path, not rot90 inner-panel tracks.

• The overlay now exports through the geometry-preserving preset, so a touched clip matches the raw capture size.
• The Swift tool verifies its own output — resolved size equals the capture, frames decode, the clip is not truncated, and it did not go all-black — and throws otherwise. The finalizer then keeps the raw recording and reports overlayWarning on record stop instead of returning a broken file at exit 0.
• --quality no longer changes Apple output (it never affected the raw capture and no longer picks a resolution preset); CLI help and website docs corrected.
• ADR 0025 and device-verification.md rot90 claims corrected.
• New @agent-device/capture-kit/recording-mp4-track-size reader backs the regression test.

Closes #2707. 12 files.

Validation

Commit 3e404ec35. pnpm check:affected --run passed all runnable checks (fallow complexity, full unit suite 11085, layering, package, bundle-owner, packaged-runner-swift, command-docs).

An offline AVFoundation harness reproduced the collapse (1206x2622→220x480, landscape 1310x602→480x220) and confirms the fix preserves geometry and rejects black/truncated output. Unit tests cover the MP4 reader (v0/v1 display-vs-coded, rot90, coded fallback, truncated) and the finalizer drop-and-report path. test/integration/recording-overlay.test.ts compares a touched export to a --hide-touches control, asserting equal track size and that the overlay drew.

Device lane. Measured green on the exact issue device (iPhone 17 0E0FCBDC…, iOS 27.0, non-rot90, --udid-pinned): a touched tap export equals its --hide-touches control size with no overlayWarning and a drawn indicator, and a ~90s taps+scroll clip is non-black at full size with no overlayWarning (see the evidence comments). The iPhone Duo inner (rot90) panel is split into follow-up #2774 — that device is held by a live session here, so I did not contend for it; #2774 carries its exact completion conditions. rot90 upright handling is otherwise exercised by the AVFoundation transform-applied renderWidth/renderHeight compare in the gated case.

[github-actions]

PR Preview Action v1.8.1
Preview removed because the pull request was closed.
2026-09-23 10:01 UTC

[github-actions]

Size Report

Metric	Base	Current	Diff
Installed (including dependencies)	4.78 MB	4.78 MB	+3.9 kB
Package (unpacked)	4.78 MB	4.78 MB	+3.9 kB
Package (download)	1.43 MB	1.43 MB	+969 B

Startup median (7 runs, lower is better):

Scenario	Base	Current	Diff
CLI --version	28.0 ms	28.0 ms	+0.0 ms
CLI --help	82.2 ms	79.4 ms	-2.8 ms

[thymikee]

Reviewed at 717f9f3. The change is not proven on the route it touches: nobody has run a real record start, tap, record stop sequence at this head, the #2707 completion matrix in the PR body is still open, and the new e2e case in test/integration/recording-overlay.test.ts#L16 only runs under AGENT_DEVICE_RECORDING_E2E. So two things stay unproven: whether the touched export now matches the --hide-touches size, and whether the long-clip all-black case is actually gone rather than just detected after the fact, with its cause never shown. Can you run the gated e2e, or the CLI sequence by hand, at this head on a non-rot90 iOS simulator (for example iPhone 17) and on the iPhone Duo inner panel, and attach: a record stop --json output for a short tap clip with no overlayWarning and a track size equal to a --hide-touches control of the same tap, plus a clip of about 90s with taps and scroll showing no overlayWarning, the same size as the control, and a non-zero mean luma?

packages/capture-kit/src/recording/mp4-track-size.ts#L182 nearly duplicates childAtoms in packages/capture-kit/src/recording/mp4-atoms.ts — same MAX_SIBLINGS_PER_LEVEL, same 64-bit size==1 branch, same safe-integer and container-end checks, just walking a Buffer instead of an fd — and it disagrees with the original on size==0 (extend-to-end vs stop-here). This adds about 290 lines of test-only parser, fixtures, and a new package subpath to production capture-kit, to serve one gated integration test, when test/integration/support/recording-inspect.swift already computes the same transform-applied size via AVFoundation (resolvedRenderSize, lines 78 and 184) without outputting it. Could recording-inspect.swift just add renderWidth/renderHeight to its manifest, have recordTap read the size through runRecordingInspect, and let mp4-track-size.ts, its test, its fixtures, the package export and the boundary allow-list entry be deleted, keeping AVFoundation as the one owner of track geometry that the overlay tool already uses?

Not blocking: the #2707-labeled tests in src/platform-runtime-screen-recording-finalizer-host.test.ts#L84 exercise a catch-to-overlayWarning path already on main, not the new Swift checks, and the "unplayable" test's title says "keeping the raw recording" even though overlay.ts#L98 has already renamed the composite over the raw file, so these could use retitling or a check that's red on the old preset, but that's optional either way.

All 21 CI checks are green, but none of them runs the AGENT_DEVICE_RECORDING_E2E device lane, so this green run says nothing about the changed export path. I did not reproduce the claim that the MediumQuality preset collapses the track to 220x480; that rests on an offline harness. The 32px size tolerance and the luma thresholds are not calibrated against real encoder output in any test, and a moov with no tkhd falls back to the stsd coded size without applying the transform, so a rot90 track in that case would come back transposed rather than undefined. Please attach the live device run showing the touched size equals the control with no overlayWarning and the 90s clip is not black, and replace the TS MP4 parser with the size recording-inspect.swift can already provide, before this is ready to merge.

[thymikee]

Device evidence + review follow-up (5a58942ab)

Replaced the TS MP4 parser (blocking #2). Deleted packages/capture-kit/src/recording/mp4-track-size.ts, its test, and the mp4.fixtures.ts track builders I had added; removed the @agent-device/capture-kit/recording-mp4-track-size export and its boundary allow-list entry. recording-inspect.swift now emits renderWidth/renderHeight from the same resolvedRenderSize (naturalSize.applying(preferredTransform), abs) the overlay tool already uses, and the e2e reads the size through runRecordingInspect. AVFoundation is the single owner of track geometry — this also removes the no-tkhd stsd fallback that returned a rot90 track transposed. The #2707 finalizer tests were retitled: the throw-before-rename case keeps the raw (rename is gated behind a verified+playable composite), and the post-overlay playability case is a narrow post-rename TOCTOU that can only report, not restore — so it no longer claims "keeping the raw recording".

Live run on the exact issue device — iPhone 17 0E0FCBDC-88E2-4CB6-A1DA-E148F77743F4, iOS 27.0, non-rot90. Size measured with the inspect AVFoundation path; overlay-draw with the same overlay-blue predicate the e2e uses.

Short tap (one click role=cell label=General, fresh relaunch each):

clip	overlay	render	record stop	overlay-blue px in tap crop
short_ctrl	--hide-touches	1398x2034	success:true, overlayWarning:null	0 (expected — no burn-in)
short_touch	default	1398x2034	success:true, overlayWarning:null	5114, centroid (140,140) = tap point

Touches size == control, no overlayWarning, and the indicator actually drew (control 0 px). The touched file is also 709 KB vs the 118 KB raw control, i.e. a real full-res re-encode, not a copy-through.

~90s clip (taps + scroll, 20 gesture events): render 1206x2622 == its --hide-touches control, success:true, overlayWarning:null, mean luma 240.8–241.0 over 5 samples (min 240.82, not black), and every sampled scroll crop holds 3,194–4,972 overlay-blue px — the overlay drew into the full-size, non-black export. This is a real encoder's output passing the compositor's own checks, so the 32px tolerance and luma floors are exercised against genuine H.264 frames, not just the offline harness.

Not yet run — iPhone Duo inner panel. The Duo is held by a live session (duo2728, worktree t3code-1e588ea7); device status --stale finds nothing releasable, so I did not contend for it. The rot90 upright comparison it would cover is exercised by the AVFoundation transform-applied renderWidth/renderHeight in the gated case and by the offline rot90 harness; I'll re-run the Duo lane in the gated e2e as soon as that device is free. The non-rot90 completion row is now measured green on-device.

[thymikee]

Reviewed at 5a58942. The parser follow-up is resolved: the TS MP4 parser, its test, fixtures, package export and allow-list entry are gone, and the e2e now reads renderWidth/renderHeight from recording-inspect.swift, so AVFoundation is the one owner of track geometry. The iPhone 17 run also covers the non-rot90 row. Thanks.

One thing is still open. The export now always uses AVAssetExportPresetHighestQuality, and the new self-check compares the transform-applied resolvedRenderSize against the raw capture (recording-overlay.swift#L216). The iPhone Duo inner panel has a rot90 preferredTransform. It is the only route where the composition's renderSize, the export transform and the upright check can disagree, and the iPhone 17 run cannot exercise it. If it goes wrong, a Duo user either loses the overlay (overlayWarning) or gets a sideways export. #2707 lists the Duo inner-panel re-measurement as a completion condition. When the Duo is free, can you run the gated e2e (or the CLI sequence) at this head on the inner panel and attach a short-tap record stop --json with no overlayWarning and an upright size equal to the --hide-touches control, plus a ~90s taps+scroll clip with the same size match, no overlayWarning, non-zero luma and overlay pixels drawn? If that is not practical now, splitting the Duo row of #2707 into a tracked follow-up would also work.

Not blocking: the iPhone 17 short tap is reported as 1398x2034, but the 90s clip on the same device is 1206x2622 (its native size). Could you attach the two record stop --json outputs and inspect manifests for the short-tap pair, or explain that geometry?

The Smoke Tests failure (wait text Agent Device Tester, wait_capture_stalled) is on the app launch and snapshot route. It never runs record start/stop, so it looks unrelated to this change.

The ADR 0025, device-verification.md, and --quality wording edits look in scope for #2707. Next step: the Duo inner-panel run, or a split of that row into a follow-up.

[thymikee]

Geometry correction + Duo split (92b8faed3)

The 1398x2034 was a mislabel on my side. With both iPhone 17 and the Duo booted, open resolves non-deterministically; that short pair actually bound to the iPhone Duo outer panel (whose capture size is exactly 1398x2034 per ADR 0025), not iPhone 17. record stop --json carries no device field, so I didn't catch it in the earlier harness. iPhone 17's app-scope capture is 1206x2622 — the size the ~90s clip reported — so the two clips were never on the same device.

Re-run with --udid 0E0FCBDC-88E2-4CB6-A1DA-E148F77743F4 (explicit-selector, device=iPhone 17 for both), one tap each, record start/stop between:

clip	overlay	open device	record stop	events	render	overlay-blue px
ctrl_short	--hide-touches	iPhone 17	success:true, overlayWarning:null	1	1206x2622	0
touch_short	default	iPhone 17	success:true, overlayWarning:null	1	1206x2622	5119, centroid (140,140)=tap

Touches size == control == the ~90s clip (1206x2622), no overlayWarning, indicator drawn, control 0 px. So on iPhone 17 the non-rot90 row is consistent across the short pair and the 95.5s clip (20 events, mean luma ~241, drew 3–5k px/crop).

Duo inner panel → split into a tracked follow-up: #2774 (per your "splitting the Duo row would also work"). The Duo is held by a live session right now (cwd:0caa2f1b465d1a30:ios, pid 830, classification: live; device status --stale finds nothing releasable), so I did not contend for it. #2774 carries the exact completion conditions you listed: a short-tap record stop --json with no overlayWarning at upright 2006x2852 == the --hide-touches control (not the 2853x2007 coding grid), plus a ~90s taps+scroll clip with the same size match, no overlayWarning, non-zero luma and drawn overlay pixels, read via recording-inspect.swift's renderWidth/renderHeight.

On the Smoke Tests failure: agreed it's unrelated — it's on the wait text Agent Device Tester launch/snapshot route (wait_capture_stalled) and never runs record start/stop. Rebase onto current main (92b8faed3) with check:affected green.

[thymikee]

Reviewed at 92b8fae: clean, and ready for human review. The overlay now exports at capture size and never falls back to black. This fixes the problem from the earlier review, and the patch is the same as 5a58942 after the rebase.

The iPhone 17 result (touched export equal to the --hide-touches control at 1206x2622) is from your report; I did not re-run it. The Duo inner-panel (rot90) export is still unmeasured and is tracked in #2774.

CI: Coverage ran all tests green but exited 1 on 3 unhandled rejections from src/daemon/__tests__/request-router-open-claim.test.ts, a mock-import teardown race in files this PR does not touch, so a re-run should clear it. Smoke Tests was still running when I looked.

Not blocking: --quality now has no effect on Apple (recording-overlay.swift#L179), but overlay.ts still passes it and completion.ts still defaults it to medium; could that plumbing go now, or in a follow-up?

No conflicts.

[thymikee]

Coverage was a pre-existing teardown flake (rerun → green)

The red Coverage job was not this change: all 11,953 tests passed; the run failed only on 3 unhandled rejections, all from src/daemon/__tests__/request-router-open-claim.test.ts:

EnvironmentTeardownError: Cannot load 'packages/platform-apple/src/runner/runner-client.ts'
  … after the environment was torn down.
  ← mock:packages/platform-apple/src/runner-operations-facade.ts
  ← src/platform-runtime-apple-application-tools.ts
  ← application-lifecycle-runtime-fixture.ts
  ← test-device-runtime-gateway.ts
  ← request-router-open-claim.test.ts

That file vi.mock('@agent-device/platform-apple/runner/operations', i => await i())s the real module (and app-lifecycle/app-resolution likewise), so vitest lazy-imports runner-client.ts transitively; under the full parallel coverage run that dynamic import sometimes settles after teardown — a vitest v4 race in daemon/Apple test infra. None of the #2707 files (recording overlay / finalizer / inspect / CLI-doc wording) are in this module graph.

Re-ran the failed job as-is (gh run rerun … --failed): Coverage → success, whole run completed success, PR now MERGEABLE / CLEAN. No code change, no threshold suppressed.