Repository navigation
Full-sensor stills: photo mode runs the photo format, and the mode toggle pre-warms it - #248
Merged
Merged
Conversation
…ggle pre-warms it Stills were frames lifted from the 16:9 video pipeline: the session never left the video format, so a 12MP sensor delivered ~10MP upscaled, digitally zoomed frames with no Deep Fusion (quality prioritization was tied to the HDR toggle). The capture engine now follows the rig's currentCameraMode: photo mode runs the .photo preset (full sensor, preview-sized buffers to the streamer), video mode runs the selected video format, and every shot asks for maxPhotoDimensions at .quality. Aspect crops go through ImageIO, so HEIC stays HEIC and EXIF (the rig sync stamp included) survives; any non-native ratio is now a crop of the full capture. On Catalyst the .photo preset throws (measured on hardware) and buys nothing, so Mac photo mode keeps the video format and gets its ceiling via maxPhotoDimensions. New wire action SetCameraMode = 35: the director's PHOTO/VIDEO toggle pre-warms every capable camera, paying the ~0.5s sensor reconfigure at the toggle instead of at the shutter. It is a setting push on the standard control-reply contract (refused while recording), gated on the camera reporting camera_mode in its state (presence is the capability), and reconciled on join like aspect. Capture commands still set the mode at arrival — ScheduledCapture arms photo configuration during the countdown — so a missed push can never desync a shot. Verified: full suite green under the simulator (serialization round-trips, coordinator apply/refuse matrix, director gating + join reconcile, ImageIO crop codec/metadata); Catalyst hardware suite green with stills measured at exactly the active format's photo ceiling and crops matching the cropRect oracle. iPhone hardware run pending (device offline). Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
…only on real change Inline narration moved into doc comments on the declarations it described (cloneCameraSettings, applyPhotoFormatLocked, setVideoQualityLocked, the rig's currentCameraMode) or deleted. cropPhotoData's dictionary surgery is now a pure destinationProperties helper built from let + merging. The platform .photo-preset constant is static. applyCaptureModeLocked announces the preview-encoder scene cut only when the session actually reconfigured, so the Catalyst no-op path no longer forces a gratuitous keyframe. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Contributor
📊 Test Coverage Report✅ 755/774 tests passed · ⏭️ 19 skipped
🎥 Capture & session core
📁 Full report — 101 files
Generated from |
A fresh session now starts on HEIC when the photo output offers HEVC — Apple's own default: half the bytes of JPEG at equal quality, which matters now that full-sensor stills cross the P2P link — and JPEG otherwise. The policy is one pure function (defaultPhotoFormat), the director's SetPhotoQuality still overrides it, and the crop path already preserves the container. Hardware run measured the fallback live: this MacBook's camera offers no HEVC and correctly starts on JPEG. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Member
Author
|
Added in this PR per review: stills default to HEIC where the hardware encodes HEVC (pure |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
What you get
Photos from the remote stop looking like screenshots of the preview. The camera's session was permanently configured for video, so every still was a 16:9 video frame — upscaled past the sensor (the 14 Oct dinner shot: 4224×2376 from a 4032-wide sensor, digital zoom 1.04×, no Deep Fusion). Now:
.qualitymulti-frame processing on every shot — no longer tied to the HDR toggle.SetCameraMode = 35). The ~0.5s sensor reconfigure happens when you flip the toggle, while you're composing — not at the shutter. First shot after a flip is instant, and the preview shows the real 4:3 photo frame while you frame it.How it behaves
currentCameraMode— no new seam; capture commands still set the mode at arrival (a synced rig shot arms photo configuration during the countdown), so a missed pre-warm can never desync a shot.SetCameraModerides the standard control contract: one full-state reply, refused while recording, sent only to cameras that reportcamera_modein their state (presence is the capability), reconciled when a camera joins mid-session — same shape as aspect ratio..photopreset throws on Mac cameras (measured on hardware) and buys nothing there — a Mac camera's photo ceiling is its video format — so the Mac keeps the video format and gets its full ceiling viamaxPhotoDimensions.What was checked
cropRectoracle, frames flowing across photo↔video flips (423ms / 53ms).🤖 Generated with Claude Code