Repository navigation
fix(🤖): keep a canvas's frame while its view is out of its window - #565
YevheniiKotyrlo wants to merge 3 commits into
Conversation
Device evidence — before / afterUNFIXED — on this PR's base, a FIXED — with this PR it shows its last frame as it returns: the magenta one it drew while it was away.
Galaxy S23 (Android 14), the FirstFrameHole diagnostic's reattach cycle a quarter of a second after the canvas scrolls back into view: before on the left, after on the right. A debug build of the same change before the rebase onto #574; the counts below are of this head. Device measurement — before / after
The reattach cycle scrolls the canvas out of the row and back every 1.5 s; a return counts when the canvas's place shows the stage or black before a frame the canvas drew. On the phones, release builds opening the diagnostic at launch, this PR's JavaScript over the base's native code and over this PR's; on the emulator, debug builds. Before the rebase (base |
A SurfaceView canvas sits behind its window, and the window opens its hole once the SurfaceView reports its draw finished. WebGPUSurfaceView reported it as soon as surfaceChanged returned, so the hole opened on a surface with no frame and showed black until the first one landed. The view now holds the report until a frame is presented into the surface, as GLSurfaceView holds it until its next frame. attachSurface takes a callback for the surface's first present, and each surface carries a token, so a report for a replaced surface is ignored. Before Android 14 the wait is bounded at a second; from Android 14 the platform bounds it. A report that goes out without a frame, at that second or for a later surface of the view, which SurfaceView counts as reported already, no longer opens the hole: the view punches it, and adds it to the window's transparent region, only while its surface has a frame. A context configured before its surface exists, or left by a lost one, no longer presents an empty frame when the next surface attaches, and detach is one operation for a lost surface and a destroyed view. The canvas docs say when the hole opens, the example gains a SurfaceView First Frame diagnostic that can also draw its first frame late, FirstFrameDrawGate is unit tested, and CI runs the Android library's unit tests.
…eated A SurfaceView's surface goes with its buffers when the app goes to the background or the view leaves its window, and the canvas presented nothing into the next one, so a SurfaceView canvas that draws once came back black. surfaceDestroyed is the last moment that frame can be read: the view reads it back there with PixelCopy and writes it into the canvas's offscreen texture, which the next surface's attach presents. Nothing is copied while the canvas draws. The canvas docs say a canvas that stops drawing keeps its last frame.
A view that leaves its window and comes back, as a covered native-stack screen or a canvas a removeClippedSubviews list scrolls out does, lost its Android surface, so a TextureView or HardwareBufferView canvas that did not draw again came back empty, where iOS keeps the CAMetalLayer's contents and the web the drawing buffer. A TextureView now keeps its SurfaceTexture while out of the window, and the canvas suspends its surface instead of detaching it: frames render offscreen, nothing is presented, and the surface stays connected, so it keeps its frame and the canvas can present into it again. Detaching would leave the window connected to the old Vulkan surface, and the surface made for the same window on the view's return failed to connect. resume() is latched like an attach and draws the latest offscreen frame onto the surface when one was rendered meanwhile: FrameBlit draws it texel for texel, which needs only RenderAttachment, so a resumed surface keeps its configuration. A WebGPUHardwareBufferView keeps its displayed Bitmap, and re-enabling its pool presents a frame drawn while it was away. WebGPUView retires the backing view it drops, which then releases what it keeps, and after destroy() it attaches no surface again. The FirstFrameHole diagnostic gains a reattach cycle, which scrolls the canvas out of a clipping row and back, and a backing-view switch.
5ceda7a to
56a01ec
Compare
This branch has not been deployed
1 participant
Problem
On Android, a canvas whose view leaves its window and comes back — a native-stack screen covered and revealed again, a
removeClippedSubviewslist scrolling it out and back — shows what is behind it until it draws again: aTextureViewgets a newSurfaceTexture, and aHardwareBufferViewdrops its bitmap. iOS keeps theCAMetalLayer's contents and the web keeps a canvas's drawing buffer.Keeping the
SurfaceTextureis not enough on its own: the canvas detaches its surface, Dawn keeps the window connected to the old Vulkan surface, and the surface made for the same window on the view's return fails to connect (E vulkan: native_window_api_connect() failed: Invalid argument (-22)at every return), so the canvas could not present again.The FirstFrameHole diagnostic's reattach cycle scrolls a canvas out of a
removeClippedSubviewsrow and back every 1.5 s. On #544 the canvas's place shows the stage as the canvas returns: at 9 of 13 returns with aTextureViewand at all 14 with aHardwareBufferViewon a Galaxy S23 (Android 14), at 13 of 13 with aTextureViewon a Mi MIX 3 (Android 10), and at 7 of 7 with aTextureViewon an Android 13 emulator. With this change it shows the canvas's frame at every return (measurement).Solution
SurfaceInfo::suspend()/resume(): a view that keeps its surface while out of its window suspends it instead of detaching. Frames render offscreen as while detached, nothing is presented, and the surface stays connected, so it keeps its frame.resume()is latched like an attach, and draws the latest offscreen frame onto the surface first when one was rendered meanwhile.FrameBlit): a full-screen triangle that loads the offscreen frame texel for texel. A copy needsCopyDst, and configuring a surface for it replaces the swapchain, which on Android reconnects the window and drops the frame the surface keeps. The attach path draws too, so it configures the surface with the usage the canvas asked for; a canvas configured withoutRENDER_ATTACHMENTkeeps the copy.WebGPUTextureViewkeeps itsSurfaceTexture(onSurfaceTextureDestroyedreturnsfalse) and hands it back on attach;WebGPUHardwareBufferViewkeeps its displayed bitmap, and re-enabling its pool draws the latest offscreen frame into a slot.WebGPUViewretires its backing view as it drops it (FrameKeeper.retire()), which releases what the view keeps, at once or as it leaves the window if an exit transition still shows it. Afterdestroy()it attaches no surface:attachSurfaceis find-or-create, so a latesurfaceCreatedwould have recreated the registry entryonViewDestroyedremoved.SurfaceViewkeeps its frame through fix(🤖): keep the last frame of a canvas whose Android surface is recreated #544, which reads it back as the window destroys its surface.HardwareBufferViewdocs say what keeps its frame.Tests
The suspend and resume path runs only where Android owns the surface, so the evidence is the reattach recordings: the diagnostic scrolls the canvas out of a
removeClippedSubviewsrow and back every 1.5 s, and draws a frame while it is away (magenta) and another while it shows (red); every frame of the canvas's place after a return should show one of them, never the stage or black.SurfaceInfo::suspend()andresume()belong in the host C++ tier #550 adds.Verification
yarn lint·yarn tsc·yarn turbo run build --filter='./packages/*'· the Expo template's web build ·yarn test:plugin·yarn check-dawn·yarn build:docs·cpplintwith the package's flags ·clang-formaton the changed lines ·./gradlew :react-native-webgpu:testDebugUnitTest· the example's iOS build for the simulator SDK, on5ceda7a8, whose iOS-compiled files this head keeps. CI's Android e2e step (debug build, Metro withCI=true) on an Android 15 emulator: 220 of 222, as onmainc080934e; the two failures, the YUV cases ofExternalTextureColorSpaceandNativeBufferWrapper, fail onmainthere too. On a Galaxy S23 (Android 14) the whole suite passes, 222 of 222;mainc080934epasses 222 of 222. On an iPad Pro 12.9-inch (4th generation, iOS 27.0.1), built and run as CI's iOS job does on a simulator (Debug build, Metro withCI=true), it passed 221 of 221 before the rebase onto #574 (main8f664fca: 221 of 221). Not run here: the SwiftPM example's build and the semantic-release dry run, which needs the repository's token.Known limits
A canvas the view keeps holds its frame's buffers while out of the window, as iOS does; the view releases them when React drops it.
This builds on #544 and #561 and shows their commits until they merge.