Skip to content

Fix native window stepping blocked by render completion - #1154

Merged
yuecideng merged 11 commits into
mainfrom
codex/window-render-sync
Nov 13, 2026
Merged

yuecideng merged 11 commits into
mainfrom
codex/window-render-sync

Conversation

@acrlw

@acrlw acrlw commented Nov 11, 2026 •

Copy link
Copy Markdown
Collaborator

Description

Fix the native-window stepping regression introduced by #833. update(step=1) entered render_frame(), whose state publication called _flush_render_updates() on every physics step, including scenes without markers. On RTX 5090 D v2 with Hybrid at 1920×1080, the wait averaged 61 ms and reduced the create_scene.py loop to about 15 steps/s.

Separate state publication from native render completion. Ordinary windows still publish physics and marker transforms, while cameras, due recording/visualization captures, and explicit sync_render_state() retain completion waits. A synchronous consumer following window publication completes the same staged state once per frame. Fluid scenes retain completion waits because their bounded native staging can discard updates while both copy slots are busy. Due recording time is consumed after successful capture, so a failed completion wait remains retryable.

Marker attachment refresh and the explicit synchronization API retain their behavior. Frame exit invalidates publication and completion flags so subsequent state writes remain visible. The render fix is confined to sim_manager.py scheduling.

Measured with DexSim 0.6.0rc1, one environment, 1920×1080, a 10 ms physics timestep, 15 warmup steps and three repetitions of 50 steps:

Renderer CPU after (steps/s) CUDA after (steps/s) Full waits per 150 steps after
Hybrid 100.01 99.05 0
FastRT 100.00 100.00 0
OfflineRT 100.00 100.00 0

Hybrid before the fix measured 15.00 CPU / 14.93 CUDA steps/s, with 150 full waits. Values are median physics-loop throughput; the native window renders asynchronously. Each run advanced simulation time by 1.5 seconds over the measured 150 steps. Other GPU processes were present; the reported runs had no concurrent tests from this change. Fluid scenes retain their previous completion policy and were not benchmarked here.

Type of change

  • Bug fix (non-breaking change which fixes an issue)

Validation

Latest main integration — ccfb55bbf / main@fb640fc3f (2026-11-13): merged current main and resolved both conflicts. Asset context retains the official-source fallback and upstream task-asset catalog guidance. The SPH configuration tests now match main's current contract rejecting standalone SPH in favor of DexUni composition. Render scheduling merged without conflicts.

Local focused validation passed in the existing Python 3.12 simulation environment: 455 passed / 15 skipped across dataset fallback, simulation manager, fluid configuration/object, runtime controls, marker rendering, wheel configuration and context tests. Black 26.10.1 left all 2510 files unchanged; API docs coverage is 4887/4887; context validation and diff checks passed. Existing tests cover these integration boundaries, so no new tests were added. Context remains accurate after retaining both asset paragraphs. This run does not replace native GPU or full-container CI coverage; fresh Actions checks for this integration have not yet completed.

Previous main integration — aba50d107 / main@ac714ec86 (2026-11-12): resolved the overlapping SPH configuration test by retaining the non-default option-forwarding checks, and removed the duplicate reset-fixture assignment introduced by automatic merging. Both test files match the previously validated branch version. Nine stepping/render publication/completion/camera/recording methods are AST-identical to 205fa4990; upstream additions are retained.

Focused validation in the previously successful CI container runtime passed: 715 CPU tests; 26 native GPU tests with three SDL-dummy window skips; Black 26.10.1 over 2464 files; API coverage 4832/4832; context checks. Native coverage includes markers, mounted cameras, batch reads/depth, fluid visibility, CUDA state reset and graph-motor reset. Existing tests cover the merge interactions; no new cases or context edits were needed. The tested source tree equals the pushed commit tree.

The complete full-pr job was not rerun after this merge. The full-container evidence below remains tied to 205fa4990 before the main integration.

  • Completed one uninterrupted local full-pr test job in the SDK image declared by the workflow, on 205fa4990 including main@d95a5f161. The job exited 0: fast 13,924 passed / 56 skipped; GPU 586 passed / 11 skipped; independent sim 300 passed / 4 skipped. All 14,881 selected nodes have recorded outcomes, with zero failures, missing nodes or incomplete nodes. The distributed lane has no matching non-slow cases and follows the runner's existing empty-lane handling.
  • The test/build shell steps match the workflow verbatim. The saved plan matches Actions run 42998110633; merge commit eb9c21510 has the same tracked source as the validated head. The test used Python 3.11.16, DexSim 0.6.0rc1, PyTorch 2.7.0+cu128, eight CPU workers, default plugin loading, SDL dummy, and --no-finalize-resource-lanes --freeze-native-imports. Installation, import verification, the complete test plan and duration-history update all exited 0. The main test step ran continuously from 20:14:06 to 21:01:19 on 2026-11-11 (UTC+8).
  • Local execution used the official asset cache, host-provided Node 22, and the local GPU/driver. GitHub artifact downloads were replaced by the saved plan. A pre-existing Nsight session was stopped before the successful fresh-container run. No code, test or dependency constraints changed during this container validation.
  • The container's six native marker checks passed, covering retained materials after removal, group rendering and sensor isolation, articulation attachments, selected batch mutations, textures, opacity and material switching. Three real-window cases skip under SDL dummy; the other GPU skips are five optional roma cases and three existing CUDA skips.
  • Separate verification of the same commit in the local Python environment and an isolated X11 server passed 9 Hybrid, 4 FastRT and 4 OfflineRT checks. These create actual SDL/X11 windows and exercise window/offscreen and mono/stereo captures through production manager/camera paths. Three preceding native initialization probes passed.
  • Resolved the eight deterministic failures exposed by integration with Support custom Newton providers with a standalone tutorial #844 and Add CableObject state and lifecycle support for DexSim 0.6 #838 in two existing test files. The SPH cases now check registered-provider option forwarding; the mimic reset fixture supplies the scene's position-target capability and still checks both state and hold-target values. Unknown-solver and DexUni-only fluid checks remain covered. No new skip/xfail or production behavior change was added for these compatibility fixes. Focused configuration/runtime/backend coverage: 160 passed, 2 skipped.
  • The complete documentation build in the workflow SDK image exited 0, with 943 warnings; the workflow does not use -W. Supplemental checks in that image passed: Python documentation 102; architecture frontend 17 unit, 8 browser and 10 Sphinx integration tests; Prettier; Black 2411 files; API coverage 4770/4770. The corresponding remote lint/documentation-test jobs use ubuntu-latest and have passed. black . was run before each commit.
  • Regression coverage preserves ordinary-window publication, same-frame camera/recording completion, direct writes, fluid staging and failed-wait recording retry. Existing rendering context describes these boundaries; the test-only compatibility fixes require no additional context change.

A preceding container attempt aborted in the GPU lane with SIGABRT. Kernel logs recorded an existing Nsight process Xid 13 and the pytest process Xid 43 in the same second. The affected CUDA cases passed 60 repetitions on this branch and 60 on main; the complete run above then passed after stopping that profiling session, including the prior interruption point. No Xid occurred in the successful run, although one NVIDIA reference-state warning remained. The native abort's cause is not established; the failed attempt and diagnostics are retained alongside the passing evidence.

Additional native initialization finding: on the shared desktop (DISPLAY=:0), intermittent X11 BadAtom failures reproduce with dexsim.init_sim_engine() alone, without importing EmbodiChain. GDB reaches SDL 3.4.4's X11 event handling before the rendering tests. Disabling the input method or preinitializing Xlib threads did not reliably resolve it. This desktop-specific native failure remains unresolved; the isolated X11 verification above must not be presented as a fix for it. The earlier interpreter-path attribution was not supported by repeated controls. No DexSim binary or production startup workaround is included in this PR.

OfflineRT synchronization checks use explicitly unlit markers. With DexSim 0.6.0rc1, lit RobotMarkers texture previews appear gray through both the previous manual-flush capture route and the new manager route. That existing appearance difference remains unresolved; this change preserves production material behavior.

Asset download reliability

The shared dataset loader now appends the official Hugging Face source for known EmbodiChain asset URLs on hf-mirror.com. A failed mirror download falls through to the same asset path and revision with the original checksum and extraction settings. Custom hosts and unrelated repositories retain their declared sources.

This addresses the asset-download failure in #1154 CI. The identical repair is included in #837, #1137 and #1154. Validation on #1154: 154 data tests passed, including seven fallback cases; the original contact-sensor test passed. An empty-cache download reproduced the mirror SSL failure and then downloaded, verified and extracted the asset from the official source. The data-assets context documents this behavior. Full Actions checks for the new commits are pending.

Checklist

  • I have run the black . command to format the code base.
  • I reviewed affected documentation and agent context, updated it where needed, or explained why no update was needed.
  • Public API changes are reflected in the API docs (python docs/scripts/check_api_docs.py), if applicable.
  • I verified appropriate behavioral coverage, reviewed the necessity of added or extended tests, and explained when no new tests were needed.
  • Dependencies have been updated, if applicable (unchanged).

@acrlw acrlw added bug Incorrect existing behavior or a fix restoring the intended behavior. rendering Image generation, lighting, material rendering, native renderers and image quality. visualization Browser viewers, markers, trajectory previews and interactive debugging displays. performance Performance, memory, throughput and synchronization-cost improvements. labels Nov 11, 2026
@yuecideng
yuecideng self-requested a review November 11, 2026 06:39
@yuecideng
yuecideng merged commit a722790 into main Nov 13, 2026
2 of 3 checks passed
@yuecideng
yuecideng deleted the codex/window-render-sync branch November 13, 2026 06:58
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

bug Incorrect existing behavior or a fix restoring the intended behavior. performance Performance, memory, throughput and synchronization-cost improvements. rendering Image generation, lighting, material rendering, native renderers and image quality. visualization Browser viewers, markers, trajectory previews and interactive debugging displays.

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants