Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
58 commits
Select commit Hold shift + click to select a range
4de52af
feat(expansion): add rolling single-world workload execution
yuecideng Nov 14, 2026
2cbe6a5
fix(expansion): own validated metadata inputs
yuecideng Nov 14, 2026
61f922d
fix(sensors): wait for contact sample before permission checks
yuecideng Nov 14, 2026
4723cba
feat(expansion): expose task ports and measured ghost comparison
yuecideng Nov 14, 2026
ab04d97
feat(graspkit): scope pose NMS memory budgets per task
yuecideng Nov 14, 2026
bc5ae3f
perf: reduce single-world planning and row execution overhead
yuecideng Nov 15, 2026
504f44f
docs(expansion): document native ghost preview and WebRTC follow-up
yuecideng Nov 15, 2026
bddf4d7
fix(physics): remove direct GPU API configuration override
yuecideng Nov 15, 2026
24c21fb
feat(expansion): preview confirmed results with native ghost rendering
yuecideng Nov 15, 2026
80d2d51
feat(expansion): collect and preview bounded parameter distributions …
yuecideng Nov 15, 2026
39201fe
fix(motion): preserve planning contexts and phase velocities
yuecideng Nov 15, 2026
85704e1
fix(expansion): preview recorded mesh scenes and native runtime metadata
yuecideng Nov 15, 2026
239a3d5
feat(expansion): compose native single-world workload collection
yuecideng Nov 15, 2026
78dc36c
fix(expansion): isolate workload grasp frames and reset settling
yuecideng Nov 15, 2026
7f97064
feat(expansion): expose declarative collection and preview entrypoints
yuecideng Nov 15, 2026
ed01657
Merge latest main and fix expansion CI regressions
yuecideng Nov 16, 2026
722e385
fix(sim): collect released native views before closing their scene
yuecideng Nov 16, 2026
f3e08ae
fix(expansion): address admission, acceptance and preview review find…
yuecideng Nov 16, 2026
8c244cd
fix(expansion): sync main and preserve placement phase paths
yuecideng Nov 16, 2026
2dfcbed
fix(expansion): stabilize hold timing and joint selection checks
yuecideng Nov 16, 2026
565d981
fix(gensim): clear supports and settle opening before release
yuecideng Nov 16, 2026
4a0f34c
merge: sync main before expansion performance fixes
yuecideng Nov 17, 2026
33fe553
test(gensim): align binding coverage with release clearance defaults
yuecideng Nov 17, 2026
bf4a254
fix(expansion): preserve selected rows and batch measured validation
yuecideng Nov 17, 2026
d083f61
merge: preserve concurrent GenSim binding coverage
yuecideng Nov 17, 2026
c3e6cd1
test(expansion): use native monitors in hold sampling coverage
yuecideng Nov 17, 2026
d887e99
fix(expansion): validate prepared scenes and PickUp path conflicts
yuecideng Nov 17, 2026
cba37a4
fix(expansion): retain reset holds and batch seeded solver paths
yuecideng Nov 17, 2026
4dad707
feat(expansion): reuse idle rows for bounded retry candidates
yuecideng Nov 17, 2026
db388b3
fix(expansion): retain owner-proven holds after terminal runtime
yuecideng Nov 17, 2026
9e29ad6
fix(expansion): keep speculative retry reuse opt-in
yuecideng Nov 17, 2026
86303c7
fix(expansion): compare effective scene preparation defaults
yuecideng Nov 17, 2026
5c557e8
feat(expansion): add workspace pose coverage and distribution reports
yuecideng Nov 18, 2026
910db94
feat(expansion): add bidirectional conditional layout sampling
yuecideng Nov 18, 2026
97493f3
fix(expansion): address point-IK and idle-row review feedback
yuecideng Nov 18, 2026
cfdc9f3
merge: sync main into expansion workloads
yuecideng Nov 18, 2026
5b5a307
docs: refresh scene expansion architecture evidence
yuecideng Nov 18, 2026
0557cdc
feat(expansion): support rolling layouts and measured phase coverage
yuecideng Nov 18, 2026
88c2e18
fix(packaging): include expansion distribution report template
yuecideng Nov 18, 2026
4132d86
docs(expansion): correct distribution API heading
yuecideng Nov 18, 2026
95990b8
merge: sync main into expansion workloads
yuecideng Nov 18, 2026
65f7d5d
feat(expansion): separate nominal scene and proposal seeds
yuecideng Nov 18, 2026
657daec
fix(expansion): stabilize installation identity in offline reports
yuecideng Nov 18, 2026
f18d7f7
fix(expansion): enforce task identity and layout admission contracts
yuecideng Nov 18, 2026
6dd084f
merge: integrate concurrent expansion updates
yuecideng Nov 18, 2026
f38aacb
fix(expansion): preserve admission deadlines for active rollouts
yuecideng Nov 18, 2026
c67c721
Merge remote-tracking branch 'origin/codex/expansion-workloads' into …
yuecideng Nov 18, 2026
93c4e40
feat(expansion): compare trajectory ghosts against a fixed reference
Nov 19, 2026
dff0d4e
fix(expansion): handle decomposed colliders in scene-first screening
Nov 19, 2026
86d9d54
fix(expansion): plot only measured object_pose starting layouts
Nov 19, 2026
f3bcfb2
fix(expansion): fit heatmap axes to measured starting positions
Nov 19, 2026
f266329
fix(expansion): decompose physical colliders and verify bounded suppo…
Nov 19, 2026
6530d9c
docs(expansion): show the verified eight-environment Piper replay
Nov 19, 2026
f00f903
Merge main into the expansion workload visualization branch
Nov 19, 2026
6e53943
fix(cli): resolve packaged environment components in the task catalog
Nov 19, 2026
bbe4804
feat(expansion): use bilingual WebRTC replay by default
yuecideng Nov 19, 2026
76ad5aa
refactor(expansion): group browser assets in a frontend package
yuecideng Nov 19, 2026
c4cd5f7
refactor(expansion): simplify replay and drop tutorial screenshots
yuecideng Nov 19, 2026
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
27 changes: 22 additions & 5 deletions agent_context/topics/expansion-engine/expansion-engine.md
Original file line number Diff line number Diff line change
Expand Up @@ -100,14 +100,31 @@ cross-group frame through the public marker layer. Task assets and any collectio
process orchestration remain outside the replay core.

`embodichain expansion preview --input` reads confirmed job outputs through `_preview_data.py`,
then replays full measured states using `_preview_render.py` and a loopback-only
`_preview_server.py`. Inputs retain their original evidence; no collection or
then replays full measured states using `_preview_render.py` and
`_preview_server.py`. Browser replay defaults to the DexSim WebRTC observer
and its owner-thread application. `_frontend/` owns the packaged preview
HTML, CSS and JavaScript and the offline distribution report template.
Video is separate from the HTTP replay state; authenticated commands queue only
validated replay controls. Inputs retain their original evidence; no collection or
physics stepping occurs. It accepts one URDF robot and recorded rigid visual
objects (including file Mesh/USD via native visual loading), with fixed layouts;
unsupported geometry is rejected. Constant static backgrounds remain single
instances. Space filters group
input directories; missing motion annotations remain explicit. HTTP publishes a
single immutable image/state snapshot; WebRTC is follow-up work. FK and episode
instances. The solid robot and objects always follow the explicit reference,
or the first input recording when a reference is absent. The play button and
Space key toggle playback. Candidate buttons toggle individual ghosts. See the
[UR5 layout and grasp tutorial](../../../scripts/tutorials/expansion/ur5_pick_place/README.md)
for collection and replay. Filters preserve each toggle and playback of the solid
trajectory. Space filters group input directories and missing motion annotations
remain explicit. `_preview_deviations.py` reuses recorded world FK and gripper
calibration for TCP errors. At each reference time it reports the minimum, mean
and maximum across recordings still in range for TCP translation distance,
TCP rotation angle and each episode's worst joint deviation. `_preview_heatmap.py`
bins the measured first-frame XY of each rigid target in an `object_pose` input
episode's `joint_scene.pose_changes`. Each axis fits the object's measured starts
independently with half a bin of padding at either end, or a 1 mm window for a
constant coordinate. The recipe's selected motion operator takes
precedence over the job-level operator. The explicit reference belongs to replay.
Static chart payloads and HTTP replay metadata are independent of the encoded video. FK and episode
budgets bound preload, controls are queued for the render owner, and native
resources/ports close on exit. See the example's `ghost-preview.md` for usage.

Expand Down
10 changes: 10 additions & 0 deletions agent_context/topics/expansion-engine/workloads.md
Original file line number Diff line number Diff line change
Expand Up @@ -228,6 +228,16 @@ preparation/conditional generation and `RobotFeasibilityBackend`/`plan_task_mode
it creates no physics World and owns only the isolated collision-query resources.
`layout` and `task_mode` are explicit, validated task declarations. Root layout
variables currently use XY only; selected nested members retain declared XY/yaw.
Placement checks group bounds on the main support. For a bounded leaf root
overhanging that rectangle, preparation can verify the projected centroid
against contact patches at the authored height on the named support. Full
projected extents still determine clearance to other groups.
All layout-screened operators use native convex shapes and primitives. Physical
triangle colliders on rigid objects and robot links are decomposed in their scaled local frame for the isolated
Newton phase/path query. V-HACD handles volumetric concavities; planar meshes
retain their footprint and holes. `scene-first-checks.json` records part counts
by physical collider in `mesh_convex_parts`. Rollout contacts use the
simulator's original physical shapes.
Homogeneous replicas support rolling W times B collection, including eight rows.
Preparation snapshots the common prototype geometry, installation and nominal
full-joint planning start before any row executes. The feasibility adapter uses
Expand Down
14 changes: 9 additions & 5 deletions agent_context/topics/scene-expansion/feasibility.md
Original file line number Diff line number Diff line change
Expand Up @@ -39,12 +39,16 @@ mimic constraints. Hypothetical whole-robot FK supplies solver-root transforms;
using live roots for projected torso/base states is incorrect. Complete path
screening uses explicit physical geometry and phase contact permissions.
`collision.py` lazily builds a Newton `CollisionPipeline` over isolated query
states. The physical extractor preserves native convex shapes, includes the
global ground, and rejects triangle meshes, SDFs, non-robot articulations and
unsupported robot asset scales/formats. No collider is silently convexified.
states. The standalone adapter uses native convex shapes, primitives and the
global ground. Triangle meshes, SDFs, non-robot articulations and unsupported
robot scales require an explicit adapter. The layout-screened collector cooks
physical triangle colliders on rigid objects and robot links into convex parts.
Permissions and reported failures retain the original physical collider IDs.
Halfspaces use analytic physical-shape support. Phase masks do not mutate live
filters, and the query never advances the simulation clock. Shape/pair/contact
storage is bounded; saturation and nonfinite results remain unverified.
filters, and the query never advances the simulation clock. Explicit pair
lists and phase masks contain only eligible pairs. Batched rows share a
bounded pair/contact budget and run in chunks when needed. Callers can set a
separate shape quota. Saturation and nonfinite results remain unverified.
The host closes the owned query arrays/graphs after every attempt.

Query transforms are rebased before composing local offsets; precision budgets
Expand Down
5 changes: 5 additions & 0 deletions agent_context/topics/scene-expansion/scene-expansion.md
Original file line number Diff line number Diff line change
Expand Up @@ -17,6 +17,11 @@ must also be anchored. Anchored roots reserve space before unanchored groups
are sampled. Invalid domains reject before generation; unsupported
geometry returns budget exhaustion rather than silently moving the anchors.
The source/policy remain reusable; empty conditions preserve the old stream.
A rigid leaf root with an explicit XY offset range may overhang the primary
support rectangle. Preparation verifies a bounded translation range on the
named mesh using faces near the authored bottom, the projected mass centroid
and the convex hull of the contact patches. Fixed neighbors retain their full
projected clearance bounds. Saved layouts revalidate these offsets on load.
`SimulationSceneExpansionHost` prepares/restores ordered rows with independent
validation. Offline scheduling and confirmed persistence belong to the
[Expansion Engine](../expansion-engine/expansion-engine.md).
Expand Down
8 changes: 5 additions & 3 deletions embodichain/cli/_task_catalog.py
Original file line number Diff line number Diff line change
Expand Up @@ -30,6 +30,8 @@
from pathlib import Path, PurePosixPath
from typing import Any

from embodichain.utils.config_paths import resolve_config_path

try:
from importlib.resources.abc import Traversable
except ModuleNotFoundError:
Expand Down Expand Up @@ -316,10 +318,10 @@ def _backend(config: Mapping[str, Any], resource: Traversable) -> str | None:
if isinstance(environment, Mapping) and isinstance(
environment.get("component"), str
):
# Components are normally siblings or ancestors. Path resources support
# this without importing simulator-dependent configuration decoders.
if isinstance(resource, Path):
component = resource.parent / environment["component"]
component = resolve_config_path(
environment["component"], base_dir=resource.parent
)
if not component.is_file():
raise ValueError(f"Missing environment component: {component}")
return _load_mapping(component).get("physics")
Expand Down
10 changes: 8 additions & 2 deletions embodichain/cli/expansion.py
Original file line number Diff line number Diff line change
Expand Up @@ -82,7 +82,10 @@ def main(argv: Sequence[str] | None = None) -> int:
)
command.add_argument(
"--reference",
help="Optional result directory containing exactly one reference episode",
help=(
"Job directory containing one confirmed original trajectory for solid "
"playback. Defaults to the first loaded input recording as a comparison."
),
)
command.add_argument(
"--robot-asset", help="Matching URDF override if recorded asset paths moved"
Expand All @@ -93,7 +96,10 @@ def main(argv: Sequence[str] | None = None) -> int:
)
command.add_argument("--gpu", type=int, default=0)
command.add_argument(
"--port", type=int, default=8769, help="Loopback port; 0 selects a free port"
"--port",
type=int,
default=8769,
help="Browser preview port; 0 selects an available port",
)
command.add_argument("--max-episodes", type=int, default=32)
command.add_argument(
Expand Down
27 changes: 20 additions & 7 deletions embodichain/compute/geometry/_hierarchical.py
Original file line number Diff line number Diff line change
Expand Up @@ -213,6 +213,7 @@ def _sample_hierarchy(
movable_roots: tuple[int, ...] | None = None,
root_offset_bounds: dict[int, torch.Tensor] | None = None,
anchored_roots: tuple[int, ...] = (),
root_support_offsets: dict[int, torch.Tensor] | None = None,
) -> tuple[torch.Tensor, torch.Tensor, torch.Tensor]:
"""One proposal per ordinal, with no per-member retry loops."""
local = [matrix.expand(count, 4, 4).clone() for matrix in relative]
Expand Down Expand Up @@ -300,6 +301,7 @@ def _sample_hierarchy(
world.append(
local[node] if parent < 0 else _matrix_product(world[parent], local[node])
)
root_support_offsets = root_support_offsets or {}
root_boxes, deltas = [], {}
fixed_boxes = {}
if randomize_groups and movable_roots is not None:
Expand All @@ -316,9 +318,13 @@ def _sample_hierarchy(
box = torch.stack(
(points[:, :, :2].amin(1), points[:, :, :2].amax(1)), dim=1
)
valid &= (box[:, 0] >= table[0] - 1e-10).all(-1) & (
box[:, 1] <= table[1] + 1e-10
).all(-1)
if root in root_support_offsets:
low, high = root_support_offsets[root]
valid &= (low <= 0).all() & (high >= 0).all()
else:
valid &= (box[:, 0] >= table[0] - 1e-10).all(-1) & (
box[:, 1] <= table[1] + 1e-10
).all(-1)
for other in root_boxes:
valid &= (
torch.maximum(
Expand Down Expand Up @@ -354,7 +360,10 @@ def _sample_hierarchy(
key=key, start=start, count=count, width=3, device=relative.device
)
half = (box[:, 1] - box[:, 0]) / 2
center_region = torch.stack((table[0] + half, table[1] - half), dim=1)
if root in root_support_offsets:
center_region = box.mean(1)[:, None] + root_support_offsets[root]
else:
center_region = torch.stack((table[0] + half, table[1] - half), dim=1)
if root_offset_bounds and root in root_offset_bounds:
offset = root_offset_bounds[root]
center_region = torch.stack(
Expand Down Expand Up @@ -392,9 +401,13 @@ def _sample_hierarchy(
(count, 2), dtype=relative.dtype, device=relative.device
)
box = box + delta[:, None, :]
valid &= (box[:, 0] >= table[0] - 1e-10).all(-1) & (
box[:, 1] <= table[1] + 1e-10
).all(-1)
if root in root_support_offsets:
low, high = root_support_offsets[root]
valid &= (delta >= low - 1e-10).all(-1) & (delta <= high + 1e-10).all(-1)
else:
valid &= (box[:, 0] >= table[0] - 1e-10).all(-1) & (
box[:, 1] <= table[1] + 1e-10
).all(-1)
for other in root_boxes:
valid &= (
torch.maximum(box[:, 0] - other[:, 1], other[:, 0] - box[:, 1]).amax(-1)
Expand Down
12 changes: 8 additions & 4 deletions embodichain/compute/geometry/_warp/batch_contact_query.py
Original file line number Diff line number Diff line change
Expand Up @@ -21,6 +21,8 @@

import warp as wp

from .contact_query import _pair_slot

__all__: list[str] = []


Expand All @@ -36,6 +38,7 @@ def contact_summary(
margin1: wp.array(dtype=float),
shape_body: wp.array(dtype=wp.int32),
poses: wp.array(dtype=wp.transform),
pair_keys: wp.array(dtype=wp.int64),
active: wp.array(dtype=wp.int32),
width: int,
batches: int,
Expand Down Expand Up @@ -74,7 +77,10 @@ def contact_summary(
else:
row = a // width
x, y = a % width, b % width
if active[x * width + y] != 0:
slot = _pair_slot(pair_keys, width, x, y)
if slot < 0:
wp.atomic_min(output, row, 2, 0.0)
elif active[slot] != 0:
p0 = wp.transform_point(poses[shape_body[a]], point0[index])
p1 = wp.transform_point(poses[shape_body[b]], point1[index])
distance = (
Expand All @@ -85,6 +91,4 @@ def contact_summary(
else:
wp.atomic_min(output, row, 0, distance)
if distance <= required + tolerance[row]:
wp.atomic_min(
output, row, 1, float(wp.min(x, y) * width + wp.max(x, y))
)
wp.atomic_min(output, row, 1, float(slot))
21 changes: 17 additions & 4 deletions embodichain/compute/geometry/_warp/contact_query.py
Original file line number Diff line number Diff line change
Expand Up @@ -23,6 +23,16 @@
__all__ = []


@wp.func
def _pair_slot(keys: wp.array(dtype=wp.int64), width: int, a: int, b: int):
key = wp.int64(wp.min(a, b)) * wp.int64(width) + wp.int64(wp.max(a, b))
slot = wp.lower_bound(keys, key)
if slot < keys.shape[0]:
if keys[slot] == key:
return slot
return int(-1)


@wp.kernel(enable_backward=False)
def _contact_summary(
count: wp.array(dtype=wp.int32),
Expand All @@ -35,6 +45,7 @@ def _contact_summary(
margin1: wp.array(dtype=float),
shape_body: wp.array(dtype=wp.int32),
poses: wp.array(dtype=wp.transform),
pair_keys: wp.array(dtype=wp.int64),
active: wp.array(dtype=wp.int32),
shapes: int,
required: float,
Expand Down Expand Up @@ -65,7 +76,11 @@ def _contact_summary(
a, b = first[index], second[index]
if a < 0 or b < 0 or a >= shapes or b >= shapes:
wp.atomic_min(output, 3, 0.0)
elif active[a * shapes + b] != 0:
return
slot = _pair_slot(pair_keys, shapes, a, b)
if slot < 0:
wp.atomic_min(output, 3, 0.0)
elif active[slot] != 0:
p0 = wp.vec3(point0[index][0], point0[index][1], point0[index][2])
p1 = wp.vec3(point1[index][0], point1[index][1], point1[index][2])
owner0, owner1 = shape_body[a], shape_body[b]
Expand All @@ -79,6 +94,4 @@ def _contact_summary(
else:
wp.atomic_min(output, 0, distance)
if distance <= required + tolerance[0]:
wp.atomic_min(
output, 1, float(wp.min(a, b) * shapes + wp.max(a, b))
)
wp.atomic_min(output, 1, float(slot))
7 changes: 6 additions & 1 deletion embodichain/lab/expansion/_distribution.py
Original file line number Diff line number Diff line change
Expand Up @@ -26,11 +26,14 @@
from collections.abc import Mapping
from copy import deepcopy
import hashlib
from importlib.resources import files
import json
from pathlib import Path
from typing import Any
import numpy as np

from . import _frontend

__all__: list[str] = []


Expand Down Expand Up @@ -345,7 +348,9 @@ def _run(
(output / "summary.json").write_text(
json.dumps(report, ensure_ascii=False, indent=2, allow_nan=False)
)
template = Path(__file__).with_name("_distribution.html").read_text()
template = (
files(_frontend).joinpath("distribution.html").read_text(encoding="utf-8")
)
(output / "report.html").write_text(
template.replace(
"__REPORT__",
Expand Down
49 changes: 49 additions & 0 deletions embodichain/lab/expansion/_frontend/__init__.py
Original file line number Diff line number Diff line change
@@ -0,0 +1,49 @@
# ----------------------------------------------------------------------------
# Copyright (c) 2021-2026 DexForce Technology Co., Ltd.
#
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.
# ----------------------------------------------------------------------------

"""Packaged browser assets for expansion playback and distribution reports."""

from __future__ import annotations

from importlib.resources import files

__all__: list[str] = []


def _preview_assets() -> dict[str, tuple[str, bytes]]:
"""Load same-origin assets for the native WebRTC preview."""
directory = files(__name__)
assets = {
"/": (
"text/html; charset=utf-8",
directory.joinpath("preview.html").read_bytes(),
),
"/preview.css": (
"text/css; charset=utf-8",
directory.joinpath("preview.css").read_bytes(),
),
}
for name in (
"preview-text.js",
"preview-controls.js",
"preview-charts.js",
"preview-heatmap.js",
):
assets[f"/{name}"] = (
"text/javascript; charset=utf-8",
directory.joinpath(name).read_bytes(),
)
return assets
Loading
Loading